Loading...
APPLYING MACHINE LEARNING TO ANALYZE INFLUENCERS' MULTIMODAL (NONVERBAL AND VERBAL) BEHAVIORS IN LIVESTREAMS
Citations
Altmetric:
Genre
Thesis/Dissertation
Date
2025-06
Advisor
Committee member
Group
Department
Business Administration/Marketing
Permanent link to this record
Collections
Files
Kou_temple_0225E_16233.pdf
Adobe PDF, 7.11 MB
- Embargoed until 2026-09-02
Research Projects
Organizational Units
Journal Issue
DOI
https://doi.org/10.34944/bmwc-tm17
Abstract
Influencers’ persuasion content in livestream videos is multimodal, comprising both verbal and nonverbal communication. However, prior research in influencer video marketing only focuses on influencers’ unimodal content, such as their language, facial expressions, or acoustic signals. There has been limited research on modeling and interpreting how influencers’ multimodal behaviors contribute to their sales presentations. My dissertation addresses this gap by applying advanced machine learning techniques to analyze influencers’ multimodal behaviors extracted from livestream videos, aiming to uncover patterns that explain the effectiveness of influencer sales presentations.In the first essay, “Modeling Influencers’ Multimodal Selling Behaviors”, I develop a machine learning pipeline to evaluate the effectiveness of multimodal selling behaviors. Initially, I randomly collect over 11,000 selling videos from TikTok influencers and preprocess them to separate the verbal, vocal, and visual modalities. Subsequently, I extract behavioral variables from these modalities using machine learning tools. I then employ XGBoost, trained on these multimodal behavior variables, to predict sales outcomes. Finally, to elucidate the contribution of these multimodal behaviors to sales, I utilize the SHAP framework for detailed interpretation. The results show that a distribution pattern of 40% verbal, 20% vocal, and 40% visual behaviors reveals the power of influencers’ multimodal selling. This pattern displays the heterogeneity across the follower tiers (mid-tier, macro, and mega influencers) and across consumption types (hedonic vs. utilitarian).
In the second essay, “Mining Creativity in Sales Presentation Videos in Live Commerce”, I focus on using deep learning methods to predict and interpret influencers’ creative selling performance. I design 2 studies. Study 1 is at the stage of prediction. I customize a cutting-edge multimodal deep learning approach, the Multimodal Transformer (MulT) as proposed by Tsai et al. (2019), in my live-streaming dataset for the purpose of predicting influencer creative selling performance. The reason that I employ this model is that it could rigorously treat the concept of creative selling as a multimodal interactive behavior, aligning closely with the concept's definition. Study 2 is at the stage of interpretation. First, I utilize another cutting-edge deep learning model, Multimodal Routing (MulT Routing), as proposed by Tsai et al. (2020). This model is specifically designed to open the black box architecture of MulT and interpret the prediction results. This interpretation could help me figure out the relative importance of each explanatory behavioral component contributing to the predicted creative selling performance. Second, I extract behavioral indicators of human creativity from influencers’ verbal, vocal and visual behavioral modalities and study the correlation between these behavioral indicators and the creative selling performance predicted by the deep learning model. The results provide validity for my deep learning predictions and contribute more interpretability of creative selling.
Overall, the findings from my dissertation have the potential to assist influencers in enhancing their sales presentation skills and contribute to the AI-driven management revolution on live commerce platforms.
Description
Citation
Citation to related work
Has part
ADA compliance
For Americans with Disabilities Act (ADA) accommodation, including help with reading this content, please contact scholarshare@temple.edu
Embedded videos
License
IN COPYRIGHT- This Rights Statement can be used for an Item that is in copyright. Using this statement implies that the organization making this Item available has determined that the Item is in copyright and either is the rights-holder, has obtained permission from the rights-holder(s) to make their Work(s) available, or makes the Item available under an exception or limitation to copyright (including Fair Use) that entitles it to make the Item available.