Abstract
Momentary affect intensity refers to the momentary strength or intensity of emotional experience and shows wide associations with critical psychological and behavioral phenomena. Traditional self-report approaches to assess affect intensity require people’s attention and are prone to missingness and recall bias. Wide and cheap smartphone access offers opportunities to monitor and understand real-time affect intensity. This study had three specific aims: (a) To evaluate the potential for a state-of-the-art explainable artificial intelligence (AI) algorithm to predict momentary affect intensities without active inputs, we tested how well Temporal Fusion Transformers (TFTs) can predict momentary affect intensity in an unseen test sample; (b) to explore how model performances and important predictors differ across different time resolutions to aggregated smartphone sensor data, and (c) to explore whether incorporating self-reported semantic location (e.g., home, leisure places) would improve the performances or interpretation of models. We collected data from community adults (N = 179) who completed a 14-day experience sampling method (ESM) protocol of self-reported emotional experiences and continuous monitoring of behaviors through smartphones. The final analytic sample (N = 102) was those who completed at least 40 ESM surveys (out of 70 possible ones). We created a test set (n = 408 surveys) by extracting the last four surveys for each individual, with the rest of the data being the training set (n = 5264 surveys). On the training set, we performed twelve TFTs, with six time resolutions for NA and six for PA, and revealed important predictors in the models. Results showed that in the unseen test set, the best model explained 40.7% of the variance in NA intensity and 38.0% of the variance in PA intensity. The optimal aggregation time periods were three to six hours. Across time resolutions, we found that frequencies of smartphone social interactions (i.e., number of outgoing calls, number of correspondents called) and a mobility marker (i.e., total distance traveled) were consistently identified as top predictors for momentary NA and PA. Incorporating self-report locations was not associated with significant improvement in model performance, but was associated with the pattern of important predictors. Results suggested that TFTs could make moderate predictions of future momentary affect without active input. Aggregating sensor data in three to six hours may be sufficient. Finally, self-reported locations may help refine digital behavioral markers to inform intervention strategies and our understanding of affect-behavior relationships. Overall, the study highlighted the potential of continuous monitoring of affect intensity after a short period of data collection and the potential benefits of explainable AI for both psychological theories and interventions.
Committee Chair
Renee J. Thompson
Committee Members
Ellen E. Fitzsimmons-Craft Joshua R. Oltmanns
Degree
Master of Arts (AM/MA)
Author's Department
Psychology
Document Type
Thesis
Date of Award
Summer 8-2025
Language
English (en)
Author's ORCID
0000-0002-9456-2375
Recommended Citation
Zhu, Yiqin, "Monitoring and Understanding Momentary Affect Intensity from Smartphone Sensors: Investigating the Role of Time Resolutions and Semantic Locations" (2025). Arts & Sciences Theses and Dissertations. 3638.
https://openscholarship.wustl.edu/art_sci_etds/3638