Date of Award
Fall 12-18-2024
Degree Name
Master of Arts (AM/MA)
Degree Type
Thesis DLA
Abstract
Speech perception is significantly enhanced when auditory signals are complemented by visual information, such as lip movements. This study examines how audiovisual (AV) integration benefits differ between native English speakers and native Mandarin speakers when processing English, and how these benefits compare for native Mandarin speakers in both their first language (L1) and second language (L2). Specifically, we sought to determine whether native Mandarin speakers can achieve the same AV benefits in English as native English speakers, and whether they experience similar levels of AV benefit in Mandarin and English. Additionally, we explored how semantic context interacts with AV integration in both languages.
Participants included 42 native Mandarin speakers and 54 native English speakers. They were presented with words and sentences under both auditory-only (AO) and audiovisual (AV) conditions, with sentences further categorized as either contextually consistent or inconsistent.
For Mandarin speakers, the tasks were completed in both English and Mandarin. For native English participants, testing was performed only in the L1. Participants were asked to identify target words within a noisy background across the different modalities, and performance accuracy was recorded to assess AV benefit.
The findings indicated that native English speakers showed greater AV benefit when processing English compared to native Mandarin speakers. However, for native Mandarin speakers, performance in Mandarin was significantly better than in English, particularly in consistent sentence conditions. These results suggest that predictable syntactic and semantic patterns are crucial for improving speech comprehension, especially under challenging listening conditions. Furthermore, these findings underscore the importance of considering linguistic proficiency, contextual factors, and the role of L1 and L2 differences when assessing AV integration across languages.
Language
English (en)
Chair and Committee
Mitchell Sommers
Committee Members
Kristin Van Engen, Sandra Hale
Recommended Citation
Chen, Hao, "Audio-Visual Benefit Across English and Mandarin" (2024). Arts & Sciences Electronic Theses and Dissertations. 3442.
https://openscholarship.wustl.edu/art_sci_etds/3442