Investigating the Relationship Between Gaze Behavior and Audiovisual Benefit Across Various Speech-to-Noise Ratios
Date of Award
Master of Arts (AM/MA)
Speech perception improves when listeners are able to see as well as hear a talker, compared to listening alone. This phenomenon is commonly referred to as audiovisual (AV) benefit (Sommers et al., 2005). According to the Principle of Inverse Effectiveness (PoIE), the benefit of multimodal (e.g. audiovisual) input should increase as unimodal (e.g. auditory-only) stimulus clarity decreases. However, recent findings contradict the PoIE, indicating that it should be reassessed. One method for investigating the factors that contribute to AV speech benefit is to examine listeners’ gaze behavior with eye tracking. The present study compared young adults’ (N=50) gaze behavior during AV speech presentations across a range of signal-to-noise ratios in order to determine the relationship between speech-to-noise ratio, gaze behavior, and audiovisual benefit. Participants completed the Build-A-Sentence (BAS) Test, a closed-set test in which participants are asked to identify 3 target words in sentences. Stimuli were presented in auditory-only and audiovisual conditions across four speech-to-noise ratios. Findings were considered from the perspective of the PoIE, which predicts that participants’ AV benefit will increase as the auditory signal becomes less intelligible. Additionally, participants’ rank order of AV benefit relative to other participants’ was compared across speech-to-noise ratios in order to examine individual differences. Participants’ AV benefit was consistent with the PoIE, such that AV benefit increased as auditory-only intelligibility decreased. Additionally, participants increased the amount of time spent fixating on the talker’s mouth as speech-to-noise ratio decreased. However, gaze behavior was not a significant predictor of audiovisual benefit, and differences between participants’ AV benefit were inconsistent across speech-to-noise ratios. These findings have important implications for research on factors contributing to AV benefit and individual differences in AV benefit.
Chair and Committee
Kristin Van Engen, Jonathan Peelle
Gaunt, Lauren, "Investigating the Relationship Between Gaze Behavior and Audiovisual Benefit Across Various Speech-to-Noise Ratios" (2019). Arts & Sciences Electronic Theses and Dissertations. 1975.
Permanent URL: https://doi.org/10.7936/5xkr-ky68