Date of Award

Fall 12-10-2019

Author's School

Graduate School of Arts and Sciences

Author's Department

Psychology

Degree Name

Master of Arts (AM/MA)

Degree Type

Thesis

Abstract

Speech perception improves when listeners are able to see as well as hear a talker, compared to listening alone. This phenomenon is commonly referred to as audiovisual (AV) benefit (Sommers et al., 2005). According to the Principle of Inverse Effectiveness (PoIE), the benefit of multimodal (e.g. audiovisual) input should increase as unimodal (e.g. auditory-only) stimulus clarity decreases. However, recent findings contradict the PoIE, indicating that it should be reassessed. One method for investigating the factors that contribute to AV speech benefit is to examine listeners’ gaze behavior with eye tracking. The present study compared young adults’ (N=50) gaze behavior during AV speech presentations across a range of signal-to-noise ratios in order to determine the relationship between speech-to-noise ratio, gaze behavior, and audiovisual benefit. Participants completed the Build-A-Sentence (BAS) Test, a closed-set test in which participants are asked to identify 3 target words in sentences. Stimuli were presented in auditory-only and audiovisual conditions across four speech-to-noise ratios. Findings were considered from the perspective of the PoIE, which predicts that participants’ AV benefit will increase as the auditory signal becomes less intelligible. Additionally, participants’ rank order of AV benefit relative to other participants’ was compared across speech-to-noise ratios in order to examine individual differences. Participants’ AV benefit was consistent with the PoIE, such that AV benefit increased as auditory-only intelligibility decreased. Additionally, participants increased the amount of time spent fixating on the talker’s mouth as speech-to-noise ratio decreased. However, gaze behavior was not a significant predictor of audiovisual benefit, and differences between participants’ AV benefit were inconsistent across speech-to-noise ratios. These findings have important implications for research on factors contributing to AV benefit and individual differences in AV benefit.

Language

English (en)

Chair and Committee

Mitchell Sommers

Committee Members

Kristin Van Engen, Jonathan Peelle

Comments

Permanent URL: https://doi.org/10.7936/5xkr-ky68

Download

Share

COinS