Speaker
Description
This study investigates listening effort in audiovisual speech perception, with a focus on the use of virtual avatars as an alternative to real speakers. An audiovisual version of the Italian Matrix Sentence Test was developed, together with a dedicated toolbox (SPiNE - SPeech in Noise & Listening Effort Assessment) for administering speech intelligibility and listening effort measurements. The audiovisual material was created using recordings of a real speaker and a corresponding avatar generated through audio-driven facial animation within Unreal Engine. Two experiments were designed to compare speech intelligibility and listening effort between real and avatar-based stimuli under controlled conditions. The experiments included different types of background noise and signal-to-noise ratios. Listening effort was assessed through both behavioral (i.e., response time) and subjective measures, alongside speech intelligibility metrics. The study aims to provide an ecologically valid methodology for audiovisual speech testing. Indeed, by enabling precise control over visual speech cues, virtual avatars represent a promising tool for advancing research on listening effort and for supporting future clinical applications.