Speaker
Description
Listening attentively and remembering spoken information are essential for many everyday life communication scenarios. We rely on these cognitive abilities when engaging in conversations, attending lectures, or following political debates. Prior research has examined how acoustic and audiovisual factors influence listeners’ ability to retain heard information (content memory), but far less is known about their impact on content-to-talker attribution (source memory). This Virtual Reality study investigated the effects of steady-state background noise and gestural turn-taking cues on listeners’ content and source memory in conversations focusing on audiovisual aspects. Participants were immersed in a simplified virtual laboratory environment in which they listened to dialogues between two embodied conversational agents (ECAs), one male and one female, positioned at ±60° azimuth relative to the listener. After each dialogue, participants answered content- and source-memory questions. In a within-subject design, dialogues were presented either in silence or in white noise at –3 dB SNR, and either with or without gestural turn-taking cues. Regarding turn-taking gestures, we explored two competing possibilities: whether they improve memory by facilitating audiovisual integration or impair it due to additional processing demands. Background noise was expected to impair both content and source memory, with potential mitigation of these effects in the presence of turn-taking cues. Results of this study will be presented and discussed with regards to their practical implications for real-life listening situations as well as their theoretical relevance in auditory cognition research.