Wednesday, February 5, 2025 2/5/2025

Natural audiovisual speech encoding in the early stages of the human cortical hierarchy

Award Number: R01DC016297
ORGANIZATION: NATIONAL INSTITUTE ON DEAFNESS AND OTHER COMMUNICATION DISORDERS
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

Group Awards By:

View Award Abstract

PROJECT SUMMARY Speech is central to human life. Yet how the human brain processes speech in complex everyday situations remains poorly understood. One prominent idea is that speech perception is carried out using brain areas and mechanisms that are used for processing sounds more generally. And it has been suggested that these mechanisms become specialized for speech through learning, resulting in a speech processing network in the brain that processes increasingly complex aspects of the speech signal at successive hierarchical stages. But questions about the function of this hierarchy remain. In particular, while it is commonly acknowledged that seeing a speaker’s face in noisy environments can improve comprehension, our understanding of how visual speech influences the hierarchical processing of speech remain unclear. This is unfortunate as speech processing, and multisensory speech processing in particular, have been reported to be affected in a number of clinical disorders, including autism and schizophrenia. Thus, as well as contributing to our understanding of this most fundamental of human abilities, better knowledge of the neural mechanisms underpinning audiovisual speech processing could have important clinical research implications. One of the principal reasons for our lack of knowledge on the neurophysiology of audiovisual speech is the technical challenge associated with indexing the neural processing of natural speech with high temporal resolution and at multiple levels of the speech processing hierarchy. Non-human primates represent a less than perfect model for studying human speech processing, the hemodynamic changes underlying functional magnetic resonance imaging are too slow to track natural speech dynamics, and electrocorticography samples only a limited number of brain areas and cannot be broadly applied in clinical research. Recently, our group has introduced several new approaches for indexing natural speech processing using electroencephalography (EEG). These include entirely novel frameworks for producing dependent measures of the hierachical encoding of natural speech, and for quantifying multisensory integration of natural audiovisual speech. The present proposal seeks to exploit this opportunity to test the hypothesis that the integration of audio and visual speech is a flexible, multistage process that adapts to optimize comprehension based on the current listening conditions. Across three objectives the proposal aims to characterize this flexibility by determining how the hierarchical processing stage at which visual and audio speech are integrated varies as a function of 1) the listening environment, 2) the visual information available and 3) the deployment of attention. The work promises to bring a new depth of understanding to the perception of one of humanity’s most essential signals. And it will introduce several novel analyses and experimental paradigms that should be easily deployable in tackling research on clinical cohorts in which speech processing and/or multisensory integration is impaired.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2025 ( Subtotal = $471,790 )
2025	2025	UNIVERSITY OF ROCHESTER	910 GENESEE ST	ROCHESTER	NY	14611	MONROE	USA	Research Related to Deafness and Communication Disorders	000	6	11/27/2024	COMPETING CONTINUATION	$471,790
														Subtotal = $471,790

Issue Date FY: 2024 ( Subtotal = -$782 )
2024	2022	UNIVERSITY OF ROCHESTER	910 GENESEE ST	ROCHESTER	NY	14611	MONROE	USA	Research Related to Deafness and Communication Disorders	000	5	6/18/2024	NON-COMPETING CONTINUATION	-$782
														Subtotal = -$782

Issue Date FY: 2023 ( Subtotal = $0 )
2023	2022	UNIVERSITY OF ROCHESTER	910 GENESEE ST	ROCHESTER	NY	14611	MONROE	USA	Research Related to Deafness and Communication Disorders	000	5	5/31/2023	NON-COMPETING CONTINUATION	$0
														Subtotal = $0

Issue Date FY: 2022 ( Subtotal = $385,000 )
2022	2022	UNIVERSITY OF ROCHESTER	500 JOSEPH C WILSON BLVD	ROCHESTER	NY	14627	MONROE	USA	Research Related to Deafness and Communication Disorders	001	5	5/3/2022	NON-COMPETING CONTINUATION	$38,500
2022	2022	UNIVERSITY OF ROCHESTER	500 JOSEPH C WILSON BLVD	ROCHESTER	NY	14627	MONROE	USA	Research Related to Deafness and Communication Disorders	000	5	4/20/2022	NON-COMPETING CONTINUATION	$346,500
														Subtotal = $385,000

Issue Date FY: 2021 ( Subtotal = $385,000 )
2021	2021	UNIVERSITY OF ROCHESTER	500 JOSEPH C WILSON BLVD	ROCHESTER	NY	14627	MONROE	USA	Research Related to Deafness and Communication Disorders	000	4	2/22/2021	NON-COMPETING CONTINUATION	$385,000
														Subtotal = $385,000

Issue Date FY: 2020 ( Subtotal = $385,000 )
2020	2020	UNIVERSITY OF ROCHESTER	500 JOSEPH C WILSON BLVD	ROCHESTER	NY	14627	MONROE	USA	Research Related to Deafness and Communication Disorders	000	3	2/19/2020	NON-COMPETING CONTINUATION	$385,000
														Subtotal = $385,000

Issue Date FY: 2019 ( Subtotal = $385,000 )
2019	2019	UNIVERSITY OF ROCHESTER	500 JOSEPH C WILSON BLVD	ROCHESTER	NY	14627	MONROE	USA	Research Related to Deafness and Communication Disorders	000	2	4/17/2019	NON-COMPETING CONTINUATION	$385,000
														Subtotal = $385,000

Issue Date FY: 2018 ( Subtotal = $377,440 )
2018	2018	UNIVERSITY OF ROCHESTER	500 JOSEPH C WILSON BLVD	ROCHESTER	NY	14627	MONROE	USA	Research Related to Deafness and Communication Disorders	000	1	3/12/2018	NEW	$377,440
														Subtotal = $377,440

Grand Total All Awards = $2,388,448

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Natural audiovisual speech encoding in the early stages of the human cortical hierarchy

Award Number: R01DC016297

ORGANIZATION: NATIONAL INSTITUTE ON DEAFNESS AND OTHER COMMUNICATION DISORDERS

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer