Sunday, May 11, 2025 5/11/2025

Multimodal communication and cognition: The role of gesture in language processing and word learning in individuals with traumatic brain injury

Award Number: F31DC020388
ORGANIZATION: NATIONAL INSTITUTE ON DEAFNESS AND OTHER COMMUNICATION DISORDERS
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: FELLOWSHIP/SCHOLARSHIP/STUDENT LOANS

Group Awards By:

View Award Description

Multimodal communication and cognition: The role of gesture in language processing and word learning in individuals with traumatic brain injury - PROJECT SUMMARY Communication is multimodal, containing speech and gesture. When people talk, co-speech gestures (spontaneous movements of the hands and arms) can visually depict information conveyed in speech but often communicate unique information not conveyed in speech. For example, a speaker might say, “I searched for a new recipe,” while making a typing gesture, conveying only in gesture that the speaker searched online rather than through a cookbook. Listeners must bind linguistic information from speech and visuospatial information from gesture to generate an integrated representation of a message. The benefits of gesture for communication and cognition are well-documented in neurotypical individuals. For example, gesture improves comprehension and memory for spoken information and facilitates word learning, abilities critical for academic and vocational success. However, gesture has not yet received the same attention in clinical populations with cognitive-communication disorders, such as traumatic brain injury (TBI). In this proposal, we examine whether the benefits of gesture extend to individuals with TBI, or if the very nature of their deficits prevent gesture’s facilitatory role in communication and cognition. Using a novel approach combining methods and theory from speech-language pathology, gesture studies, psycholinguistics, and neuropsychology, we test the ability of individuals with TBI to use gesture during multimodal language processing and word learning across three experiments. Aim 1 (Experiments 1 and 2) investigates the ability of individuals with TBI to integrate information from speech and gesture during multimodal language processing. Experiment 1 tests the effect of observing a narrator’s gestures on subsequent retellings of stories to determine whether individuals with TBI report information provided uniquely in gesture and integrate it into their representation of the stories across time. Experiment 2 uses eye-tracking to determine if individuals with TBI can use information from gesture to resolve referential ambiguity in a visual-world paradigm during rapid on-line language processing. Aim 2 (Experiment 3) investigates whether individuals with TBI benefit from observing and producing gesture during word learning. Exploratory Aim 3 examines the relation between speech-gesture integration and working memory abilities to explore individual differences in gesture processing and inform future confirmatory studies. Studying gesture along with speech is critical for providing ecologically valid assessments of language that more closely approximate the real-world communication contexts that characterize and enrich everyday life. The proposed research will directly advance the study of gesture in clinical populations by providing new insight into the ability to integrate speech and gesture in the context of multimodal language processing and testing whether gesture can be leveraged to support new learning in individuals with TBI. This research represents a new direction in TBI research and promises to offer novel and impactful contributions to theories of multimodal communication and to the nature of cognitive-communication deficits after TBI.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2024 ( Subtotal = -$5,426 )
2024	2023	VANDERBILT UNIVERSITY	110 21ST AVE S	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	000	2	6/24/2024	NON-COMPETING CONTINUATION	-$5,426
														Subtotal = -$5,426

Issue Date FY: 2023 ( Subtotal = $21,937 )
2023	2023	VANDERBILT UNIVERSITY	110 21ST AVE S	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	002	2	8/17/2023	NON-COMPETING CONTINUATION	-$10,580
2023	2023	VANDERBILT UNIVERSITY	110 21ST AVE S	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	000	2	12/8/2022	NON-COMPETING CONTINUATION	$31,575
2023	2023	VANDERBILT UNIVERSITY	110 21ST AVE S	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	001	2	2/5/2023	NON-COMPETING CONTINUATION	$942
														Subtotal = $21,937

Issue Date FY: 2022 ( Subtotal = $31,575 )
2022	2022	VANDERBILT UNIVERSITY, THE	110 21ST AVENUE S STE 800	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	003	1	5/19/2022	NEW	$0
2022	2022	VANDERBILT UNIVERSITY, THE	110 21ST AVENUE S STE 800	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	000	1	12/16/2021	NEW	$30,859
2022	2022	VANDERBILT UNIVERSITY, THE	110 21ST AVENUE S STE 800	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	001	1	4/25/2022	NEW	$717
2022	2022	VANDERBILT UNIVERSITY, THE	110 21ST AVENUE S STE 800	NASHVILLE	TN	37203	DAVIDSON	USA	Research Related to Deafness and Communication Disorders	002	1	5/19/2022	NEW	-$1
														Subtotal = $31,575

Grand Total All Awards = $48,086

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Multimodal communication and cognition: The role of gesture in language processing and word learning in individuals with traumatic brain injury

Award Number: F31DC020388

ORGANIZATION: NATIONAL INSTITUTE ON DEAFNESS AND OTHER COMMUNICATION DISORDERS

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: FELLOWSHIP/SCHOLARSHIP/STUDENT LOANS

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer