Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

A computer vision toolbox for computational analysis of nonverbal social communication

Award Number: R01MH122599
ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 09/01/2020
PERIOD OF PERFORMANCE END DATE: 06/30/2031

Group Awards By:

View Award Description

A computer vision toolbox for computational analysis of nonverbal social communication - PROJECT SUMMARY We will develop a computational toolbox to quantify multimodal, multi-person social behavior and validate it for use in behavioral and clinical research. Significance: Difficulties in social communication are transdiagnostic and associated with significant impairment. However, reliable and valid tools to capture and quantify complex multimodal and interpersonal social behavior are still lacking. Advances in computer vision, natural language processing, and machine learning offer new tools to capture human social behavior with greater scalability, granularity, precision, and richness, alleviating many limitations of traditional assessments. Yet, no computational assessment tools are readily available to researchers. Approach: Under our initial NIH funding (R01MH122599), we developed an initial version of a toolbox for advanced computational behavior analysis tailored for behavioral scientists. With this competitive renewal, we aim to enhance our toolbox and address three key gaps: (1) measuring intrapersonal integration of multimodal social cues, (2) measuring interpersonal dynamics, and (3) validation for psychiatric applications. Previous Support: We developed novel methods that led to 14 publications and the initial release of the toolbox. We also exceeded the original aims by conducting preliminary tests in mental health contexts, demonstrating the toolbox’s readiness for rigorous validation in a larger clinical sample and further refinement to better capture multimodal and interpersonal aspects of social communication. Aims. In Aim 1, we will develop tools to automatically measure the integration of multimodal communicative cues within a person; in Aim 2, we will develop tools to quantify interpersonal dynamics of social communication, including coordination of multimodal cues across people; in Aim 3, we will perform an initial assessment of the clinical utility of our computational measures to enhance their clinical relevance. Impact: Successful completion of this proposal will provide researchers with access to advanced computational tools for fast and rigorous assessment of social behavior, providing a scalable solution to measure individual variability within a dimensional and transdiagnostic framework.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $846,660 )
2026	2026	THE CHILDREN'S HOSPITAL OF PHILADELPHIA	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research - Research Projects	002	6	9/1/2026	COMPETING CONTINUATION	$846,660
2026	2024	THE CHILDREN'S HOSPITAL OF PHILADELPHIA	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	5	3/2/2026	NON-COMPETING CONTINUATION	$0
2026	2024	THE CHILDREN'S HOSPITAL OF PHILADELPHIA	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	001	5	8/28/2026	NON-COMPETING CONTINUATION	$0
														Subtotal = $846,660

Issue Date FY: 2024 ( Subtotal = $835,771 )
2024	2024	THE CHILDREN'S HOSPITAL OF PHILADELPHIA	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	5	2/29/2024	NON-COMPETING CONTINUATION	$752,194
2024	2024	THE CHILDREN'S HOSPITAL OF PHILADELPHIA	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	001	5	8/14/2024	NON-COMPETING CONTINUATION	$83,577
														Subtotal = $835,771

Issue Date FY: 2023 ( Subtotal = $822,518 )
2023	2023	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	4	3/16/2023	NON-COMPETING CONTINUATION	$822,518
														Subtotal = $822,518

Issue Date FY: 2022 ( Subtotal = $656,918 )
2022	2022	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	3	2/25/2022	NON-COMPETING CONTINUATION	$656,918
														Subtotal = $656,918

Issue Date FY: 2021 ( Subtotal = $660,945 )
2021	2021	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	2	3/10/2021	NON-COMPETING CONTINUATION	$660,945
														Subtotal = $660,945

Issue Date FY: 2020 ( Subtotal = $705,169 )
2020	2020	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	1	4/24/2020	NEW	$705,169
														Subtotal = $705,169

Grand Total All Awards = $4,527,981

Top

All Categories

About

Search

Reports

Data Submission

Award Information

A computer vision toolbox for computational analysis of nonverbal social communication

Award Number: R01MH122599

ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 09/01/2020

PERIOD OF PERFORMANCE END DATE: 06/30/2031

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer