Wednesday, December 4, 2024 12/4/2024

Predicting Autism and Social Functioning from Computer Vision Analyses of Motor Synchrony During Dyadic Interactions

Award Number: R01MH118327
ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

Group Awards By:

View Award Abstract

PROJECT SUMMARY Our innovative and fully automated approach to the analysis of social behavior addresses the pressing need for precise and scalable measurements of the autism spectrum disorder (ASD) phenotype. Using computer vision and machine learning methods, we have created a novel, quantitative method for fine-grained analysis of social interactions. Our approach directly measures interpersonal motor synchrony, a construct which we use as a lens for understanding the social interaction differences that are at the core of ASD. Significance: Genomics and neuroimaging methods continuously evolve, providing deeper insights into the biology of ASD. However, methods for measuring the outward manifestations of ASD have not changed substantially in decades. ASD is fundamentally a disorder of social interaction, but current clinical tools do not directly measure observable social interactions. Instead, they summarize global impressions of these interactions via informant report questionnaires or observational coding schemes that typically lack the behavioral granularity needed to robustly measure individual differences and changes across time (e.g., treatment related change). Inter-rater agreement on questionnaires is typically modest, while the alternative “deep phenotyping” by expert clinicians is a time- consuming and often cost-prohibitive burden to studies, especially when large samples are required (e.g., in genomics research). Approach: To resolve these problems, our team created a novel computational framework that leverages advances in markerless video motion capture, computer vision, and machine learning to directly capture dyadic social interactions. This allows us to capture all behaviors observable by expert clinicians but with exquisite digital precision and objectivity. Preliminary Data: We developed a fully automatic quantitative assessment of interpersonal social behavior focused on features of dyadic facial motor synchrony. When applied to videos of brief conversations between confederates and young adults with or without ASD, our assessment predicted diagnostic status with 91% accuracy – significantly better than highly trained clinical experts assessing the same video recordings. The set of predictive social motor synchrony features that we identified also correlated significantly with symptom severity in the ASD group, suggesting that it can be used for both diagnostic classification and evaluating individual differences (vital for advancing precision medicine goals). Importantly, our findings were reproducible across samples: the same features identified in our adult analysis also predicted diagnosis in a child sample with high accuracy. Aims. In Aim 1, we will test the specificity of our computer vision approach by expanding comparisons to include a mixed psychiatric disorder group; Aim 2 will test dyadic synchrony in other body movements, and Aim 3 will define associations between interpersonal motor synchrony and dimensional aspects of social communication that span diagnostic categories. Impact: Our approach is designed for fast and rigorous assessment of social communication, providing a scalable solution to diagnosing ASD diagnosis and measuring individual variability, within a transdiagnostic, precision medicine framework.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2025 ( Subtotal = $0 )
2025	2023	THE CHILDREN'S HOSPITAL OF PHILADELPHIA	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	5	11/11/2024	NON-COMPETING CONTINUATION	$0
														Subtotal = $0

Issue Date FY: 2023 ( Subtotal = $645,982 )
2023	2023	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	5	2/6/2023	NON-COMPETING CONTINUATION	$645,982
														Subtotal = $645,982

Issue Date FY: 2022 ( Subtotal = $701,006 )
2022	2022	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	001	4	5/23/2022	NON-COMPETING CONTINUATION	$70,101
2022	2022	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	4	12/13/2021	NON-COMPETING CONTINUATION	$630,905
														Subtotal = $701,006

Issue Date FY: 2021 ( Subtotal = $720,921 )
2021	2021	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	001	3	4/14/2021	NON-COMPETING CONTINUATION	$72,093
2021	2021	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	3	11/25/2020	NON-COMPETING CONTINUATION	$648,828
														Subtotal = $720,921

Issue Date FY: 2020 ( Subtotal = $774,965 )
2020	2020	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	001	2	3/11/2020	NON-COMPETING CONTINUATION	$77,498
2020	2020	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	2	11/18/2019	NON-COMPETING CONTINUATION	$697,467
														Subtotal = $774,965

Issue Date FY: 2019 ( Subtotal = $835,525 )
2019	2019	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	000	1	12/21/2018	NEW	$835,525
2019	2019	CHILDREN'S HOSPITAL OF PHILADELPHIA, THE	3401 CIVIC CENTER BLVD	PHILADELPHIA	PA	19104	PHILADELPHIA	USA	Mental Health Research Grants	001	1	9/24/2019	NEW	$0
														Subtotal = $835,525

Grand Total All Awards = $3,678,399

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Predicting Autism and Social Functioning from Computer Vision Analyses of Motor Synchrony During Dyadic Interactions

Award Number: R01MH118327

ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer