Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

Crowd-Powered Machine Learning to Diagnose ASD and ADHD in Adolescents from Digital Social Interactions

Award Number: DP2EB035858
ORGANIZATION: NATIONAL INSTITUTE Of BIOMEDICAL IMAGING & BIOENGINEERING
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 09/21/2023
PERIOD OF PERFORMANCE END DATE: 09/20/2028

Group Awards By:

View Award Description

Crowd-Powered Machine Learning to Diagnose ASD and ADHD in Adolescents from Digital Social Interactions - Project Summary Digital technologies have the potential to provide remote and accessible psychiatric diagnostics to underserved families who have traditionally been left out of the healthcare system. Several recent research efforts have explored the use of structured behavioral data collection using digital devices coupled with automatic machine learning (ML) algorithms to distinguish a particular psychiatric condition from neurotypical controls. While these pure ML approaches have achieved performances >90% on balanced classification metrics on binary prediction tasks, there are limits to their ability to feasibly quantify the complex, social behavioral features needed for multi- condition and higher precision diagnostics. To enable such specificity for digital psychiatric diagnostics, I propose a novel paradigm-shifting approach which incorporates original crowdsourcing algorithms into the ML feature extraction process to create representation vectors of nuanced social behaviors with sufficient discriminative power to distinguish related and overlapping neuropsychiatric conditions such as Autism Spectrum Disorder and Attention-Deficit/Hyperactivity Disorder. Crowdsourcing, or the use of distributed workers to collectively work towards a larger task, is traditionally used to label training data for ML and is increasingly leveraged as a tool to run public health studies. However, crowdsourcing has yet to be thoroughly explored as a central tool in precision diagnostics for psychiatry. In the proposed paradigm, each crowd worker will answer targeted multiple choice questions about each video, reducing the feature space into a socially rich feature vector corresponding to the behaviors displayed in the video. My innovative crowdsourcing framework involves creating a quantified profile of each crowd worker to dynamically assign them to labeling tasks based on the categories of questions they rate in accordance with clinical experts. This crowdsourcing pipeline will be tested with respect to 3 major components of the digital diagnostics pipeline: (1) gamified structured video data curation from each participant, (2) behavioral feature extraction, and (3) diagnostic prediction with deep learning. The structured data collection will occur through paired social interactions between participants who remotely interact by playing social games on the web while their webcam and microphone record their behaviors. Crowd workers will watch the videos and answer multiple choice questions pertaining to the subject’s behavior in the video. The crowd annotations and metadata will be supplemented with computationally extracted eye gaze, facial emotion expression, vocal pitch, and speech timing features. These features will be collectively used to train a deep learning model which outputs both diagnostic categories and indicators of the presence of individual behavioral characteristics (e.g., hyperactivity and distractibility). Based on crowd labels of earlier games, additional games will be assigned to each subject in future data curation sessions. This paradigm has the potential to enable more nuanced behavior- based digital diagnostics, working towards a clinical workflow which can provide remote access to diagnoses for underserved populations who typically struggle to obtain neuropsychiatric healthcare.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $959,853 )
2026	2026	REGENTS OF THE UNIVERSITY OF CALIFORNIA, SAN FRANCISCO, THE	1855 FOLSOM ST STE 425	SAN FRANCISCO	CA	94103	SAN FRANCISCO	USA	Trans-NIH Research Support	001	3	8/12/2026	EXTENSION WITH OR WITHOUT FUNDS	$959,853
2026	2023	UNIVERSITY OF HAWAII	2425 CAMPUS RD SINCLAIR RM 1	HONOLULU	HI	96822	HONOLULU	USA	Trans-NIH Research Support	000	1	4/22/2026	NEW	$0
														Subtotal = $959,853

Issue Date FY: 2025 ( Subtotal = $0 )
2025	2023	REGENTS OF THE UNIVERSITY OF CALIFORNIA, SAN FRANCISCO, THE	1855 FOLSOM ST STE 425	SAN FRANCISCO	CA	94103	SAN FRANCISCO	USA	Trans-NIH Research Support	001	2	8/14/2025	CHANGE OF GRANTEE / TRAINING INSTITUTION / AWARDING INSTITUTION	$1,215,463
2025	2023	UNIVERSITY OF HAWAII	2425 CAMPUS RD SINCLAIR RM 1	HONOLULU	HI	96822	HONOLULU	USA	Trans-NIH Research Support	000	1	8/14/2025	NEW	-$1,215,463
														Subtotal = $0

Issue Date FY: 2023 ( Subtotal = $1,309,500 )
2023	2023	UNIVERSITY OF HAWAII	2425 CAMPUS RD SINCLAIR RM 1	HONOLULU	HI	96822	HONOLULU	USA	Trans-NIH Research Support	000	1	9/21/2023	NEW	$1,309,500
														Subtotal = $1,309,500

Grand Total All Awards = $2,269,353

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Crowd-Powered Machine Learning to Diagnose ASD and ADHD in Adolescents from Digital Social Interactions

Award Number: DP2EB035858

ORGANIZATION: NATIONAL INSTITUTE Of BIOMEDICAL IMAGING & BIOENGINEERING

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 09/21/2023

PERIOD OF PERFORMANCE END DATE: 09/20/2028

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer