Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

Multimodal AI-enabled synchronized neural, physiological, and behavioral recordings in humans during unstructured behavior to study dynamic task control

Award Number: R61MH142354
ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 09/01/2026
PERIOD OF PERFORMANCE END DATE: 05/31/2029

Group Awards By:

View Award Description

Multimodal AI-enabled synchronized neural, physiological, and behavioral recordings in humans during unstructured behavior to study dynamic task control - Project summary Complex human behavior - such as driving a car - involves the flexible sequencing of many brief tasks in order to achieve a goal. Current constrained cognitive tasks completely miss this “task control” aspect of cognition, leaving us with little understanding of the underlying mechanisms. The behavior of subjects with lesions in the frontal cortex or with neuropsychiatric disease illustrates what is being missed: such patients can often perform simple behaviors, but fail to properly sequence these behaviors to achieve a goal. Therefore, the fundamental deficit that these patients suffer from is not visible during simple behaviors, making its study impossible. There are two major challenges to studying sufficiently complex behavior that involves task control. First one needs to quantify complex unconstrained behaviors such as social interactions or driving a car. Second, one needs to simultaneously observe human brain activity at cellular-level resolution. Here, as part of the BRAIN initiatives Brain and Behavior Quantification and Synchronization consortium, we tackle these challenges by (i) developing instrumentation to monitor and quantify human behavior in unprecedented detail, and (ii) combining this new system with large-scale recordings from single neurons in the brains of epilepsy patients. We will develop a system to acquire and synchronize multi-modal streams of behavioral data, including a head- mounted eye tracking device to track gaze position, egocentric and allocentric video and audio, and a wearable device to track biophysiological signals (Aim 1, R61). This data will then be labeled automatically using artificial intelligence methods, including labeling objects fixated by the subject, transcribing what is being said from audio, and identifying behavioral states based on the biophysiology (Aim 2). The new system will be validated for two use cases: ambulatory control subjects and patients in the epilepsy monitoring unit. We will then utilize the newly developed system to examine the neural mechanisms of task control with synchronized simultaneous single-neuron recordings in human epilepsy patients (Aim 3). We will identify behavioral and neural modules (“microtasks”) and the sequence by which they are engaged, and will then examine the overarching hypothesis that groups of neurons are specialized for executing a specific microtask. These experiments promise to reveal new insights into the brain architecture of task control that were made possible by the availability of quantified multi-modal behavior. Lastly, an overarching aim is to construct and train a large foundation model that learns relationships between the different streams of behavior and the neural recordings. Our system will be released as open source and requires only commercially available devices and a cellphone, making the system widely available for others to use in their research. Our multi-modal behavior and synchronized neural recording approach will lay the groundwork for revealing how the human brain achieves “task control”, an understanding of which is urgently needed to develop new treatments for the effects of frontal lobe lesions and psychiatric diseases that affect executive control.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $1,417,650 )
2026	2026	CALIFORNIA INSTITUTE OF TECHNOLOGY	1200 E CALIFORNIA BLVD	PASADENA	CA	91125	LOS ANGELES	USA	Mental Health Research Grants	001	1	9/11/2026	NEW	$0
2026	2026	CALIFORNIA INSTITUTE OF TECHNOLOGY	1200 E CALIFORNIA BLVD	PASADENA	CA	91125	LOS ANGELES	USA	Mental Health Research Grants	000	1	8/26/2026	NEW	$1,417,650
														Subtotal = $1,417,650

Grand Total All Awards = $1,417,650

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Multimodal AI-enabled synchronized neural, physiological, and behavioral recordings in humans during unstructured behavior to study dynamic task control

Award Number: R61MH142354

ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 09/01/2026

PERIOD OF PERFORMANCE END DATE: 05/31/2029

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer