Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

The role of distributional reinforcement learning in human neurons during impulsive choices

Award Number: R01MH128187
ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 02/03/2022
PERIOD OF PERFORMANCE END DATE: 12/31/2026

Group Awards By:

View Award Description

The role of distributional reinforcement learning in human neurons during impulsive choices - ABSTRACT Recent developments in artificial intelligence and neuroscience have revealed neural codes for reinforcement that represent predictions of a range of possible future reward outcomes, rather than a singular expected value. This distributional reinforcement learning has enabled improved performance of artificial agents and has straightforward implications for numerous neuropsychiatric disorders, particularly impulse control and substance use disorders. This proposal aims to leverage our experience recording neuronal activity from the brains of human neurosurgical patients in order to translate these recordings in a novel research direction: to understand the mechanisms of human choice behavior. We will determine where distributional codes exist in the human prefrontal and mesial temporal cortices, and how those codes are expressed dynamically in time as humans make impulsive choices during the Balloon Analog Risk Task (BART) and a probabilistic reversal learning task. The results of these experiments will have both important basic scientific implications and will begin to address how distributional reinforcement learning in the human brain contributes to impulsive choices. In order to begin translating this new area of knowledge to understand the underpinnings of human decisions, we will first establish the presence of distributional reinforcement learning in four brain areas that comprise a human decision-making circuit: Orbitofrontal Cortex, Anterior Cingulate Cortex, Amygdala, and Hippocampus. Specific Aim 1 will test the three essential predictions of distributional RL: whether populations of neurons in each of these brain areas exhibit 1) asymmetric scaling of reward prediction errors, 2) diverse reversal points, and 3) that prediction error asymmetries and reversal points correlate across neurons. Specific Aim 2 seeks to decode BART reward prediction distributions from neurons in the aforementioned brain areas and determine how changes in BART reward distributions correlate with the propensity to make impulsive choices. Specific Aim 3 will test how diversity in optimism and pessimism in each neuron recorded from the aforementioned brain areas correlates with valuation or devaluation across trials. The completion of these aims will constitute important basic research findings in discovering distributional RL in the human prefrontal and mesial temporal cortices. By uncovering neural population codes that underlie potentially impulsive choices in human decision-making circuits, these experiments also address fundamental neural mechanisms underlying impulsive choices. This issue is central to addressing important problems for contemporary mental health including substance use disorder and a many other neuropsychiatric disorders. These findings will have readily translatable implications for improving targeted electrical therapies for psychiatric disorders.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $447,773 )
2026	2026	UNIVERSITY OF UTAH	201 PRESIDENTS CIR	SALT LAKE CITY	UT	84112	SALT LAKE	USA	Mental Health Research Grants	000	5	2/11/2026	NON-COMPETING CONTINUATION	$447,773
														Subtotal = $447,773

Issue Date FY: 2025 ( Subtotal = $420,363 )
2025	2025	UNIVERSITY OF UTAH	201 PRESIDENTS CIR	SALT LAKE CITY	UT	84112	SALT LAKE	USA	Mental Health Research Grants	000	4	1/7/2025	NON-COMPETING CONTINUATION	$420,363
														Subtotal = $420,363

Issue Date FY: 2024 ( Subtotal = $494,545 )
2024	2024	UNIVERSITY OF UTAH	201 PRESIDENTS CIR	SALT LAKE CITY	UT	84112	SALT LAKE	USA	Mental Health Research Grants	000	3	1/10/2024	NON-COMPETING CONTINUATION	$445,091
2024	2024	UNIVERSITY OF UTAH	201 PRESIDENTS CIR	SALT LAKE CITY	UT	84112	SALT LAKE	USA	Mental Health Research Grants	001	3	8/15/2024	NON-COMPETING CONTINUATION	$49,454
														Subtotal = $494,545

Issue Date FY: 2023 ( Subtotal = $503,595 )
2023	2023	UNIVERSITY OF UTAH, THE	201 PRESIDENTS CIR	SALT LAKE CITY	UT	84112	SALT LAKE	USA	Mental Health Research Grants	000	2	1/30/2023	NON-COMPETING CONTINUATION	$503,595
														Subtotal = $503,595

Issue Date FY: 2022 ( Subtotal = $509,811 )
2022	2022	UNIVERSITY OF UTAH, THE	201 PRESIDENTS CIR	SALT LAKE CITY	UT	84112	SALT LAKE	USA	Mental Health Research Grants	001	1	6/27/2022	NEW	$50,981
2022	2022	UNIVERSITY OF UTAH, THE	201 PRESIDENTS CIR	SALT LAKE CITY	UT	84112	SALT LAKE	USA	Mental Health Research Grants	000	1	2/3/2022	NEW	$458,830
														Subtotal = $509,811

Grand Total All Awards = $2,376,087

Top

All Categories

About

Search

Reports

Data Submission

Award Information

The role of distributional reinforcement learning in human neurons during impulsive choices

Award Number: R01MH128187

ORGANIZATION: NATIONAL INSTITUTE OF MENTAL HEALTH

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 02/03/2022

PERIOD OF PERFORMANCE END DATE: 12/31/2026

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer