Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

Evidence-based guidance to support internal and external validity of AI healthcare datasets

Award Number: R01HG014227
ORGANIZATION: NATIONAL HUMAN GENOME RESEARCH INSTITUTE
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 09/01/2024
PERIOD OF PERFORMANCE END DATE: 06/30/2027

Group Awards By:

View Award Description

Evidence-based guidance to support internal and external validity of AI healthcare datasets - There is widespread agreement that fundamental issues for ethical development of artificial intelligence for health care (AI-HC) are generalizability and mitigating systematic error. In the U.S., the National Institutes of Health had launched initiatives to address these issues. However, it is not yet clear whether these initiatives are sufficient to achieve goals of generalizability, mitigating systematic error, and external and internal validity in AI-HC. Concepts and practices associated with goals of generalizability shape the extent to which AI-HC researchers and developers achieve their goals for generalizability and mitigating systematic error in their models. Yet insufficient understanding of how concepts such as systematic error and generalizability are defined and put into practice within AI-HC projects is an impediment to achieving these goals for AI-HC datasets. Prior studies of biomedical research indicate that scientists often hold differing concepts related to generalizability, systematic error, and internal and external validity. In turn, these concepts shape implementation of the practices used to achieve their related goals, such as technical approaches to reduce systematic error or efforts to address the composition of datasets. We propose to assess how issues relevant to generalizability and systematic error are conceptualized and put into practice in 50 NIH-funded AI-HC research projects. We will employ a “microethics” perspective, which focuses analysis on how high-level ethical goals are understood and put into practice in technical fields. This perspective allows examination of how different actors (e.g. data scientists, clinicians, annotators) involved in AI dataset development perceive relevant issues, such as: how internal and external validity are assessed; trade-offs in scientific and dataset composition goals; and the downstream impacts on the validity of datasets. Informed by these findings, we will develop evidence-informed practical guidance for the future creation of datasets in AI-HC that support generalizability, mitigation of systematic error, and internal and external validity. We will use a multi-pronged strategy to encourage uptake and implementation of guidance through the Bridge2AI program, the Center for ELSI Resources and Analysis, and interactive, case-based design exercises and workshops. [Modified


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $533,576 )
2026	2026	THE LELAND STANFORD JUNIOR UNIVERSITY	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Human Genome Research	000	3	8/12/2026	NON-COMPETING CONTINUATION	$533,576
														Subtotal = $533,576

Issue Date FY: 2025 ( Subtotal = $490,452 )
2025	2025	THE LELAND STANFORD JUNIOR UNIVERSITY	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Human Genome Research	000	2	9/5/2025	NON-COMPETING CONTINUATION	$490,452
														Subtotal = $490,452

Issue Date FY: 2024 ( Subtotal = $499,530 )
2024	2024	THE LELAND STANFORD JUNIOR UNIVERSITY	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Human Genome Research	000	1	8/30/2024	NEW	$499,530
														Subtotal = $499,530

Grand Total All Awards = $1,523,558

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Evidence-based guidance to support internal and external validity of AI healthcare datasets

Award Number: R01HG014227

ORGANIZATION: NATIONAL HUMAN GENOME RESEARCH INSTITUTE

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 09/01/2024

PERIOD OF PERFORMANCE END DATE: 06/30/2027

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer