Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

Co-Design and Collaborative Development of an Imaging-Text-EHR Multimodal Foundation Model for Outcome Prediction of Critically Ill Patients

Award Number: OT2OD037995
ORGANIZATION: OFFICE OF THE DIRECTOR, NIH
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: OTHER
PERIOD OF PERFORMANCE START DATE: 06/30/2025
PERIOD OF PERFORMANCE END DATE: 06/29/2027

Group Awards By:

View Award Description

Co-Design and Collaborative Development of an Imaging-Text-EHR Multimodal Foundation Model for Outcome Prediction of Critically Ill Patients - ABSTRACT Mortality and morbidity are the most essential outcomes in patients admitted to the intensive care unit (ICU). Timely prediction can aid clinicians in making treatment decisions, optimizing resource allocation, improving outcomes, and managing patients’ and families’ expectations. Previously, our research team developed a multimodal deep learning model predicting mortality and morbidity among ICU patients by incorporating structured data (time-variant and time-invariant electronic health record [EHR] data) and unstructured data (clinical text and chest X-ray images). In this project, we aim to develop a multimodal foundation model that considers these three data modalities: chest X-ray (imaging), clinical notes (text), and structured data (EHR). The foundation model will then be fine-tuned to predict the probabilities of outcomes (mortality, morbidities, etc.) after different periods of time (within 4, 12, 24, or 48 hours, for example). Explanation of the predictions will be generated leveraging the multimodality of the model to help users assess its trustworthiness. To ensure that the design will meet the needs of care providers and minimize any ethical concerns, a co-design approach will be applied where clinicians, patient family members, ethics experts, and computer scientists will collaborate and iteratively develop the model using feedback from stakeholder engagement. The approach is divided into three tasks: (1) stakeholder engagement to aid in ethical and model design; (2) development of multimodal models, including using a foundation model architecture; and (3) external validation of models and assessment of algorithmic bias. The model may serve as a generalizable architecture for other multimodal models predicting clinical outcomes.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $818,681 )
2026	2026	UNIVERSITY OF CALIFORNIA, SAN DIEGO	9500 GILMAN DR	LA JOLLA	CA	92093	SAN DIEGO	USA	Trans-NIH Research Support	002	1	6/19/2026	SUPPLEMENT FOR EXPANSION	$818,681
2026	2025	UNIVERSITY OF CALIFORNIA, SAN DIEGO	9500 GILMAN DR	LA JOLLA	CA	92093	SAN DIEGO	USA	Trans-NIH Research Support	000	1	2/20/2026	NEW	$0
2026	2025	UNIVERSITY OF CALIFORNIA, SAN DIEGO	9500 GILMAN DR	LA JOLLA	CA	92093	SAN DIEGO	USA	Trans-NIH Research Support	001	1	6/18/2026	NEW	$0
														Subtotal = $818,681

Issue Date FY: 2025 ( Subtotal = $845,558 )
2025	2025	UNIVERSITY OF CALIFORNIA, SAN DIEGO	9500 GILMAN DR	LA JOLLA	CA	92093	SAN DIEGO	USA	Trans-NIH Research Support	000	1	6/28/2025	NEW	$845,558
														Subtotal = $845,558

Grand Total All Awards = $1,664,239

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Co-Design and Collaborative Development of an Imaging-Text-EHR Multimodal Foundation Model for Outcome Prediction of Critically Ill Patients

Award Number: OT2OD037995

ORGANIZATION: OFFICE OF THE DIRECTOR, NIH

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: OTHER

PERIOD OF PERFORMANCE START DATE: 06/30/2025

PERIOD OF PERFORMANCE END DATE: 06/29/2027

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer