Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

Developing Multimodal Foundation Models for Eye Disease Screening and Prognosis

Award Number: R01EY038687
ORGANIZATION: NATIONAL EYE INSTITUTE
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 08/01/2026
PERIOD OF PERFORMANCE END DATE: 07/31/2030

Group Awards By:

View Award Description

Developing Multimodal Foundation Models for Eye Disease Screening and Prognosis - Project Summary Motivation: The three leading irreversible eye diseases in the US causing unrestorable vision loss with currently available clinical treatment are age-related macular degeneration, diabetic retinopathy, and glaucoma, affecting approximately 20 million, 9.6 million, and 3 million individuals, respectively. Timely diagnosis and accurate prognosis enable prompt and personalized treatment, reducing the risk of further vision loss. Supervised deep learning is promising to provide affordable screening and accurate prognosis for eye diseases, but it requires large amounts of labeled data that are costly to acquire. Foundation models have emerged as a promising approach to best leverage vast amounts of unlabeled data to benefit downstream supervised tasks using labeled data with a smaller sample size for ophthalmic applications. Previous ophthalmic foundation models are limited by the following critical issues: (1) Enriched information from clinical notes has not been incorporated; (2) Model domain adaptability has not been addressed; (3) Federated learning strategies to enable cross-institutional foundation model training have not yet been developed. Overall Approach: The vision-language (VL) foundation model is trained using pairs of retinal images and clinical notes and used to initialize downstream supervised models for eye disease detection and progression prediction. The ground truth labels will be ascertained by clinician assessment. For eye disease detection, only the image component is used; for progression prediction, both image and text components are utilized. Aims: (1) Developing and optimizing a reciprocal training paradigm MutualAssist. We will develop a reciprocal training paradigm termed MutualAssist to simultaneously update clinical text summarization by a large language model (LLM) and feature learning by the VL foundation model. Low-rank adaptation method will be used to update the LLM. (2) Developing and optimizing a domain adaptation method DomainDuet. We will develop a domain adaptation method termed DomainDuet to learn both domain-invariant and domain-specific features either through adversarial and cooperative classifiers or by minimizing and maximizing feature distances between demographic groups. We will integrate DomainDuet into MutualAssist framework with either a concurrent or alternate updating scheme. (3) Developing and optimizing a federated foundation model method FedFound. We will develop and optimize a federated learning approach for foundation model training, FedFound, which balances cross-institutional global learning and institution-specific local learning by combining the global full parameter updating with the local low-rank parameter updating. Main Deliverables and Impacts: We will develop innovative multimodal VL foundation models to improve the performance in downstream supervised tasks for eye disease screening and prognosis by novel methods to improve clinical note summarization, domain adaptability, and federated foundation model training.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $858,394 )
2026	2026	SCHEPENS EYE RESEARCH INSTITUTE, INC., THE	20 STANIFORD ST	BOSTON	MA	02114	SUFFOLK	USA	Vision Research	000	1	7/20/2026	NEW	$858,394
														Subtotal = $858,394

Grand Total All Awards = $858,394

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Developing Multimodal Foundation Models for Eye Disease Screening and Prognosis

Award Number: R01EY038687

ORGANIZATION: NATIONAL EYE INSTITUTE

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 08/01/2026

PERIOD OF PERFORMANCE END DATE: 07/31/2030

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer