Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( 🔒 ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Need Help

Advancing AI for WeaklySupervised Multimodal Alignment and Query-Based Interpretation in Biological Imaging

Award Number: R01LM015260
ORGANIZATION: NATIONAL LIBRARY OF MEDICINE
OPDIV: NIH
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 06/01/2026
PERIOD OF PERFORMANCE END DATE: 05/31/2030

Group Awards By:

View Award Description

Advancing AI for WeaklySupervised Multimodal Alignment and Query-Based Interpretation in Biological Imaging - PROJECT SUMMARY A major challenge in modern biological data analysis is integrating and reasoning over the vast volumes of unstructured, multimodal data now available, such as images and text. While each modality offers complementary biological insight, they are not encoded in a shared format or language, making it difficult to align them or reason across them computationally. A core unmet need is the development of AI systems that can bridge this divide by learning shared representations across data types. This project targets building AI systems for aligning and reasoning jointly over biological image and text data. We focus on microscopy and the challenge of interpreting microscopy images often requiring integration with broader biological context—relating observed phenotypes to those seen in other experiments, identifying plausible mechanisms, and connecting to relevant prior studies. This knowledge is frequently buried in unstructured images and text scattered across publications, databases, and annotations. Conventional AI systems rely on supervised learning, which demands large amounts of manually annotated data and cannot scale to the complexity or breadth of modern biology. Training AI models using weak supervision offers a promising alternative: by learning from loosely aligned image-text pairs, models can capture cross-modal associations from noisy but abundant sources. Vision-language models (VLMs) built on this principle embed images and text into a shared semantic space and support flexible reasoning tasks such as retrieval and question answering. However, current models often suffer from “blurry vision”—they can identify broad semantic matches between images and text but fail to resolve the fine-grained visual distinctions essential for biological interpretation. The goal of our project is to overcome this limitation by advancing weak supervision methods that enable fine-grained alignment between biological images and text, with a focus on microscopy. We will curate a large and diverse dataset of fine-grained image-text pairs and train a visual encoder using multi-scale contrastive learning to integrate both global and local alignment signals. This encoder will power an agentic AI system for query-based interpretation of microscopy images, that can retrieve relevant biological evidence and generate natural-language interpretations of microscopy images in response to researcher queries. We will validate the system in expert-driven use cases spanning single-cell perturbation and tissue-level pathology, and disseminate it through integration into widely used imaging workflows. By building AI tools that help researchers connect microscopy image content to pathways, phenotypes, and prior studies, we aim to support flexible, biologically grounded exploration and accelerate data-driven discovery. By open-sourcing our datasets, methods, and trained models for fine-grained image-text alignment, we also aim to advance the broader capabilities of multimodal AI for biological data analysis.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2026 ( Subtotal = $1,393,955 )
2026	2026	THE LELAND STANFORD JUNIOR UNIVERSITY	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Medical Library Assistance	000	1	5/12/2026	NEW	$1,393,955
														Subtotal = $1,393,955

Grand Total All Awards = $1,393,955

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Advancing AI for WeaklySupervised Multimodal Alignment and Query-Based Interpretation in Biological Imaging

Award Number: R01LM015260

ORGANIZATION: NATIONAL LIBRARY OF MEDICINE

OPDIV: NIH

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 06/01/2026

PERIOD OF PERFORMANCE END DATE: 05/31/2030

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer