Sunday, May 18, 2025
5/18/2025
Integrating Radiologist Insights for Safe and Accurate AI-Assisted Prostate MRI Interpretation - Abstract: Advances in natural language processing can build on prior breakthroughs in image processing to offer clinicians new AI tools in the fight against prostate cancer, the second leading cause of cancer death. Current AI for interpreting multiparametric MRI (mpMRI) scans, the primary imaging modality in PCa, achieves high performance through training on visual encodings of human expertise (e.g. lesion annotations). Such models fail at translation to patient care because they do not derive and communicate their results through the standardized format accepted by clinicians, PIRADS. The expertise encoded in PIRADS reports offers a major resource for training AI to achieve clinical acceptance. The proposed research is needed to overcome two major gaps in knowledge necessary to develop advanced AI systems that can learn from both visual (annotations) and text (PIRADS) exemplars. (1) Data availability: Public PCa data repositories provide mpMRI scans and annotations but omit accompanying PIRADS reports. (2) AI modeling: Existing AI approaches focus on image processing and lack the capacity to integrate complex radiologist expertise expressed through language. This study will test the hypothesis that PIRADS reports can be made machinereadable and combined with visual data so that AI can be trained to interpret MRIs according to the reasoning processes of radiologists. Memorial Sloan Kettering (MSK) radiologists and University of Delaware (UD) researchers will collaborate to achieve the following aims. Aim 1): The MSK team will curate a comprehensive dataset by annotating 300 public and 50 MSK MRI scans with corresponding PIRADS reports. The UD team will leverage GPT4 to automatically extract radiologists’ reasoning processes, i.e., radiologist rationale, from each report. Aim 2): The UD team will develop a Prostate Vision Language Model (ProstateVLM), building on supportive pilot data to leverage medical foundation models and successfully integrate both images (MRIs and annotations) and text (rationales) in a uniform embedding space. ProstateVLM will be trained on the comprehensive dataset to accurately segment the prostate gland, anatomical zones, and lesions on MRI scans and align its segmentation with radiologist expertise. Aim 3): The UD team will evaluate whether inclusion of radiologist rationales from PIRADS as training data improves MRI segmentation vs. training on images alone, measured by Dice Score, mAP, etc. The MSK team will evaluate whether ProstateVLM’s interpretations of MRIs align with those of radiologists, measured by a questionnaire scoring metrics for accuracy and completeness. To support future research, the curated dataset and ProstateVLM will be shared through the Cancer Imaging Archive (TCIA).
HHS’ Tracking Accountability in Government Grants System (TAGGS) website provides access to detailed descriptions of grants, loans, aggregated direct payments and other types of financial assistance awarded by the HHS Operating Divisions (OPDIVs) and the Office of the Secretary Staff Divisions (STAFFDIVs).