Project Summary / Abstract
Colorectal cancer (CRC) is the third most commonly diagnosed malignancy and the second leading cause of
cancer death worldwide. There is an unmet need for accurate, cost-efficient, and broadly accessible risk-
stratification tools to identify patients at increased risk of recurrence , who are most likely to benefit from
adjuvant therapy. Current standard-of-care risk stratification approaches are inadequate. Every CRC surgical
candidate undergoes pathologic and radiologic evaluation of their tumor; these two modalities represent a rich,
readily accessible and, thus far, underutilized resource for developing new risk-stratification tools. Deep
learning (DL) has demonstrated great potential for augmenting physicians on an increasing range of diagnostic
and prognostic tasks in pathology, radiology, and clinical medicine. We hypothesize that applying integrated
DL-based analysis to multimodal (pathologic, radiologic, and electronic medical record (EMR)) data will yield
greatly improved stratification of CRC patients for adjuvant treatment planning. We propose to build the first
comprehensive, publicly-available, expert-annotated multimodal CRC dataset for deep learning, containing
annotated CRC pathology whole-slide images (WSI), preoperative CT and MRI images, and structured clinical
EMR data. Using this dataset, we will develop both single and multi-modality DL models for risk stratification of
surgically-resectable (Stage I-III) CRC patients.To test our hypothesis, we will compare the performance of
multi-modality models with that of single-modality models and existing methods of stratification. This project
benefits from the complementary expertise and resources of a unique interdisciplinary team spanning the fields
of machine learning, pathology, radiology, and oncology.