Towards Interpretable Imaging-based Gastric Cancer Prognosis via Prototypical and Attentional Deep Learning - ABSTRACT The main goal of our project is to develop innovative interpretable deep learning models for multi-modality imaging in cancer prognostic assessment. Overall, we aim to improve both the accuracy and the interpretability of prognosis decisions from deep learning models. On the one hand, accurate prognostic assessment plays a key role in personalized treatment. Gastric cancer prognosis has historically been determined based on tumor histopathology (i.e., size, grade, etc.) and TNM staging. However, the current clinical staging system cannot accurately predict the prognosis of gastric cancer, and patients in the same stage would have significant variability in treatment outcomes. Therefore, it is crucial to develop new approaches for accurate prognostic prediction of gastric cancer patients on an individual basis. With the rising of artificial intelligence (AI), deep learning models show enormous potential in cancer prognosis tasks. On the other hand, nevertheless, only limited deep learning models have been adopted in clinical settings, primarily due to their lack of transparency and interpretability, which makes it challenging to gain the trust from doctors as well as patients especially for the high-stake scenarios such as prognosis and treatment decision-making. Existing deep learning models are mostly black-box and fully data-driven in nature, without integrating domain knowledge from physician experts and/or disease pathobiology. Thus, constructing interpretable deep learning models is also of great importance in gastric cancer prognosis, to better enhance the patient's treatment outcomes and life quality. Computed tomography (CT) and hematoxylin & eosin (H&E) stained whole-slide pathological images (WSIs) are commonly used in the clinical examination of gastric cancer patients. Our prior research has demonstrated that both CT and WSI images contain valuable prognostic information for gastric cancer and validated that deep learning approaches are effective for analyzing these types of images. To bridge the research gaps in interpretable prediction with deep learning, distinct from previous work, we will integrate domain knowledge from physician experts with biological insights into the model inference process through prototypical and attentional modeling, which are informed by and designed to capture important aspects of disease pathobiology. We propose three aims specifically: AIM1) Build a knowledge-guided Vision Transformer (ViT) that employs informative visual prototypes for explainable prognosis on CT images; AIM2) Develop a knowledge-guided Graph Neural Network (GNN) that employs contributive subgraph prototypes for interpretable prediction based on singe-cell classification; AIM3) Design a novel attentional multi-modal learning approach to fuse the clinical attribute information together with the complex interactions between WSI and CT in latent spaces for transparent prognosis. If successful, the proposed methods will enhance the accuracy and interpretation of prognosis predictions. Considering that CT and H&E imaging is used as part of routine clinical care, our study could provide new tools to improve treatment decisions, without additional cost.
