Predicting Molecular Traits from Tissue Morphology Through Self-interactive Multi-instance Learning

Previous efforts to learn histology features that correlate with specific genetic/molecular traits resort to tile-level multi-instance learning (MIL) which relies on a fixed pretrained model for feature extraction and an instance-bag classifier. We argue that such a two-step approach is not optimal at capturing both fine-grained features at tile level and global features at slide level optimal to the task. We propose a self-interactive MIL that iteratively feedbacks training information between the fine-grained and global context features. We validate the proposed approach on 4 subtyping tasks: EMT status (ovarian), KRAS mutation (colon and lung), EGFR mutation (colon), and HER2 status (breast). Our approach yields an average improvement of 7.05 % - 8.34 % (in terms of AUC) over the baseline.