Universal encoding of pan-cancer histology by deep texture representations.

Abstract

Cancer histological images contain rich biological and clinical information, but quantitative representation can be problematic and has prevented the direct comparison and accumulation of large-scale datasets. Here, we show successful universal encoding of cancer histology by deep texture representations (DTRs) produced by a bilinear convolutional neural network. DTR-based, unsupervised histological profiling, which captures the morphological diversity, is applied to cancer biopsies and reveals relationships between histologic characteristics and the response to immune checkpoint inhibitors (ICIs). Content-based image retrieval based on DTRs enables the quick retrieval of histologically similar images using The Cancer Genome Atlas (TCGA) dataset. Furthermore, via comprehensive comparisons with driver and clinically actionable gene mutations, we successfully predict 309 combinations of genomic features and cancer types from hematoxylin-and-eosin-stained images. With its mounting capabilities on accessible devices, such as smartphones, universal encoding for cancer histology has a strong impact on global equalization for cancer diagnosis and therapies.

• Authors: Daisuke Komura, Akihiro Kawabe, Keisuke Fukuta, Kyohei Sano, Toshikazu Umezaki, Hirotomo Koda, Ryohei Suzuki, Ken Tominaga, Mieko Ochi, Hiroki Konishi, Fumiya Masakado, Noriyuki Saito, Yasuyoshi Sato, Takumi Onoyama, Shu Nishida, Genta Furuya, Hiroto Katoh, Hiroharu Yamashita, Kazuhiro Kakimi, Yasuyuki Seto, Tetsuo Ushiku, Masashi Fukayama, Shumpei Ishikawa
• Journal: Cell reports (Cell Rep) Vol. 38 Issue 9 Pg. 110424 (03 01 2022) ISSN: 2211-1247 [Electronic] United States
• PMID: 35235802 (Publication Type: Journal Article, Research Support, Non-U.S. Gov't)
• Copyright: Copyright © 2022 The Authors. Published by Elsevier Inc. All rights reserved.

Topics

• Genomics
• Humans
• Mutation (genetics)
• Neoplasms (genetics)
• Neural Networks, Computer