Radiomics for Detection of Pancreas Adenocarcinoma on CT Scans: Impact of Biliary Stents
Editor:
The Medical Segmentation Decathlon (MSD) data set has 420 abdomen CT scans of patients with different pancreatic tumors. It comes in two groups: a training group (n = 281 CT scans) and a testing group (n = 139 CT scans). The MSD team has provided segmentations only for the training group (1). In their carefully designed study (2) in the July 2021 issue of Radiology: Imaging Cancer, Dr Chen and colleagues used these MSD team-provided segmentations of 182 pancreatic ductal adenocarcinoma (PDAC)–containing CT scans for development of a generalized radiomics model and for external validation of their local model. Use of these external CT scans increased their model's sensitivity on the external MSD test set by nearly 20% and 10% in patch- and patient-based analyses, respectively.
The use of public data sets to augment internal data sets and to evaluate generalizability of radiomics is laudable. However, caution is warranted because quality gaps in some public data sets have previously resulted in inadvertent oversights in artificial intelligence experiments (1,3). We believe that certain features of the CT scans with PDAC from the MSD are germane to the study by Dr Chen and colleagues. As we have recently documented (1), 74 (approximately 40%) of these CT scans with PDAC from the MSD training group have biliary stents. Devices such as stents are a source of bias for machine learning models because a model learns to associate the presence of such devices with the underlying diagnosis, which leads to inadvertent overestimation of the model's performance (1,4). Second, such stents result in streak artifacts that obscure margins of PDAC, a tumor with highly infiltrative morphology, and increase the variability in tumor segmentation. Finally, such devices result in undesirable, unpredictable, and nonreducible variations in intensity and texture features (5). Such variations impact the reproducibility and robustness of radiomics, which has at least partly contributed to the clinical translation gap that exists in this domain. In view of these considerations, we wonder about the number of CT scans with stents in their local Taiwanese data set and the process that the authors adopted to address the confounding effect of biliary stents in both the local and the MSD data sets. Such information would be highly informative for future studies on this topic.
References
- 1. . Quality gaps in public pancreas imaging datasets: Implications & challenges for AI applications. Pancreatology 2021;21(5):1001–1008.
- 2. . Radiomic features at CT can distinguish pancreatic cancer from noncancerous pancreas. Radiol Imaging Cancer 2021;3(4):e210010.
- 3. . Convolutional neural network for the detection of pancreatic cancer on CT scans. Lancet Digit Health 2020;2(9):e453.
- 4. . On the Composition and Limitations of Publicly Available COVID-19 X-Ray Imaging Datasets. arXiv:2008.11572. [preprint] https://arxiv.org/abs/2008.11572. Posted August 26, 2020. Accessed July 24, 2021.
- 5. . Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017;14(12):749–762.
References
- 1. . Radiomic features at CT can distinguish pancreatic cancer from noncancerous pancreas. Radiol Imaging Cancer 2021;3(4):e210010.
- 2. . A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv 1902.09063. [preprint] http://arxiv.org/abs/1902.09063. Posted February 25, 2019. Accessed November 22, 2021.
References
| 1. | Suman G, Patra A, Korfiatis P, . Quality gaps in public pancreas imaging datasets: Implications & challenges for AI applications. Pancreatology 2021;21(5):1001–1008. Crossref Medline Google Scholar |
| 2. | Chen PT, Chang D, Yen H, . Radiomic features at CT can distinguish pancreatic cancer from noncancerous pancreas. Radiol Imaging Cancer 2021;3(4):e210010. Link Google Scholar |
| 3. | Suman G, Panda A, Korfiatis P, Goenka AH. Convolutional neural network for the detection of pancreatic cancer on CT scans. Lancet Digit Health 2020;2(9):e453. Crossref Medline Google Scholar |
| 4. | Garcia Santa Cruz B, Sölter J, Bossa MN, Husch AD. On the Composition and Limitations of Publicly Available COVID-19 X-Ray Imaging Datasets. arXiv:2008.11572. [preprint] https://arxiv.org/abs/2008.11572. Posted August 26, 2020. Accessed July 24, 2021. Google Scholar |
| 5. | Lambin P, Leijenaar RTH, Deist TM, . Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017;14(12):749–762. Crossref Medline Google Scholar |
Response
We appreciate the comments on our article (1). In our study, 4.3% (15 of 349) of PDAC cases in the local training and validation set and 11.2% (21 of 187) of cases in the local test sets had biliary stents. In local test sets, the sensitivity for PDAC in patients with and without stents was 95.2% (95% CI: 76.2, 99.9) and 93.4% (95% CI: 88.5, 96.6) with the local model and 95.2% (95% CI: 76.2, 99.9) and 94.6% (95% CI: 90.0, 97.5) with the generalized model, respectively. Sixteen (44.4%) among the 36 patients with PDAC in the external test set 2 from the MSD data set (2) had biliary stents, and the sensitivity for PDAC in patients with and without stents was 62.5% (95% CI: 35.4, 84.8) and 80.0% (95% CI: 56.3, 94.3) with the local model and 75.0% (95% CI: 47.6, 92.7) and 85.0% (95% CI: 62.1, 96.8) with the generalized model, respectively. Collectively, the comparable sensitivities between patients without and with stents in all test sets argue against the possibility that including patients with stents caused biased overestimation of model performance in our study.
Several reasons might explain why biliary stents had little influence on model performance in our study. Overall, only 6.7% of patients with PDAC in the local data sets had stents. The local data constituted all and 70.5% of the training data for the local and the generalized model, respectively. Second, when manually segmenting the tumors in local data sets for subsequent generation of cancerous patches and further model training, care was taken not to include the region occupied by stents as much as possible. Furthermore, manually segmented tumors were cropped into patches, each of which subjected to extraction of radiomic features. Although several patches near the stents could still be affected by stent-induced streak artifacts, most of the patches were away from the stents and hence little affected. Therefore, compared with subjecting the whole tumor with or without stents to analysis, our patch-based approach for model training and inference should mitigate the confounding effects of biliary stents. We agree that stents may cause unpredictable and nonreducible variations in intensity and texture features, but images with stents are inevitable when applying radiomics models in clinical practice. Our patch-based analytic approach might be a potential solution for images with stents and warrants further exploration.
Disclosures of conflicts of interest: P.T.C. No relevant relationships. D.C. Grant with Ministry of Science and Technology, Taiwan (author receives salary from this grant); inventor of pending patent entitled Medical Image Analyzing System and Method Thereof. W.C.L. No relevant relationships. W.W. No relevant relationships.References
| 1. | Chen PT, Chang D, Yen H, . Radiomic features at CT can distinguish pancreatic cancer from noncancerous pancreas. Radiol Imaging Cancer 2021;3(4):e210010. Link Google Scholar |
| 2. | Simpson AL, Antonelli M, Bakas S, . A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv 1902.09063. [preprint] http://arxiv.org/abs/1902.09063. Posted February 25, 2019. Accessed November 22, 2021. Google Scholar |
