The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping
Abstract
The Image Biomarker Standardization Initiative validated consensus-based reference values for 169 radiomics features, thus enabling calibration and verification of radiomics software.
Background
Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use.
Purpose
To standardize a set of 174 radiomic features.
Materials and Methods
Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features.
Results
Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI).
Conclusion
A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software.
© RSNA, 2020
Online supplemental material is available for this article.
See also the editorial by Kuhl and Truhn in this issue.
References
- 1. . Predictive biomarkers: A paradigm shift towards personalized cancer medicine. Nat Rev Clin Oncol 2011;8(10):587–596.
- 2. . Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol 2017;14(3):169–186.
- 3. . Radiomics: The bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017;14(12):749–762.
- 4. . A Deep Look into the Future of Quantitative Imaging in Oncology: A Statement of Working Principles and Proposal for Change. Int J Radiat Oncol Biol Phys 2018;102(4):1074–1082.
- 5. . Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016;278(2):563–577.
- 6. . Decoding tumor phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014;5(1):4006.
- 7. . A radiomics approach to assess tumor-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: An imaging biomarker, retrospective multicohort study. Lancet Oncol 2018;19(9):1180–1191.
- 8. . A mathematical-descriptor of tumor-mesoscopic-structure from computed-tomography images annotates prognostic- and molecular-phenotypes of epithelial ovarian cancer. Nat Commun 2019;10(1):764.
- 9. . Radiogenomics: Bridging imaging and genomics. Abdom Radiol (NY) 2019;44(6):1960–1984.
- 10. . Quantitative MRI Brain Studies in Mild Cognitive Impairment and Alzheimer’s Disease: A Methodological Review. IEEE Rev Biomed Eng 2018;11:97–111.
- 11. . Multi-scale radiomic analysis of sub-cortical regions in MRI related to autism, gender and age. Sci Rep 2017;7(1):45639.
- 12. . Radiomics of CT Features May Be Nonreproducible and Redundant: Influence of CT Acquisition Parameters. Radiology 2018;288(2):407–415.
- 13. . Vulnerabilities of radiomic signature development: The need for safeguards. Radiother Oncol 2019;130:2–9.
- 14. . Reproducibility of CT Radiomic Features within the Same Patient: Influence of Radiation Dose and CT Reconstruction Settings. Radiology 2019;293(3):583–591.
- 15. . Radiomics of Lung Nodules: A Multi-Institutional Study of Robustness and Agreement of Quantitative Imaging Features. Tomography 2016;2(4):430–437.
- 16. . Post-radiochemotherapy PET radiomics in head and neck cancer: The influence of radiomics implementation on the reproducibility of local control tumor models. Radiother Oncol 2017;125(3):385–391.
- 17. . Characterization of PET/CT images using texture analysis: The past, the present…any future? Eur J Nucl Med Mol Imaging 2017;44(1):151–165.
- 18. . Variation in algorithm implementation across radiomics software. J Med Imaging (Bellingham) 2018;5(4):044505.
- 19. . Repeatability and Reproducibility of Radiomic Features: A Systematic Review. Int J Radiat Oncol Biol Phys 2018;102(4):1143–1158.
- 20. . A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys Med Biol 2015;60(14):5471–5496.
- 21. . Radiomics Digital Phantom. https://doi.org/10.17195/candat.2016.08.1. Published 2016. Accessed April 24, 2019.
- 22. . The Cancer Imaging Archive (TCIA): Maintaining and operating a public information repository. J Digit Imaging 2013;26(6):1045–1057https://doi.org/10.1007/s10278-013-9622-7.
- 23. . Data from: A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. The Cancer Imaging Archive. Published 2015. Accessed October 18, 2019.
- 24. . Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 2014;67(4):401–409.
- 25. . Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 1979;86(2):420–428.
- 26. . Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996;1(1):30–46.
- 27. . A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15(2):155–163.
- 28. . Towards clinical application of image mining: A systematic review on artificial intelligence and radiomics. Eur J Nucl Med Mol Imaging 2019;46(13):2656–2672.
- 29. . Tumor texture analysis in 18F-FDG PET: Relationships between texture parameters, histogram indices, standardized uptake values, metabolic volumes, and total lesion glycolysis. J Nucl Med 2014;55(3):414–422.
- 30. . Test-Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific? Tomography 2016;2(4):361–365.
- 31. . Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD Statement. Br J Surg 2015;102(3):148–158.
- 32. . Three-dimensional solid texture analysis in biomedical imaging: Review and opportunities. Med Image Anal 2014;18(1):176–196.
- 33. . Radiomics in nuclear medicine: Robustness, reproducibility, standardization, and how to avoid data analysis traps and replication crisis. Eur J Nucl Med Mol Imaging 2019;46(13):2638–2655.
- 34. . Validation of a Method to Compensate Multicenter Effects Affecting CT Radiomics. Radiology 2019;291(1):53–59.
- 35. . Deep Learning-based Image Conversion of CT Reconstruction Kernels Improves Radiomics Reproducibility for Pulmonary Nodules or Masses. Radiology 2019;292(2):365–373.
Article History
Received: May 22 2019Revision requested: July 3 2019
Revision received: Dec 9 2019
Accepted: Jan 6 2020
Published online: Mar 10 2020
Published in print: May 2020