Independent Evaluation of Commercial Diagnostic AI Solutions: A Necessary Step toward Increased Transparency

Published Online:
Free first page


  • 1. Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health 2021;3(3):e195–e203.
  • 2. Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices. U.S. Food and Drug Administration. Updated October 19, 2023. Accessed December 1, 2023.
  • 3. Roemer FW. The role of speed and possible implications. Radiology 2023;307(2):e222872.
  • 4. Allen B, Agarwal S, Coombs L, Wald C, Dreyer K. 2020 ACR Data Science Institute artificial intelligence survey. J Am Coll Radiol 2021;18(8):1153–1159.
  • 5. van Leeuwen KG, Schalekamp S, Rutten MJCM, van Ginneken B, de Rooij M. Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 2021;31(6):3797–3804.
  • 6. Mongan J, Moy L, Kahn CE Jr. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell 2020;2(2):e200029.
  • 7. Kelly BS, Judge C, Bollard SM, et al. Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE). Eur Radiol 2022;32(11):7998–8007. [Published correction appears in Eur Radiol 2022;32(11):8054.]
  • 8. Yu AC, Mohajer B, Eng J. External validation of deep learning algorithms for radiologic diagnosis: a systematic review. Radiol Artif Intell 2022;4(3):e210064.
  • 9. van Leeuwen KG, Schalekamp S, Rutten MJCM, et al. Comparison of commercial AI software performance for radiograph lung nodule detection and bone age prediction. Radiology 2024;310(1):e230981.
  • 10. Omoumi P, Ducarouge A, Tournier A, et al. To buy or not to buy—evaluating commercial AI solutions in radiology (the ECLAIR guidelines). Eur Radiol 2021;31(6):3786–3796.

Article History

Received: Dec 5 2023
Revision requested: Dec 7 2023
Revision received: Dec 8 2023
Accepted: Dec 11 2023
Published online: Jan 09 2024