Search
  • Quick Search in Journals
  • Quick Search anywhere
  • Quick search in Citations
Advanced Search
Menu
  • Information
    • Previous
    Next
    Original ResearchOpen Access

    Detection of Breast Cancer with Mammography: Effect of an Artificial Intelligence Support System

    Published Online:Nov 20 2018https://doi.org/10.1148/radiol.2018181371
    See editorial byManisha Bahl

    Abstract

    Radiologists had improved diagnostic performance for detection of breast cancer at mammography when using an artificial intelligence computer system for support, with no additional reading time required.

    Purpose

    To compare breast cancer detection performance of radiologists reading mammographic examinations unaided versus supported by an artificial intelligence (AI) system.

    Materials and Methods

    An enriched retrospective, fully crossed, multireader, multicase, HIPAA-compliant study was performed. Screening digital mammographic examinations from 240 women (median age, 62 years; range, 39–89 years) performed between 2013 and 2017 were included. The 240 examinations (100 showing cancers, 40 leading to false-positive recalls, 100 normal) were interpreted by 14 Mammography Quality Standards Act–qualified radiologists, once with and once without AI support. The readers provided a Breast Imaging Reporting and Data System score and probability of malignancy. AI support provided radiologists with interactive decision support (clicking on a breast region yields a local cancer likelihood score), traditional lesion markers for computer-detected abnormalities, and an examination-based cancer likelihood score. The area under the receiver operating characteristic curve (AUC), specificity and sensitivity, and reading time were compared between conditions by using mixed-models analysis dof variance and generalized linear models for multiple repeated measurements.

    Results

    On average, the AUC was higher with AI support than with unaided reading (0.89 vs 0.87, respectively; P = .002). Sensitivity increased with AI support (86% [86 of 100] vs 83% [83 of 100]; P = .046), whereas specificity trended toward improvement (79% [111 of 140]) vs 77% [108 of 140]; P = .06). Reading time per case was similar (unaided, 146 seconds; supported by AI, 149 seconds; P = .15). The AUC with the AI system alone was similar to the average AUC of the radiologists (0.89 vs 0.87).

    Conclusion

    Radiologists improved their cancer detection at mammography when using an artificial intelligence system for support, without requiring additional reading time.

    Published under a CC BY 4.0 license.

    See also the editorial by Bahl in this issue.

    References

    • 1. Smith RA, Cokkinides V, Brooks D, Saslow D, Brawley OW. Cancer screening in the United States, 2010: a review of current American Cancer Society guidelines and issues in cancer screening. CA Cancer J Clin 2010;60(2):99–119. Crossref, Medline, Google Scholar
    • 2. Broeders M, Moss S, Nyström L et al. The impact of mammographic screening on breast cancer mortality in Europe: a review of observational studies. J Med Screen 2012;19(Suppl 1):14–25. Crossref, Medline, Google Scholar
    • 3. Rimmer A. Radiologist shortage leaves patient care at risk, warns Royal College. BMJ 2017;359:j4683. Crossref, Medline, Google Scholar
    • 4. Bird RE, Wallace TW, Yankaskas BC. Analysis of cancers missed at screening mammography. Radiology 1992;184(3):613–617. Link, Google Scholar
    • 5. Majid AS, de Paredes ES, Doherty RD, Sharma NR, Salvador X. Missed breast carcinoma: pitfalls and pearls. RadioGraphics 2003;23(4):881–895. Link, Google Scholar
    • 6. Weber RJ, van Bommel RM, Louwman MW et al. Characteristics and prognosis of interval cancers after biennial screen-film or full-field digital screening mammography. Breast Cancer Res Treat 2016;158(3):471–483. Crossref, Medline, Google Scholar
    • 7. Broeders MJ, Onland-Moret NC, Rijken HJ, Hendriks JH, Verbeek AL, Holland R. Use of previous screening mammograms to identify features indicating cases that would have a possible gain in prognosis following earlier detection. Eur J Cancer 2003;39(12):1770–1775. Crossref, Medline, Google Scholar
    • 8. Gilbert FJ, Astley SM, Gillan MG et al. Single reading with computer-aided detection for screening mammography. N Engl J Med 2008;359(16):1675–1684. Crossref, Medline, Google Scholar
    • 9. Bargalló X, Santamaría G, Del Amo M et al. Single reading with computer-aided detection performed by selected radiologists in a breast cancer screening program. Eur J Radiol 2014;83(11):2019–2023. Crossref, Medline, Google Scholar
    • 10. Fenton JJ, Xing G, Elmore JG et al. Short-term outcomes of screening mammography using computer-aided detection: a population-based study of Medicare enrollees. Ann Intern Med 2013;158(8):580–587. Crossref, Medline, Google Scholar
    • 11. Gromet M. Comparison of computer-aided detection to double reading of screening mammograms: review of 231,221 mammograms. AJR Am J Roentgenol 2008;190(4):854–859. Crossref, Medline, Google Scholar
    • 12. Fenton JJ, Taplin SH, Carney PA et al. Influence of computer-aided detection on performance of screening mammography. N Engl J Med 2007;356(14):1399–1409. Crossref, Medline, Google Scholar
    • 13. Lehman CD, Wellman RD, Buist DS et al. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern Med 2015;175(11):1828–1837. Crossref, Medline, Google Scholar
    • 14. Azavedo E, Zackrisson S, Mejàre I, Heibert Arnlind M. Is single reading with computer-aided detection (CAD) as good as double reading in mammography screening? A systematic review. BMC Med Imaging 2012;12(1):22. Crossref, Medline, Google Scholar
    • 15. Litjens G, Kooi T, Bejnordi BE et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60–88. Crossref, Medline, Google Scholar
    • 16. Kooi T, Litjens G, van Ginneken B et al. Large scale deep learning for computer aided detection of mammographic lesions. Med Image Anal 2017;35:303–312. Crossref, Medline, Google Scholar
    • 17. Trister AD, Buist DSM, Lee CI. Will machine learning tip the balance in breast cancer screening? JAMA Oncol 2017;3(11):1463–1464. Crossref, Medline, Google Scholar
    • 18. Hupse R, Samulski M, Lobbes MB et al. Computer-aided detection of masses at mammography: interactive decision support versus prompts. Radiology 2013;266(1):123–129. Link, Google Scholar
    • 19. Samulski M, Hupse R, Boetes C, Mus RD, den Heeten GJ, Karssemeijer N. Using computer-aided detection in mammography as a decision support. Eur Radiol 2010;20(10):2323–2330. Crossref, Medline, Google Scholar
    • 20. Hillis SL, Obuchowski NA, Berbaum KS. Power estimation for multireader ROC methods an updated and unified approach. Acad Radiol 2011;18(2):129–142. Crossref, Medline, Google Scholar
    • 21. Bria A, Karssemeijer N, Tortorella F. Learning from unbalanced data: a cascade-based approach for detecting clustered microcalcifications. Med Image Anal 2014;18(2):241–252. Crossref, Medline, Google Scholar
    • 22. Mordang JJ, Janssen T, Bria A, Kooi T, Gubern-Mérida A, Karssemeijer N, eds. Automatic microcalcification detection in multi-vendor mammography using convolutional neural networks. International Workshop on Digital Mammography. Cham, Switzerland: Springer, 2016.ses in mammograms. IEEE Trans Med Imaging 2009;28(12):2033–2041. Crossref, Google Scholar
    • 23. Hupse R, Karssemeijer N. Use of normal tissue context in computer-aided detection of masses in mammograms. IEEE Trans Med Imaging 2009;28(12):2033–2041. Crossref, Medline, Google Scholar
    • 24. Karssemeijer N, Te Brake GM. Detection of stellate distortions in mammograms. IEEE Trans Med Imaging 1996;15(5):611–619. Google Scholar
    • 25. Karssemeijer N. Automated classification of parenchymal patterns in mammograms. Phys Med Biol 1998;43(2):365–378. Crossref, Medline, Google Scholar
    • 26. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis: generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992;27(9):723–731. Crossref, Medline, Google Scholar
    • 27. Obuchowski NA. Multireader, multimodality receiver operating characteristic curve studies: hypothesis testing and sample size estimation using an analysis of variance approach with dependent observations. Acad Radiol 1995;2(Suppl 1):S22–S29; discussion S57–S64, S70–S71 pas. Medline, Google Scholar
    • 28. Hillis SL. A comparison of denominator degrees of freedom methods for multiple observer ROC analysis. Stat Med 2007;26(3):596–619. Crossref, Medline, Google Scholar
    • 29. Hillis SL, Berbaum KS, Metz CE. Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Acad Radiol 2008;15(5):647–661. Crossref, Medline, Google Scholar
    • 30. McCullagh P, Nelder JA. Generalized linear models. Boca Raton, Fla: CRC, 1989. Crossref, Google Scholar
    • 31. Tucker L, Gilbert FJ, Astley SM et al. Does reader performance with digital breast tomosynthesis vary according to experience with two-dimensional mammography? Radiology 2017;283(2):371–380. Link, Google Scholar
    • 32. Becker AS, Marcon M, Ghafoor S, Wurnig MC, Frauenfelder T, Boss A. Deep learning in mammography: diagnostic accuracy of a multipurpose image analysis software in the detection of breast cancer. Invest Radiol 2017;52(7):434–440. Crossref, Medline, Google Scholar
    • 33. Kim EK, Kim HE, Han K et al. Applying data-driven imaging biomarker in mammography for breast cancer screening: preliminary study. Sci Rep 2018;8(1):2762. Crossref, Medline, Google Scholar
    • 34. Evans KK, Birdwell RL, Wolfe JM. If you don’t find it often, you often don’t find it: why some cancers are missed in breast cancer screening. PLoS One 2013;8(5):e64366. Crossref, Medline, Google Scholar
    • 35. Gur D, Bandos AI, Cohen CS et al. The “laboratory” effect: comparing radiologists’ performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 2008;249(1):47–53. Link, Google Scholar
    • 36. Gennaro G, Hendrick RE, Ruppel P et al. Performance comparison of single-view digital breast tomosynthesis plus single-view digital mammography with two-view digital mammography. Eur Radiol 2013;23(3):664–672. Crossref, Medline, Google Scholar
    • 37. Warren RM, Duffy SW. Comparison of single reading with double reading of mammograms, and change in effectiveness with experience. Br J Radiol 1995;68(813):958–962. Crossref, Medline, Google Scholar
    • 38. Thurfjell EL, Lernevall KA, Taube AA. Benefit of independent double reading in a population-based mammography screening program. Radiology 1994;191(1):241–244. Link, Google Scholar

    Article History

    Received: June 10 2018
    Revision requested: July 30 2018
    Revision received: Sept 21 2018
    Accepted: Sept 28 2018
    Published online: Nov 20 2018
    Published in print: Feb 2019