Published Online:

We demonstrated that computer-aided detection has an additional beneficial effect on the detection of pulmonary nodules beyond the effect of bone-suppressed images alone.


To evaluate the added value of computer-aided detection (CAD) for lung nodules on chest radiographs when radiologists have bone-suppressed images (BSIs) available.

Materials and Methods

Written informed consent was waived by the institutional review board. Selection of study images and study setup was reviewed and approved by the institutional review boards. Three hundred posteroanterior (PA) and lateral chest radiographs (189 radiographs with negative findings and 111 radiographs with a solitary nodule) in 300 subjects were selected from image archives at four institutions. PA images were processed by using a commercially available CAD, and PA BSIs were generated. Five radiologists and three residents evaluated the radiographs with BSIs available, first, without CAD and, second, after inspection of the CAD marks. Readers marked locations suspicious for a nodule and provided a confidence score for that location to be a nodule. Location-based receiver operating characteristic analysis was performed by using jackknife alternative free-response receiver operating characteristic analysis. Area under the curve (AUC) functioned as figure of merit, and P values were computed with the Dorfman-Berbaum-Metz method.


Average nodule size was 16.2 mm. Stand-alone CAD reached a sensitivity of 74% at 1.0 false-positive mark per image. Without CAD, average AUC for observers was 0.812. With CAD, performance significantly improved to an AUC of 0.841 (P = .0001). CAD detected 127 of 239 nodules that were missed after evaluation of the radiographs together with BSIs pooled over all observers. Only 57 of these detections were eventually marked by the observers after review of CAD candidates.


CAD improved radiologists’ performance for the detection of lung nodules on chest radiographs, even when baseline performance was optimized by providing lateral radiographs and BSIs. Still, most of the true-positive CAD candidates are dismissed by observers.

© RSNA, 2014


  • 1. Austin JH, Romney BM, Goldsmith LS. Missed bronchogenic carcinoma: radiographic findings in 27 patients with a potentially resectable lesion evident in retrospect. Radiology 1992;182(1):115–122. LinkGoogle Scholar
  • 2. Quekel LG, Kessels AG, Goei R, van Engelshoven JM. Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 1999;115(3):720–724. Crossref, MedlineGoogle Scholar
  • 3. Monnier-Cholley L, Arrivé L, Porcel A, et al. Characteristics of missed lung cancer on chest radiographs: a French experience. Eur Radiol 2001;11(4):597–605. Crossref, MedlineGoogle Scholar
  • 4. Shah PK, Austin JHM, White CS, et al. Missed non-small cell lung cancer: radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 2003;226(1):235–241. LinkGoogle Scholar
  • 5. Freedman MT, Lo SCB, Seibel JC, Bromley CM. Lung nodules: improved detection with software that suppresses the rib and clavicle on chest radiographs. Radiology 2011;260(1):265–273. LinkGoogle Scholar
  • 6. Li F, Engelmann R, Pesce LL, Doi K, Metz CE, Macmahon H. Small lung cancers: improved detection by use of bone suppression imaging—comparison with dual-energy subtraction chest radiography. Radiology 2011;261(3):937–949. LinkGoogle Scholar
  • 7. Xu Y, Ma D, He W. Assessing the use of digital radiography and a real-time interactive pulmonary nodule analysis system for large population lung cancer screening. Eur J Radiol 2012;81(4):e451–e456. Crossref, MedlineGoogle Scholar
  • 8. De Boo DW, Uffmann M, Weber M, et al. Computer-aided detection of small pulmonary nodules in chest radiographs: an observer study. Acad Radiol 2011;18(12):1507–1514. Crossref, MedlineGoogle Scholar
  • 9. de Hoop B, De Boo DW, Gietema HA, et al. Computer-aided detection of lung cancer on chest radiographs: effect on observer performance. Radiology 2010;257(2):532–540. LinkGoogle Scholar
  • 10. Kuhnigk JM, Dicken V, Bornemann L, et al. Morphological segmentation and partial volume analysis for volumetry of solid pulmonary lesions in thoracic CT scans. IEEE Trans Med Imaging 2006;25(4):417–434. Crossref, MedlineGoogle Scholar
  • 11. Dorfman DD, Berbaum KS, Metz CE. Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. Invest Radiol 1992;27(9):723–731. Crossref, MedlineGoogle Scholar
  • 12. Roe CA, Metz CE. Variance-component modeling in the analysis of receiver operating characteristic index estimates. Acad Radiol 1997;4(8):587–600. Crossref, MedlineGoogle Scholar
  • 13. Austin JH, Müller NL, Friedman PJ, et al. Glossary of terms for CT of the lungs: recommendations of the Nomenclature Committee of the Fleischner Society. Radiology 1996;200(2):327–331. LinkGoogle Scholar
  • 14. Li F, Hara T, Shiraishi J, Engelmann R, MacMahon H, Doi K. Improved detection of subtle lung nodules by use of chest radiographs with bone suppression imaging: receiver operating characteristic analysis with and without localization. AJR Am J Roentgenol 2011;196(5):W535–W541. Crossref, MedlineGoogle Scholar
  • 15. van Beek EJR, Mullan B, Thompson B. Evaluation of a real-time interactive pulmonary nodule analysis system on chest digital radiographic images: a prospective study. Acad Radiol 2008;15(5):571–575. Crossref, MedlineGoogle Scholar
  • 16. Kligerman S, Cai L, White CS. The effect of computer-aided detection on radiologist performance in the detection of lung cancers previously missed on a chest radiograph. J Thorac Imaging 2013;28(4):244–252. Crossref, MedlineGoogle Scholar
  • 17. Bley TA, Baumann T, Saueressig U, et al. Comparison of radiologist and CAD performance in the detection of CT-confirmed subtle pulmonary nodules on digital chest radiographs. Invest Radiol 2008;43(6):343–348. Crossref, MedlineGoogle Scholar
  • 18. Meziane M, Obuchowski NA, Lababede O, Lieber ML, Philips M, Mazzone P. A comparison of follow-up recommendations by chest radiologists, general radiologists, and pulmonologists using computer-aided detection to assess radiographs for actionable pulmonary nodules. AJR Am J Roentgenol 2011;196(5):W542–W549. Crossref, MedlineGoogle Scholar
  • 19. Szucs-Farkas Z, Schick A, Cullmann JL, et al. Comparison of dual-energy subtraction and electronic bone suppression combined with computer-aided detection on chest radiographs: effect on human observers’ performance in nodule detection. AJR Am J Roentgenol 2013;200(5):1006–1013. Crossref, MedlineGoogle Scholar
  • 20. Lee KH, Goo JM, Park CM, Lee HJ, Jin KN. Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers’ performance. Korean J Radiol 2012;13(5):564–571. Crossref, MedlineGoogle Scholar

Article History

Received June 7, 2013; revision requested August 7; final revision received October 25; accepted November 22; final version accepted January 9, 2014.
Published online: Mar 12 2014
Published in print: July 2014