CT Colonography: A Systematic Review of Standard of Reporting for Studies of Computer-aided Detection

Charlotte Robinson

Steve Halligan

Stuart A. Taylor

Susan Mallett

Douglas G. Altman

Author Affiliations

Published Online:Feb 1 2008https://doi.org/10.1148/radiol.2461070121

Abstract

Purpose: To determine objectively the current standard of reporting for studies of computer-aided detection (CAD) for computed tomographic (CT) colonography by systematically reviewing published articles.

Materials and Methods: MEDLINE was searched to identify study articles meeting the inclusion criteria for describing CAD for CT colonography in human subjects. Data were extracted from eligible articles, grouped into five domains: technical description of CAD algorithm, description of subjects, acquisition of data, evaluation strategy used, and presentation of results. Primary studies were scored for each domain and overall findings plotted as star plots.

Results: Although 21 (91%) of the 23 studies included presented technical details of the CAD algorithm, methodologic details used for model development and validity were generally poor. Investigators in six (26%) studies described the evaluation data set sufficiently for replication; investigators in eight (35%) studies described age and sex demographics for subjects in whom CAD was tested. Investigators in 11 (48%) studies presented polyps per subject. Investigators in 12 (52%) studies described the reference standard against which CAD was judged; 11 (48%) studies explicitly distinguished between development and evaluation data. In nine (39%) studies, the evaluation strategy used to test CAD could not be deduced at all. Description of subjects included for CAD development and evaluation was most poorly reported, with an average score per study of 33% in this domain.

Conclusion: The reporting quality for studies of CAD for CT colonography is highly variable; key methodologic details needed for informed assessment of the generalizability of results are frequently omitted, for which a minimum data set based on the observations is proposed.

References

1 Macari M, Bini EJ. CT colonography: where have we been and where are we going? Radiology 2005; 237: 819–833. Link, Google Scholar
2 Burling D, Taylor SA, Halligan S. Virtual colonoscopy: current status and future directions. Gastrointest Endosc Clin N Am 2005;15:773–795. Crossref, Medline, Google Scholar
3 Fenlon HM, Nunes DP, Schroy PC 3rd, Barish MA, Clarke PD, Ferrucci JT. A comparison of virtual and conventional colonoscopy for the detection of colorectal polyps. N Engl J Med 1999;341:1496–1503. Crossref, Medline, Google Scholar
4 Yee J, Akerkar GA, Hung RK, Steinauer-Gebauer AM, Wall SD, McQuaid KR. Colorectal neoplasia: performance characteristics of CT colonography for detection in 300 patients. Radiology 2001;219:685–692. Link, Google Scholar
5 Van Gelder RE, Nio CY, Florie J, et al. Computed tomographic colonography compared with colonoscopy in patients at increased risk for colorectal cancer. Gastroenterology 2004;127:41–48. Crossref, Medline, Google Scholar
6 Pickhardt PJ, Choi JR, Hwang I, et al. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. N Engl J Med 2003;349:2191–2200. Crossref, Medline, Google Scholar
7 Cotton PB, Durkalski VL, Pineau BC, et al. Computed tomographic colonography (virtual colonoscopy): a multicenter comparison with standard colonoscopy for detection of colorectal neoplasia. JAMA 2004;291:1713–1719. Crossref, Medline, Google Scholar
8 Rockey DC, Paulson E, Niedzwiecki D, et al. Analysis of air contrast barium enema, computed tomographic colonography, and colonoscopy: prospective comparison. Lancet 2005;365:305–311. Crossref, Medline, Google Scholar
9 Johnson CD, Harmsen WS, Wilson LA, et al. Prospective blinded evaluation of computed tomographic colonography for screen detection of colorectal polyps. Gastroenterology 2003;125:311–319. Crossref, Medline, Google Scholar
10 Halligan S, Taylor SA, Burling D. Virtual colonoscopy. JAMA 2004;292:432. Google Scholar
11 Ferrucci J, Barish M, Choi R, et al. Virtual colonoscopy. JAMA 2004;292:431–432. Crossref, Google Scholar
12 Warren Burhenne LJ, Wood SA, D'Orsi CJ, et al. Potential contribution of computer aided detection to the sensitivity of screening mammography. Radiology 2000;215:554–562. Link, Google Scholar
13 Awai K, Murao K, Ozawa A, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. Radiology 2004;230:347–352. Link, Google Scholar
14 Yoshida H, Dachman AH. CAD techniques, challenges, and controversies in computed tomographic colonography. Abdom Imaging 2005;30:26–41. Crossref, Medline, Google Scholar
15 Bond JH. Progress in refining virtual colonoscopy for colorectal cancer screening. Gastroenterology 2005;129:2103–2106. Crossref, Medline, Google Scholar
16 Summers RM, Yao J, Pickhardt P, et al. Computed tomographic virtual colonoscopy computer-aided polyp detection in a screening population. Gastroenterology 2005;129:1832–1844. Crossref, Medline, Google Scholar
17 Wagner RF, Beiden SV, Campbell G, et al. Assessment of medical imaging and computer-assist systems: lessons from recent experience. Acad Radiol 2002;9:1264–1277. Crossref, Medline, Google Scholar
18 Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med 2000;19:453–473. Crossref, Medline, Google Scholar
19 Hirsch RP. Validation samples. Biometrics 1991;47:1193–1194. Medline, Google Scholar
20 Partain CL, Chan HP, Gelovani JG, et al. Biomedical imaging research opportunities workshop 2: report and recommendations. Radiology 2005;236:389–403. Link, Google Scholar
21 Halligan S, Taylor SA, Dehmeshki J, et al. Computer assisted detection for CT colonography: external validation. Clin Radiol 2006;61:758–763. Crossref, Medline, Google Scholar
22 Mallett S, Deeks JJ, Halligan S, Hopewell S, Cornelius V, Altman DG. Systematic reviews of diagnostic tests in cancer: systematic review of methodology and reporting. BMJ 2006;333:413–416. Crossref, Medline, Google Scholar
23 Gokturk SB, Tomasi C, Acar B, et al. A statistical 3-D pattern processing method for computer-aided detection of polyps in CT colonography. IEEE Trans Med Imaging 2001;20:1251–1260. Crossref, Medline, Google Scholar
24 Nappi J, Yoshida H. Automated detection of polyps with CT colonography: evaluation of volumetric features for reduction of false-positive findings. Acad Radiol 2002;9:386–397. Crossref, Medline, Google Scholar
25 Nappi J, Dachman AH, MacEneaney P, Yoshida H. Automated knowledge-guided segmentation of colonic walls for computerized detection of polyps in CT colonography. J Comput Assist Tomogr 2002;26:493–504. Crossref, Medline, Google Scholar
26 Li H, Santago P. Automatic colon segmentation with dual scan CT colonography. J Digit Imaging 2005;18:42–54. Crossref, Medline, Google Scholar
27 Yao J, Miller M, Franaszek M, Summers RM. Colonic polyp segmentation in CT colonography-based on fuzzy clustering and deformable models. IEEE Trans Med Imaging 2004;23:1344–1352. Crossref, Medline, Google Scholar
28 Summers RM, Jerebko AK, Franaszek M, Malley JD, Johnson CD. Colonic polyps: complementary role of computer-aided detection in CT colonography. Radiology 2002;225:391–399. Link, Google Scholar
29 Mani A, Napel S, Paik DS, et al. Computed tomography colonography: feasibility of computer-aided polyp detection in a “first reader” paradigm. J Comput Assist Tomogr 2004;28:318–326. Crossref, Medline, Google Scholar
30 Bogoni L, Cathier P, Dundar M, et al. Computer-aided detection (CAD) for CT colonography: a tool to address a growing need. Br J Radiol 2005;78(suppl 1):S57–S62. Crossref, Medline, Google Scholar
31 Summers RM, Franaszek M, Miller MT, Pickhardt PJ, Choi JR, Schindler WR. Computer-aided detection of polyps on oral contrast-enhanced CT colonography. AJR Am J Roentgenol 2005;184:105–108. Crossref, Medline, Google Scholar
32 Luboldt W, Mann C, Tyron CL, et al. Computer-aided diagnosis in contrast-enhanced CT colonography: an approached based on contrast. Eur Radiol 2002;12:2236–2241. Crossref, Medline, Google Scholar
33 Kiss G, Van Cleynenbreugel J, Thomeer M, Suetens P, Marchal G. Computer-aided diagnosis in virtual colonography via combination of surface normal and sphere fitting methods. Eur Radiol 2002;12:77–81. Crossref, Medline, Google Scholar
34 Yoshida H, Nappi J, MacEneaney P, Rubin DT, Dachman AH. Computer-aided diagnosis scheme for detection of polyps at CT colonography. RadioGraphics 2002;22:963–979. Link, Google Scholar
35 Jerebko AK, Summers RM, Malley JD, Franaszek M, Johnson CD. Computer-assisted detection of colonic polyps with CT colonography using neural networks and binary classification trees. Med Phys 2003;30:52–60. Crossref, Medline, Google Scholar
36 Yoshida H, Masutani Y, MacEneaney P, Rubin DT, Dachman AH. Computerized detection of colonic polyps at CT colonography on the basis of volumetric features: pilot study. Radiology 2002;222:327–336. Link, Google Scholar
37 Nappi JJ, Frimmel H, Dachman AH, Yoshida H. Computerized detection of colorectal masses in CT colonography based on fuzzy merging and wall-thickening analysis. Med Phys 2004;31:860–872. Crossref, Medline, Google Scholar
38 Acar B, Beaulieu CF, Gokturk SB, et al. Edge displacement field-based classification for improved detection of polyps in CT colonography. IEEE Trans Med Imaging 2002;21:1461–1467. Crossref, Medline, Google Scholar
39 Nappi J, Yoshida H. Feature-guided analysis for reduction of false positives in CAD of polyps for computed tomographic colonography. Med Phys 2003;30:1592–1601. Crossref, Medline, Google Scholar
40 Jerebko AK, Malley JD, Franaszek M, Summers RM. Multiple neural network classification scheme for detection of colonic polyps in CT colonography data sets. Acad Radiol 2003;10:154–160. Crossref, Medline, Google Scholar
41 Nappi J, Okamura A, Frimmel H, Dachman A, Yoshida H. Region-based supine-prone correspondence for the reduction of false-positive CAD polyp candidates in CT colonography. Acad Radiol 2005;12:695–707. Crossref, Medline, Google Scholar
42 Jerebko AK, Malley JD, Franaszek M, Summers RM. Support vector machines committee classification method for computer-aided polyp detection in CT colonography. Acad Radiol 2005;12:479–486. Crossref, Medline, Google Scholar
43 Paik DS, Beaulieu CF, Rubin GD, et al. Surface normal overlap: a computer-aided detection algorithm with application to colonic polyps and lung nodules in helical CT. IEEE Trans Med Imaging 2004;23:661–675. Crossref, Medline, Google Scholar
44 Yoshida H, Nappi J. Three-dimensional computer-aided diagnosis scheme for detection of colonic polyps. IEEE Trans Med Imaging 2001;20:1261–1274. Crossref, Medline, Google Scholar
45 Deeks JJ. Systematic reviews of evaluations of diagnostic and screening tests. BMJ 2001;323:157–162. Crossref, Medline, Google Scholar
46 Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD Initiative. Radiology 2003;226(1):24–28. Link, Google Scholar
47 Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3:25. Crossref, Medline, Google Scholar
48 Dachman AH, Zalis ME. Quality and consistency in CT colonography and research reporting. Radiology 2004;230:319–323. Link, Google Scholar
49 Halligan S, Altman DG, Taylor SA, et al. CT colonography in the detection of colorectal polyps and cancer: systematic review, meta-analysis, and proposed minimum data set for study level reporting. Radiology 2005;237:893–904. Link, Google Scholar

Article History

Published in print: 2008

Vol. 246, No. 2

Metrics

Downloaded 287 times

Altmetric Score