Effect of Directed Training on Reader Performance for CT Colonography: Multicenter Study

    Purpose: To define the interpretative performance of radiologists experienced in computed tomographic (CT) colonography and to compare it with that of novice observers who had undergone directed training, with colonoscopy as the reference standard.

    Materials and Methods: Physicians at each participating center received ethical committee approval and followed the committees' requests regarding informed consent. Nine experienced radiologists, nine trained radiologists, and 10 trained technologists from nine centers read 40 CT colonographic studies selected from a data set of 51 studies and modeled to simulate a population with positive fecal occult blood test results: Studies were obtained in eight patients with cancer, 12 patients with large polyp, four patients with medium polyp, and 27 patients without colonic lesions. Findings were verified with colonoscopy. An experienced radiologist used 50 endoscopically validated studies to train novice observers before they were allowed to participate. Observers used one software platform to read studies over 2 days. Responses were collated and compared with the known diagnostic category for each subject. The number of correctly classified subjects was determined for each observer, and differences between groups were examined with bootstrap analysis.

    Results: Overall, 28 observers read 1084 studies and detected 121 cancers, 134 large polyps, and 33 medium polyps; 448 healthy subjects were categorized correctly. Experienced radiologists detected 116 lesions; trained radiologists and technologists detected 85 and 87 lesions, respectively. Overall accuracy of experienced observers (74.2%) was significantly better than that of trained radiologists (66.6%) and technologists (63.2%). There was no significant difference (P = .33) between overall accuracy of trained radiologists and that of technologists; however, some trainees reached the mean performance achieved by experienced observers.

    Conclusion: Experienced observers interpreted CT colonographic images significantly better than did novices trained with 50 studies. On average, no difference between trained radiologists and trained technologists was found; however, individual performance was variable and some trainees outperformed some experienced observers.

    © RSNA, 2007

    References

    • 1 Kalish GM, Bhargavan M, Sunshine JH, Forman HP. Self referred whole-body imaging: where are we now? Radiology 2004; 233: 353–358. LinkGoogle Scholar
    • 2 Illes J, Fan E, Koenig BA, Raffin TA, Kann D, Atlas SW. Self-referred whole-body CT imaging: current implications for health care consumers. Radiology 2003;228:346–351. LinkGoogle Scholar
    • 3 Burling D, Halligan S, Taylor SA, Usiskin S, Bartram CI. CT colonography practice in the United Kingdom: a national survey. Clin Radiol 2004;59:39–43. Crossref, MedlineGoogle Scholar
    • 4 Halligan S, Taylor SA, Burling D. Virtual colonoscopy [letter]. JAMA 2004;292:432. Google Scholar
    • 5 Ferrucci J, Barish M, Choi R, et al. Virtual colonoscopy [letter]. JAMA 2004;292:431–432. CrossrefGoogle Scholar
    • 6 Johnson CD, Toledano AY, Herman BA, et al. Computerized tomographic colonography: performance evaluation in a retrospective multicenter setting. Gastroenterology 2003;125:688–695. Crossref, MedlineGoogle Scholar
    • 7 Soto JA, Barish MA, Ferrucci JT. CT colonography interpretation: guidelines for training courses [abstr]. In: Radiological Society of North America scientific assembly and annual meeting program. Oak Brook, Ill: Radiological Society of North America, 2004; SSQ09-07. Google Scholar
    • 8 Taylor SA, Halligan S, Burling D, et al. CT colonography: effect of experience and training on reader performance. Eur Radiol 2004;14:1025–1033. Crossref, MedlineGoogle Scholar
    • 9 Rockey DC, Paulson E, Niedzwiecki D, et al. Prospective comparison of colon imaging tests: a determination of the relative sensitivity of air contrast barium enema, computed tomographic colonography, and colonoscopy. Lancet 2005;365:305–311. Crossref, MedlineGoogle Scholar
    • 10 Hardcastle JD, Chamberlain JO, Robinson MH, et al. Randomised controlled trial of faecal-occult-blood screening for colorectal cancer. Lancet 1996;348:1472–1477. Crossref, MedlineGoogle Scholar
    • 11 Kronborg O, Fenger C, Olsen J, Jorgensen OD, Sondergaard O. Randomised study of screening for colorectal cancer with faecal-occult-blood test. Lancet 1996;348:1467–1471. Crossref, MedlineGoogle Scholar
    • 12 Mandel JS, Bond JH, Church TR, et al. Reducing mortality from colorectal cancer by screening for fecal occult blood. N Engl J Med 1993;328:1365–1371. Crossref, MedlineGoogle Scholar
    • 13 Halligan S. Subspecialist radiology. Clin Radiol 2002;57:982–983. Crossref, MedlineGoogle Scholar
    • 14 Alderson PO. A balanced subspecialization strategy for radiology in the new millennium. AJR Am J Roentgenol 2000;175:7–8. Crossref, MedlineGoogle Scholar
    • 15 Capp MP. Subspecialization in radiology. AJR Am J Roentgenol 1990;155:451–454. Crossref, MedlineGoogle Scholar
    • 16 Halligan S, Marshall M, Taylor SA, et al. Observer variation in the detection of colorectal neoplasia on double contrast barium enema: implications for colorectal cancer screening and training. Clin Radiol 2003;58:948–954. Crossref, MedlineGoogle Scholar
    • 17 Rex DK, Vining D, Kopecky KK. An initial experience with screening for colon polyps using spiral CT with and without CT colography (virtual colonoscopy). Gastrointest Endosc 1999;50:309–313. Crossref, MedlineGoogle Scholar
    • 18 Cotton PB, Durkalski VL, Pineau BC, et al. Computed tomographic colonography (virtual colonoscopy): a multicenter comparison with standard colonoscopy for detection of colorectal neoplasia. JAMA 2004;291:1713–1719. Crossref, MedlineGoogle Scholar
    • 19 Sickles EA, Wolverton DE, Dee KE. Performance parameters for screening and diagnostic mammography: specialist and general radiologists. Radiology 2002;224:861–869. LinkGoogle Scholar
    • 20 Johnson CD, Harmsen WS, Wilson LA, et al. Prospective blinded evaluation of computed tomographic colonography for screen detection of colorectal polyps. Gastroenterology 2003;125:311–319. Crossref, MedlineGoogle Scholar
    • 21 Culpan DG, Mitchell AJ, Hughes S, Nutman M, Chapman AH. Double contrast barium enema sensitivity: a comparison of studies by radiographers and radiologists. Clin Radiol 2002;57:604–607. Crossref, MedlineGoogle Scholar
    • 22 Brown L, Desai S. Cost-effectiveness of barium enemas performed by radiographers. Clin Radiol 2002;57:129–131. Crossref, MedlineGoogle Scholar
    • 23 Pickhardt PJ, Choi JR, Hwang I, et al. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. N Engl J Med 2003;349:2191–2200. Crossref, MedlineGoogle Scholar
    • 24 Sosna J, Morrin MM, Kruskal JB, Lavin PT, Rosen MP, Raptopoulos V. CT colonography of colorectal polyps: a metaanalysis. AJR Am J Roentgenol 2003;181:1593–1598. Crossref, MedlineGoogle Scholar
    • 25 Halligan S, Altman DG, Taylor SA, et al. CT colonography in the detection of colorectal polyps and cancer: systematic review, meta-analysis, and proposed minimum data set for study level reporting. Radiology 2005;237:893–904. LinkGoogle Scholar
    • 26 Rembacken BJ, Fujii T, Cairns A, et al. Flat and depressed colonic neoplasms: a prospective study of 1000 colonoscopies in the UK. Lancet 2000;355:1211–1214. Crossref, MedlineGoogle Scholar
    • 27 Suzuki N, Talbot IC, Saunders BP. The prevalence of small, flat, colorectal cancers in a Western population. Colorectal Dis 2004;6:15–20 Crossref, MedlineGoogle Scholar
    • 28 Fidler JL, Johnson CD, MacCarty RL, Welch TJ, Hara AK, Harmsen WS. Detection of flat lesions in the colon with CT colonography. Abdom Imaging 2002;27:292–300. Crossref, MedlineGoogle Scholar
    • 29 Fidler JL, Fletcher JG, Johnson CD, et al. Understanding interpretative errors in radiologists learning computed tomography colonography. Acad Radiol 2004;11:750–756. Crossref, MedlineGoogle Scholar

    Article History

    Published in print: 2007