Optimizing Analysis, Visualization, and Navigation of Large Image Data Sets: One 5000-Section CT Scan Can Ruin Your Whole Day
Abstract
An understanding of radiologists’ visual and interpretive behaviors, explored in conjunction with the capabilities of innovative advanced technologies, may provide useful new paradigms for interpreting results from medical imaging examinations.
The technology revolution in image acquisition, instrumentation, and methods has resulted in vast data sets that far outstrip the human observers’ ability to view, digest, and interpret modern medical images by using traditional methods. This may require a paradigm shift in the radiologic interpretation process. As human observers, radiologists must search for, detect, and interpret targets. Potential interventions should be based on an understanding of human perceptual and attentional abilities and limitations. New technologies and tools already in use in other fields can be adapted to the health care environment to improve medical image analysis, visualization, and navigation through large data sets. This historical psychophysical and technical review touches on a broad range of disciplines but focuses mainly on the analysis, visualization, and navigation of image data performed during the interpretive process. Advanced postprocessing, including three-dimensional image display, multimodality image fusion, quantitative measures, and incorporation of innovative human-machine interfaces, will likely be the future. Successful new paradigms will integrate image and nonimage data, incorporate workflow considerations, and be informed by evidence-based practices. This overview is meant to heighten the awareness of the complexities and limitations of how radiologists interact with images, particularly the large image sets generated today. Also addressed is how human-machine interface and informatics technologies could combine to transform the interpretation process in the future to achieve safer and better quality care for patients and a more efficient and effective work environment for radiologists.
© RSNA, 2011
Supplemental material: http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.11091276/-/DC1
References
- 1 . Holistic component of image perception in mammogram interpretation: gaze-tracking study. Radiology 2007;242(2):396–402. Link, Google Scholar
- 2 . Finding lung nodules with and without comparative visual scanning. Percept Psychophys 1981;29(6):594–598. Crossref, Medline, Google Scholar
- 3 . Using gaze-tracking data and mixture distribution analysis to support a holistic model for the detection of cancers on mammograms. Acad Radiol 2008;15(7):881–886. Crossref, Medline, Google Scholar
- 4 . Visual search patterns and experience with radiological images. Radiology 1972;103(3):523–528. Link, Google Scholar
- 5 . Cross-sectional imaging: a technology crisis upon us. J Am Coll Radiol 2006;3(3):218–219. Crossref, Medline, Google Scholar
- 6 . Addressing the coming radiology crisis: the Society for Computer Applications in Radiology transforming the radiological interpretation process (TRIP) initiative. J Digit Imaging 2004;17(4):235–243. Crossref, Medline, Google Scholar
- 7 . Paradigms of perception in clinical practice. J Am Coll Radiol 2006;3(6):441–445. Crossref, Medline, Google Scholar
- 8 . Advanced image processing in the clinical arena: issues to consider. J Am Coll Radiol 2006;3(4):296–298. Crossref, Medline, Google Scholar
- 9 . Transforming medical imaging: the first SCAR TRIP conference—a position paper from the SCAR TRIP subcommittee of the SCAR research and development committee. J Digit Imaging 2006;19(1):6–16. Crossref, Medline, Google Scholar
- 10
American College of Radiology . Practice guideline for digital radiography. Reston, Va: American College of Radiology, 2009. Google Scholar - 11
American College of Radiology . Practice guideline for determinants of image quality in digital mammography. Reston, Va: American College of Radiology, 2007. Google Scholar - 12 . Image quality in digital mammography: image acquisition. J Am Coll Radiol 2006;3(8):589–608. Crossref, Medline, Google Scholar
- 13 . Digital mammography image quality: image display. J Am Coll Radiol 2006;3(8):615–627. Crossref, Medline, Google Scholar
- 14 . Digital radiography image quality: image processing and display. J Am Coll Radiol 2007;4(6):389–400. Crossref, Medline, Google Scholar
- 15 . How to study the chest. In: Squire’s fundamentals of radiology. Cambridge, Mass: Harvard University Press, 2004; 80–91. Google Scholar
- 16 . ROC curves predicted by a model of visual search. Phys Med Biol 2006;51(14):3463–3482. Crossref, Medline, Google Scholar
- 17 . Searching for nodules: what features attract attention and influence detection? Acad Radiol 2003;10(8):861–868. Crossref, Medline, Google Scholar
- 18 . Attention in visual search: distinguishing four causes of a set size effect. Curr Dir Psychol Sci 1995;4(4):118–123. Crossref, Google Scholar
- 19 . A signal detection model predicts the effects of set size on visual search accuracy for feature, conjunction, triple conjunction, and disjunction displays. Percept Psychophys 2000;62(3):425–451. Crossref, Medline, Google Scholar
- 20 . Lesion detection and signal-to-noise ratio in CT images. Med Phys 1981;8(1):13–23. Crossref, Medline, Google Scholar
- 21 . Detection of noisy visual targets: models for the effects of spatial uncertainty and signal-to-noise ratio. Percept Psychophys 1981;29(6):521–534. Crossref, Medline, Google Scholar
- 22 . Visual search. In: Basbaum AIKaneko AShepherd GMWestheimer G, eds. The senses: a comprehensive reference. Vol 2, Vision II. San Diego, Calif: Academic Press, 2008; 275–280. Crossref, Google Scholar
- 23 . What attributes guide the deployment of visual attention and how do they do it? Nat Rev Neurosci 2004;5(6):495–501. Crossref, Medline, Google Scholar
- 24 . Satisfaction of search in diagnostic radiology. Invest Radiol 1990;25(2):133–140. Crossref, Medline, Google Scholar
- 25 . Time course of satisfaction of search. Invest Radiol 1991;26(7):640–648. Crossref, Medline, Google Scholar
- 26 . Satisfaction of search (SOS). Invest Radiol 1992;27(7):571–573. Crossref, Medline, Google Scholar
- 27 . Prevalence effect in a laboratory environment. Radiology 2003;228(1):10–14. Link, Google Scholar
- 28 . From the laboratory to the clinic: the “prevalence effect.”. Acad Radiol 2003;10(11):1324–1326. Crossref, Medline, Google Scholar
- 29 . Cognitive psychology: rare items often missed in visual searches. Nature 2005;435(7041):439–440. Crossref, Medline, Google Scholar
- 30 . Perceptual effect of CAD in reading chest radiographs. In: Samei EKrupinski E, eds. The handbook of medical image perception and techniques. Cambridge, England: Cambridge University Press, 2010; 290–303. Google Scholar
- 31 . Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 2007;31(4-5):198–211. Crossref, Medline, Google Scholar
- 32 . State-of-the-art screening for lung cancer (part 1): the chest radiograph. Thorac Surg Clin 2004;14(1):43–52. Crossref, Medline, Google Scholar
- 33 . Does computer-aided diagnosis for lung tumors change satisfaction of search in chest radiography? Acad Radiol 2007;14(9):1069–1076. Crossref, Medline, Google Scholar
- 34 . Prevalence scaling: applications to an intelligent workstation for the diagnosis of breast cancer. Acad Radiol 2008;15(11):1446–1457. Crossref, Medline, Google Scholar
- 35 . Effect of a computer-aided diagnosis scheme on radiologists’ performance in detection of lung nodules on radiographs. Radiology 1996;199(3):843–848. Link, Google Scholar
- 36 . Methods for identifying changes in radiologists’ behavioral operating point of sensitivity-specificity trade-offs within an ROC study of the use of computer aided detection of lung cancer. In: Proceedings of SPIE: medical imaging 2001—image processing and performance. Vol 4324. Bellingham, Wash: International Society for Optical Engineering, 2001; 1311–1319. Google Scholar
- 37 . Demonstration of accuracy and clinical versatility of mutual information for automatic multimodality image fusion using affine and thin-plate spline warped geometric deformations. Med Image Anal 1997;1(3):195–206. Crossref, Medline, Google Scholar
- 38 . A survey of medical image registration. Med Image Anal 1998;2(1):1–36. Crossref, Medline, Google Scholar
- 39
American College of Radiology . ACR standard for communication: diagnostic radiology—impact of reporting errors Reston, Va: American College of Radiology, 1995. Google Scholar - 40
American College of Radiology . ACR practice guideline for communication of diagnostic imaging findings Reston, Va: American College of Radiology, 2005. Google Scholar - 41
American College of Radiology . Breast Imaging Reporting and Data System (BI-RADS) atlas. http://www.acr.org/SecondaryMainMenuCategories/ACRStore/FeaturedCategories/QualityandSafety/birads_atlas.aspx. Accessed June 10, 2010. Google Scholar - 42 . Form-based clinical input from a structured vocabulary: initial application in ultrasound reporting. Proc Annu Symp Comput Appl Med Care 1992789–790. Medline, Google Scholar
- 43 . Evaluation of UltraSTAR: performance of a collaborative structured data entry system. Proc Annu Symp Comput Appl Med Care 1994216–222. Medline, Google Scholar
- 44 . ACCF/ACR/AHA/ASE/ASNC/HRS/NASCI/RSNA/SAIP/SCAI/SCCT/SCMR 2008 health policy statement on structured reporting in cardiovascular imaging. J Am Coll Cardiol 2009;53(1):76–90. Crossref, Medline, Google Scholar
- 45
RadLex . A lexicon for uniform indexing and retrieval of radiology information resources. Radiological Society of North America. http://www.rsna.org/Radlex/. Accessed June 10, 2010. Google Scholar - 46 . A framework for improving radiology reporting. J Am Coll Radiol 2005;2(2):159–167. Crossref, Medline, Google Scholar
- 47 . The radiology report of the future: a summary of the 2007 Intersociety Conference. J Am Coll Radiol 2008;5(5):626–629. Crossref, Medline, Google Scholar
- 48 . Radiology reporting, past, present, and future: the radiologist’s perspective. J Am Coll Radiol 2007;4(5):313–319. Crossref, Medline, Google Scholar
- 49 . Communicating critical test results: safe practice recommendations. Jt Comm J Qual Patient Saf 2005;31(2):68–80. Crossref, Medline, Google Scholar
- 50 . Stereoscopic and biplane imaging. In: Samei EFlynn MJ, eds. 2003 syllabus: categorical course in diagnostic radiology physics—advances in digital mammography. Oak Brook, Ill: Radiological Society of North America, 2003; 199–209. Google Scholar
- 51 . Clinical medical applications for stereoscopic 3D displays. J Soc Inf Disp 2007;15(6):377–384. Crossref, Google Scholar
- 52 . Stereoscopic digital mammography: improved accuracy of lesion detection in breast cancer screening. In: Krupinski EA, ed. Proceedings of the International Workshop on Digital Mammography, 2008; 74–79. Crossref, Google Scholar
- 53
3D Forums Stereoscopic Discussion Forums . Active versus passive 3D glasses. http://www.3d-forums.com/active-vs-passive-3d-glasses-t81.html. Accessed June 29, 2010. Google Scholar - 54
Planar Stereo Mirror . http://www.planar3d.com/. Beaverton, Ore. Accessed June 29, 2010. Google Scholar - 55
NVIDIA Corporation 3D Vision . http://www.nvidia.com/object/product_geforce_3D_VisionKit_us.html. Santa Clara, Calif. Accessed June 29, 2010. Google Scholar - 56
3D Forums Stereoscopic Discussion Forums . Autostereoscopic displays. http://www.3d-forums.com/autostereoscopic-displays-t1.html. Accessed June 29, 2010. Google Scholar - 57
Visbox. VisCube-SX Multiscreen Immersive 3D Display. http://www.visbox.com/viscube-SX.html. Champaign, Ill. Accessed June 10, 2010. Google Scholar - 58 . An immersive virtual environment for DT-MRI volume visualization applications: a case study. In: IEEE Visualization, San Diego, Calif: IEEE Computer Society, 2001; 437–584. Google Scholar
- 59 . Volumetric analysis of volumetric data: achieving a paradigm shift. Radiology 1996;200(2):312–317. Link, Google Scholar
- 60 . Informatics in radiology (infoRAD): introduction to the language of three-dimensional imaging with multidetector CT. RadioGraphics 2005;25(5):1409–1428. Link, Google Scholar
- 61 . Spiral CT: vascular applications. Eur J Radiol 1998;28(1):18–29. Crossref, Medline, Google Scholar
- 62 . 3-D imaging with MDCT. Eur J Radiol 2003;45(suppl 1):S37–S41. Crossref, Medline, Google Scholar
- 63 . Multislice imaging for three-dimensional examinations. In: Silverman PM, ed. Multislice helical tomography: a practical approach to clinical protocols. Philadelphia, Pa: Lippincott Williams * Wilkins, 2002; 317–324. Google Scholar
- 64 . Automated generation of curved planar reformations from volume data: method and evaluation. Radiology 2002;223(1):275–280. Link, Google Scholar
- 65 . STS-MIP: a new reconstruction technique for CT of the chest. J Comput Assist Tomogr 1993;17(5):832–838. Crossref, Medline, Google Scholar
- 66 . Multiplanar and three-dimensional imaging of the thorax. Radiol Clin North Am 2003;41(3):475–489. Crossref, Medline, Google Scholar
- 67 . Skeletal 3-D CT: advantages of volume rendering over surface rendering. Skeletal Radiol 1996;25(3):207–214. Crossref, Medline, Google Scholar
- 68 . Improving target delineation on 4-dimensional CT scans in stage I NSCLC using a deformable registration tool. Radiother Oncol 2010;96(1):67–72. Crossref, Medline, Google Scholar
- 69 . Intravascular functional maps of common neurovascular lesions derived from volumetric 4D CT data. Invest Radiol 2010;45(7):370–377. Crossref, Medline, Google Scholar
- 70 . Lung perfusion imaging in small animals using 4D micro-CT at heartbeat temporal resolution. Med Phys 2010;37(1):54–62. Crossref, Medline, Google Scholar
- 71 . Advances in 4D medical imaging and 4D radiation therapy. Technol Cancer Res Treat 2008;7(1):67–81. Crossref, Medline, Google Scholar
- 72 . Technical aspects of positron emission tomography/computed tomography fusion planning. Semin Nucl Med 2008;38(2):129–136. Crossref, Medline, Google Scholar
- 73 . Single-photon emission computed tomography/computed tomography in brain tumors. Semin Nucl Med 2007;37(1):34–47. Crossref, Medline, Google Scholar
- 74 . Single-photon emission computed tomography/computed tomography in abdominal diseases. Semin Nucl Med 2007;37(1):48–61. Crossref, Medline, Google Scholar
- 75 . Color holography to produce highly realistic three-dimensional images. Appl Opt 2008;47(4):A123–A133. Crossref, Medline, Google Scholar
- 76 . Automated three-dimensional tracking of living cells by digital holographic microscopy. J Biomed Opt 2009;14(1):014018. Crossref, Medline, Google Scholar
- 77 . Volumetric multiplexed transmission holography of the heart with echocardiographic data. J Am Soc Echocardiogr 1995;8(5 pt 1):567–575. Crossref, Medline, Google Scholar
- 78 . Dynamic three-dimensional echocardiography: a new era in ultrasound technology. Rev Port Cardiol 1997;16(10):787–795; 745–746. Medline, Google Scholar
- 79 . Real-time tomographic holography for augmented reality. Opt Lett 2010;35(14):2352–2354. Crossref, Medline, Google Scholar
- 80 . Optoelectronic holographic otoscope for measurement of nano-displacements in tympanic membranes. J Biomed Opt 2009;14(3):034023. Crossref, Medline, Google Scholar
- 81 . Depiction of pelvic fractures using 3D volumetric holography: comparison of plain X-ray and CT. J Comput Assist Tomogr 1995;19(6):967–974. Crossref, Medline, Google Scholar
- 82 . The applicability of holography in forensic identification: a fusion of the traditional optical technique and digital technique. J Forensic Sci 2005;50(2):393–399. Crossref, Medline, Google Scholar
- 83 . Post-processing tools for radiologists: understanding the principles, obstacles and utility of embedding advanced processing into the routine workflow in thoracic radiology [abstr]. In: Radiological Society of North America scientific assembly and annual meeting program. Oak Brook, Ill: Radiological Society of North America, 2007; 840. Google Scholar
- 84 . Advanced processing tools in radiology: what do we know, what should we know, and how can we utilize them in an evidence-based approach? [abstr]. In: Radiological Society of North America scientific assembly and annual meeting program. Oak Brook, Ill: Radiological Society of North America, 2008; 911. Google Scholar
- 85 . Alternative user interface devices for improved navigation of CT datasets. J Digit Imaging doi:10.1007/s10278-009-9252-2. Published online December 1, 2009. Medline, Google Scholar
- 86 . Ergonomics in radiology. Clin Radiol 2009;64(2):119–126. Crossref, Medline, Google Scholar
- 87 . Ergonomics of digital imaging. Br J Radiol 2005;78(931):582–586. Crossref, Medline, Google Scholar
- 88 . Radiologist assessment of PACS user interface devices. J Am Coll Radiol 2006;3(4):265–273. Crossref, Medline, Google Scholar
- 89 . Design and development of a new facility for teaching and research in clinical anatomy. Anat Sci Educ 2009;2(1):34–40. Crossref, Medline, Google Scholar
- 90 . A hand gesture sterile tool for browsing MRI images in the OR. J Am Med Inform Assoc 2008 Feb 28. [Epub ahead of print]. Google Scholar
- 91 . A gesture-based tool for sterile browsing of radiology images. J Am Med Inform Assoc 2008;15(3):321–323. [Published correction appears in J Am Med Inform Assoc 2009;16(3):284.]. Crossref, Medline, Google Scholar
- 92 . Distributing medical images with internet technologies: a DICOM web server and a DICOM java viewer. RadioGraphics 2000;20(2):581–590. Link, Google Scholar
- 93 . Computers in imaging and health care: now and in the future. J Digit Imaging 2000;13(4):145–156. Crossref, Medline, Google Scholar
- 94 . PACS and Web-based image distribution and display. Comput Med Imaging Graph 2003;27(2-3):197–206. Crossref, Medline, Google Scholar
- 95 . Integrating multiple clinical information systems using the Java message service framework to enable the delivery of urgent exam results at the point of care. J Digit Imaging 2005;18(4):316–325. Crossref, Medline, Google Scholar
- 96 . Enterprise-wide PACS: beyond radiology, an architecture to manage all medical images. Acad Radiol 2005;12(8):1000–1009. Crossref, Medline, Google Scholar
- 97 . Augmenting the impact of technology adoption with financial incentive to improve radiology report signature times. J Am Coll Radiol 2010;7(3):198–204. Crossref, Medline, Google Scholar
- 98 . Handheld computers in radiology. RadioGraphics 2003;23(4):1035–1047. Link, Google Scholar
- 99 . Radiology on handheld devices: image display, manipulation, and PACS integration issues. RadioGraphics 2004;24(1):299–310. Link, Google Scholar
- 100 . Informatics in Radiology (infoRAD): mobile wireless DICOM server system and PDA with high-resolution display—feasibility of group work for radiologists. RadioGraphics 2005;25(1):273–283. Link, Google Scholar
- 101 . Cloud computing: what is it and could it be useful? J Am Coll Radiol 2010;7(4):252–254. Crossref, Medline, Google Scholar
- 102 . Initial observations of electronic medical record usage during CT and MRI interpretation: frequency of use and impact on workflow. AJR Am J Roentgenol 2010;195(1):188–193. Crossref, Medline, Google Scholar
Article History
Received August 12, 2009; revision requested September 29; revision received August 16, 2010; accepted October 4; final version accepted October 27.Published online: May 2011
Published in print: May 2011