How to Critically Appraise and Interpret Systematic Reviews and Meta-Analyses of Diagnostic Accuracy: A User Guide

Published Online:

This article provides a practical guide for clinicians and radiologists to critically evaluate diagnostic accuracy systematic reviews when using them to inform clinical decisions.

Systematic reviews of diagnostic accuracy studies can provide the best available evidence to inform decisions regarding the use of a diagnostic test. In this guide, the authors provide a practical approach for clinicians to appraise diagnostic accuracy systematic reviews and apply their results to patient care. The first step is to identify an appropriate systematic review with a research question matching the clinical scenario. The user should evaluate the rigor of the review methods to evaluate its credibility (Did the review use clearly defined eligibility criteria, a comprehensive search strategy, structured data collection, risk of bias and applicability appraisal, and appropriate meta-analysis methods?). If the review is credible, the next step is to decide whether the diagnostic performance is adequate for clinical use (Do sensitivity and specificity estimates exceed the threshold that makes them useful in clinical practice? Are these estimates sufficiently precise? Is variability in the estimates of diagnostic accuracy across studies explained?). Diagnostic accuracy systematic reviews that are judged to be credible and provide diagnostic accuracy estimates with sufficient certainty and relevance are the most useful to inform patient care. This review discusses comparative, noncomparative, and emerging approaches to systematic reviews of diagnostic accuracy using a clinical scenario and examples based on recent publications.

© RSNA, 2023


  • 1. Salameh JP, Bossuyt PM, McGrath TA, et al. Preferred reporting items for systematic review and meta-analysis of diagnostic test accuracy studies (PRISMA-DTA): explanation, elaboration, and checklist. BMJ 2020;370:m2632. Crossref, MedlineGoogle Scholar
  • 2. Cohen JF, Deeks JJ, Hooft L, et al. Preferred reporting items for journal and conference abstracts of systematic reviews and meta-analyses of diagnostic test accuracy studies (PRISMA-DTA for Abstracts): checklist, explanation, and elaboration. BMJ 2021;372:n265. Crossref, MedlineGoogle Scholar
  • 3. Frank RA, McInnes MDF, Levine D, et al. Are Study and Journal Characteristics Reliable Indicators of “Truth” in Imaging Research? Radiology 2018;287(1):215–223. LinkGoogle Scholar
  • 4. Higgins JPT, Thomas J. Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions version 6.0. (updated July 2019). Cochrane, 2019. Google Scholar
  • 5. Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA 2005;293(19):2362–2366. Crossref, MedlineGoogle Scholar
  • 6. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E, eds. Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Press, 2011. CrossrefGoogle Scholar
  • 7. Murad MH, Montori VM, Ioannidis JP, et al. How to read a systematic review and meta-analysis and apply the results to patient care: users’ guides to the medical literature. JAMA 2014;312(2):171–179. Crossref, MedlineGoogle Scholar
  • 8. Halligan S, Altman DG. Evidence-based practice in radiology: steps 3 and 4--appraise and apply systematic reviews and meta-analyses. Radiology 2007;243(1):13–27. LinkGoogle Scholar
  • 9. Deeks J, Bossuyt P, Leeflang M, Takwoingi Y, Flemyng E. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2.0. London: Cochrane, 2022. Accessed September 2022. Google Scholar
  • 10. University of Oxford. Systematic Reviews - Centre for Evidence-Based Medicine. 2022. Accessed September 2022. Google Scholar
  • 11. Balogh EP, Miller BT, Ball JR, eds. Committee on Diagnostic Error in Health Care; Board on Health Care Services; Institute of Medicine; The National Academies of Sciences, Engineering, and Medicine. Improving Diagnosis in Health Care. Washington, DC: National Academies Press, 2015. Google Scholar
  • 12. Singh H, Graber ML. Improving Diagnosis in Health Care--The Next Imperative for Patient Safety. N Engl J Med 2015;373(26):2493–2495. Crossref, MedlineGoogle Scholar
  • 13. McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM. Meta-Analyses of Diagnostic Accuracy in Imaging Journals: Analysis of Pooling Techniques and Their Effect on Summary Estimates of Diagnostic Accuracy. Radiology 2016;281(1):78–85. LinkGoogle Scholar
  • 14. McGrath TA, Alabousi M, Skidmore B, et al. Recommendations for reporting of systematic reviews and meta-analyses of diagnostic test accuracy: a systematic review. Syst Rev 2017;6(1):194. Crossref, MedlineGoogle Scholar
  • 15. McGrath TA, McInnes MDF, van Es N, Leeflang MMG, Korevaar DA, Bossuyt PMM. Overinterpretation of Research Findings: Evidence of “Spin” in Systematic Reviews of Diagnostic Accuracy Studies. Clin Chem 2017;63(8):1353–1362. Crossref, MedlineGoogle Scholar
  • 16. McGrath TA, McInnes MDF, Langer FW, Hong J, Korevaar DA, Bossuyt PMM. Treatment of multiple test readers in diagnostic accuracy systematic reviews-meta-analyses of imaging studies. Eur J Radiol 2017;93:59–64. Crossref, MedlineGoogle Scholar
  • 17. McInnes MD, Bossuyt PM. Pitfalls of Systematic Reviews and Meta-Analyses in Imaging Research. Radiology 2015;277(1):13–21. LinkGoogle Scholar
  • 18. McInnes MDF, Moher D, Thombs BD, et althe PRISMA-DTA Group. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA 2018;319(4):388–396 [Published correction appears in JAMA 2019;322(20):2026.]. Crossref, MedlineGoogle Scholar
  • 19. Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM; Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Ann Intern Med 2008;149(12):889–897. Crossref, MedlineGoogle Scholar
  • 20. Tunis AS, McInnes MD, Hanna R, Esmail K. Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology 2013;269(2):413–426. LinkGoogle Scholar
  • 21. Willis BH, Quigley M. Uptake of newer methodological developments and the deployment of meta-analysis in diagnostic test research: a systematic review. BMC Med Res Methodol 2011;11(1):27. Crossref, MedlineGoogle Scholar
  • 22. Willis BH, Quigley M. The assessment of the quality of reporting of meta-analyses in diagnostic research: a systematic review. BMC Med Res Methodol 2011;11(1):163. Crossref, MedlineGoogle Scholar
  • 23. McGrath TA, Bowdridge JC, Prager R, et al. Overinterpretation of Research Findings: Evaluation of “Spin” in Systematic Reviews of Diagnostic Accuracy Studies in High-Impact Factor Journals. Clin Chem 2020;66(7):915–924. Crossref, MedlineGoogle Scholar
  • 24. Duke E, Kalb B, Arif-Tiwari H, et al. A Systematic Review and Meta-Analysis of Diagnostic Performance of MRI for Evaluation of Acute Appendicitis. AJR Am J Roentgenol 2016;206(3):508–517. Crossref, MedlineGoogle Scholar
  • 25. Salameh JP, McInnes MDF, Moher D, et al. Completeness of Reporting of Systematic Reviews of Diagnostic Test Accuracy Based on the PRISMA-DTA Reporting Guideline. Clin Chem 2019;65(2):291–301. Crossref, MedlineGoogle Scholar
  • 26. Hong PJ, Korevaar DA, McGrath TA, et al. Reporting of imaging diagnostic accuracy studies with focus on MRI subgroup: Adherence to STARD 2015. J Magn Reson Imaging 2018;47(2):523–544. Crossref, MedlineGoogle Scholar
  • 27. Whiting P, Savović J, Higgins JP, et alROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016;69:225–234. Crossref, MedlineGoogle Scholar
  • 28. Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999;282(11):1061–1066. Crossref, MedlineGoogle Scholar
  • 29. de Vet HCW, Eisinga A, Riphagen II, Aertgeerts B, Pewsner D. Searching for Studies. In: Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.4, Chapter 7. London, UK: The Cochrane Collaboration, 2008. [updated September 2008.] Google Scholar
  • 30. Whiting P, Westwood M, Beynon R, Burke M, Sterne JA, Glanville J. Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies. J Clin Epidemiol 2011;64(6):602–607. Crossref, MedlineGoogle Scholar
  • 31. Beynon R, Leeflang MM, McDonald S, et al. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev 2013;2013(9):MR000022. MedlineGoogle Scholar
  • 32. Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health Technol Assess 2003;7(41):1–90. Crossref, MedlineGoogle Scholar
  • 33. Whiting PF, Rutjes AW, Westwood ME, et alQUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155(8):529–536. Crossref, MedlineGoogle Scholar
  • 34. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58(10):982–990. Crossref, MedlineGoogle Scholar
  • 35. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 2001;20(19):2865–2884. Crossref, MedlineGoogle Scholar
  • 36. Steinhauser S, Schumacher M, Rücker G. Modelling multiple thresholds in meta-analysis of diagnostic test accuracy studies. BMC Med Res Methodol 2016;16(1):97. Crossref, MedlineGoogle Scholar
  • 37. Bossuyt PM, Takwoingi Y, Deeks J, Gatsonis C. Chapter 10: Understanding meta-analysis Draft version. (March 28, 2022). In: Deeks JJ, Bossuyt PM, Leeflang MM, Takwoingi Y, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2. London, England: Cochrane. Google Scholar
  • 38. Ebrahimzadeh S, Islam N, Dawit H, et al; Cochrane COVID-19 Diagnostic Test Accuracy Group. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2022;5:CD013639. MedlineGoogle Scholar
  • 39. Rücker G, Schwarzer G, Carpenter JR, Schumacher M. Undue reliance on I(2) in assessing heterogeneity may mislead. BMC Med Res Methodol 2008;8(1):79. Crossref, MedlineGoogle Scholar
  • 40. Zhou Y, Dendukuri N. Statistics for quantifying heterogeneity in univariate and bivariate meta-analyses of binary data: the case of meta-analyses of diagnostic accuracy. Stat Med 2014;33(16):2701–2717. Crossref, MedlineGoogle Scholar
  • 41. Kuo RYL, Harrison C, Curran TA, et al. Artificial Intelligence in Fracture Detection: A Systematic Review and Meta-Analysis. Radiology 2022;304(1):50–62. LinkGoogle Scholar
  • 42. Ghazi Sherbaf F, Sair HI, Shakoor D, et al. DECT in Detection of Vertebral Fracture-associated Bone Marrow Edema: A Systematic Review and Meta-Analysis with Emphasis on Technical and Imaging Interpretation Parameters. Radiology 2021;300(1):110–119. LinkGoogle Scholar
  • 43. Eng KA, Abadeh A, Ligocki C, et al. Acute Appendicitis: A Meta-Analysis of the Diagnostic Accuracy of US, CT, and MRI as Second-Line Imaging Tests after an Initial US. Radiology 2018;288(3):717–727. LinkGoogle Scholar
  • 44. Leeflang MMG, Reitsma JB. Systematic reviews and meta-analyses addressing comparative test accuracy questions. Diagn Progn Res 2018;2(1):17. Crossref, MedlineGoogle Scholar
  • 45. Takwoingi Y, Partlett C, Riley RD, Hyde C, Deeks JJ. Methods and reporting of systematic reviews of comparative accuracy were deficient: a methodological survey and proposed guidance. J Clin Epidemiol 2020;121:1–14. Crossref, MedlineGoogle Scholar
  • 46. Dehmoobad Sharifabadi A, Leeflang M, Treanor L, et al. Comparative reviews of diagnostic test accuracy in imaging research: evaluation of current practices. Eur Radiol 2019;29(10):5386–5394. Crossref, MedlineGoogle Scholar
  • 47. Yang B, Vali Y, Dehmoobad Sharifabadi A, et al. Risk of bias assessment of test comparisons was uncommon in comparative accuracy systematic reviews: an overview of reviews. J Clin Epidemiol 2020;127:167–174. Crossref, MedlineGoogle Scholar
  • 48. Takwoingi Y, Leeflang MM, Deeks JJ. Empirical evidence of the importance of comparative studies of diagnostic test accuracy. Ann Intern Med 2013;158(7):544–554. Crossref, MedlineGoogle Scholar
  • 49. Yang B, Olsen M, Vali Y, et al. Study designs for comparative diagnostic test accuracy: A methodological review and classification scheme. J Clin Epidemiol 2021;138:128–138. Crossref, MedlineGoogle Scholar
  • 50. Yang B, Mallett S, Takwoingi Y, et alQUADAS-C Group†. QUADAS-C: A Tool for Assessing Risk of Bias in Comparative Diagnostic Accuracy Studies. Ann Intern Med 2021;174(11):1592–1599. Crossref, MedlineGoogle Scholar
  • 51. Chan KK, Joo DA, McRae AD, et al. Chest ultrasonography versus supine chest radiography for diagnosis of pneumothorax in trauma patients in the emergency department. Cochrane Database Syst Rev 2020;7:CD013031. MedlineGoogle Scholar
  • 52. Macaskill P, Gatsonis C, Deeks JJ, Harbord RM, Takwoingi Y. Analysing and Presenting Results. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0, Chapter 10. London, UK: The Cochrane Collaboration, 2010. Google Scholar
  • 53. Takwoingi Y, Riley RD, Deeks JJ. Meta-analysis of diagnostic accuracy studies in mental health. Evid Based Ment Health 2015;18(4):103–109. Crossref, MedlineGoogle Scholar
  • 54. Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004;23(20):3105–3124. Crossref, MedlineGoogle Scholar
  • 55. Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med 2015;162(11):777–784. Crossref, MedlineGoogle Scholar
  • 56. Zarin W, Veroniki AA, Nincic V, et al. Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review. BMC Med 2017;15(1):3 [Published correction appears in BMC Med 2017;15(1):61.]. Crossref, MedlineGoogle Scholar
  • 57. Dias S, Sutton AJ, Ades AE, Welton NJ. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Making 2013;33(5):607–617. Crossref, MedlineGoogle Scholar
  • 58. Ades AE, Caldwell DM, Reken S, Welton NJ, Sutton AJ, Dias S. Evidence synthesis for decision making 7: a reviewer’s checklist. Med Decis Making 2013;33(5):679–691. Crossref, MedlineGoogle Scholar
  • 59. Dias S, Sutton AJ, Welton NJ, Ades AE. Evidence synthesis for decision making 3: heterogeneity--subgroups, meta-regression, bias, and bias-adjustment. Med Decis Making 2013;33(5):618–640. Crossref, MedlineGoogle Scholar
  • 60. Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials. Med Decis Making 2013;33(5):641–656. Crossref, MedlineGoogle Scholar
  • 61. Puhan MA, Schünemann HJ, Murad MH, et alGRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ 2014;349:g5630. Crossref, MedlineGoogle Scholar
  • 62. Brignardello-Petersen R, Bonner A, Alexander PE, et alGRADE Working Group. Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis. J Clin Epidemiol 2018;93:36–44 [Published correction appears in J Clin Epidemiol 2018;98:162.]. Crossref, MedlineGoogle Scholar
  • 63. Jansen JP, Trikalinos T, Cappelleri JC, et al. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health 2014;17(2):157–173. Crossref, MedlineGoogle Scholar
  • 64. Filipe MD, Patuleia SIS, de Jong VMT, Vriens MR, van Diest PJ, Witkamp AJ. Network Meta-analysis for the Diagnostic Approach to Pathologic Nipple Discharge. Clin Breast Cancer 2020;20(6):e723–e748. Crossref, MedlineGoogle Scholar
  • 65. Veroniki AA, Tsokani S, Rücker G, Mavridis D, Takwoingi Y. Challenges in Comparative Meta-Analysis of the Accuracy of Multiple Diagnostic Tests. Methods Mol Biol 2022;2345:299–316. Crossref, MedlineGoogle Scholar
  • 66. Veroniki A, Tsokani S, Paraskevaidas E, Mavridis D. Evaluating multiple diagnostic tests: an application to cervical cancer. HJOG 2021;20(1):11–24. CrossrefGoogle Scholar
  • 67. Owen RK, Cooper NJ, Quinn TJ, Lees R, Sutton AJ. Network meta-analysis of diagnostic test accuracy studies identifies and ranks the optimal diagnostic tests and thresholds for health care policy and decision-making. J Clin Epidemiol 2018;99:64–74. Crossref, MedlineGoogle Scholar
  • 68. Menten J, Lesaffre E. A general framework for comparative Bayesian meta-analysis of diagnostic studies. BMC Med Res Methodol 2015;15(1):70. Crossref, MedlineGoogle Scholar
  • 69. Trikalinos TA, Hoaglin DC, Small KM, Terrin N, Schmid CH. Methods for the joint meta-analysis of multiple tests. Res Synth Methods 2014;5(4):294–312. Crossref, MedlineGoogle Scholar
  • 70. Nyaga VN, Aerts M, Arbyn M. ANOVA model for network meta-analysis of diagnostic test accuracy data. Stat Methods Med Res 2018;27(6):1766–1784. Crossref, MedlineGoogle Scholar
  • 71. Veroniki AA, Tsokani S, Agarwal R, et al. Diagnostic test accuracy network meta-analysis methods: A scoping review and empirical assessment. J Clin Epidemiol 2022;146:86–96. Crossref, MedlineGoogle Scholar
  • 72. van der Pol CB, McInnes MDF, Salameh JP, et al. CT/MRI and CEUS LI-RADS Major Features Association with Hepatocellular Carcinoma: Individual Patient Data Meta-Analysis. Radiology 2022;302(2):326–335. LinkGoogle Scholar
  • 73. van der Pol CB, McInnes MDF, Salameh JP, et al; LI-RADS IPD Group Collaborators. Impact of Reference Standard on CT, MRI, and Contrast-enhanced US LI-RADS Diagnosis of Hepatocellular Carcinoma: A Meta-Analysis. Radiology 2022;303(3):544–545. LinkGoogle Scholar
  • 74. van der Pol CB, Lim CS, Sirlin CB, et al. Accuracy of the Liver Imaging Reporting and Data System in Computed Tomography and Magnetic Resonance Image Analysis of Hepatocellular Carcinoma or Overall Malignancy-A Systematic Review. Gastroenterology 2019;156(4):976–986. Crossref, MedlineGoogle Scholar
  • 75. Levis B, Benedetti A, Levis AW, et al. Selective Cutoff Reporting in Studies of Diagnostic Test Accuracy: A Comparison of Conventional and Individual-Patient-Data Meta-Analyses of the Patient Health Questionnaire-9 Depression Screening Tool. Am J Epidemiol 2017;185(10):954–964. Crossref, MedlineGoogle Scholar
  • 76. Singh S, Venkatesh SK, Wang Z, et al. Diagnostic performance of magnetic resonance elastography in staging liver fibrosis: a systematic review and meta-analysis of individual participant data. Clin Gastroenterol Hepatol 2015;13(3):440–451.e6. Crossref, MedlineGoogle Scholar
  • 77. Petroff D, Blank V, Newsome PN, et al. Assessment of hepatic steatosis by controlled attenuation parameter using the M and XL probes: an individual patient data meta-analysis. Lancet Gastroenterol Hepatol 2021;6(3):185–198. Crossref, MedlineGoogle Scholar
  • 78. Stewart LA, Clarke M, Rovers M, et alPRISMA-IPD Development Group. Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data: the PRISMA-IPD Statement. JAMA 2015;313(16):1657–1665. Crossref, MedlineGoogle Scholar
  • 79. Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ 2010;340(feb05 1):c221. Crossref, MedlineGoogle Scholar
  • 80. Living systematic reviews. Cochrane Community. Accessed March 22, 2022. Google Scholar
  • 81. Islam N, Salameh JP, Leeflang MM, et al; Cochrane COVID-19 Diagnostic Test Accuracy Group. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2020;11:CD013639. MedlineGoogle Scholar
  • 82. Islam N, Ebrahimzadeh S, Salameh JP, et al; Cochrane COVID-19 Diagnostic Test Accuracy Group. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2021;3:CD013639. MedlineGoogle Scholar
  • 83. Salameh JP, Leeflang MM, Hooft L, et al; Cochrane COVID-19 Diagnostic Test Accuracy Group. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2020;9:CD013639. MedlineGoogle Scholar
  • 84. Kim HY, Park JH, Lee YJ, Lee SS, Jeon JJ, Lee KH. Systematic Review and Meta-Analysis of CT Features for Differentiating Complicated and Uncomplicated Appendicitis. Radiology 2018;287(1):104–115. LinkGoogle Scholar

Article History

Received: June 19 2022
Revision requested: July 19 2022
Revision received: Oct 25 2022
Accepted: Oct 28 2022
Published online: Mar 14 2023