CommunicationsFree Access

Coronavirus Disease 2019 and Chest CT: Do Not Put the Sensitivity Value in the Isolation Room and Look Beyond the Numbers

Published Online:https://doi.org/10.1148/radiol.2020201709

Editor:

With interest we read the systematic review and meta-analysis by Dr Kim and colleagues (1) published online in April in Radiology regarding the value of chest CT in diagnosing coronavirus disease 2019 (COVID-19) infection. Kim et al reported chest CT to have a high pooled sensitivity of 94% (95% confidence interval [CI]: 91%, 96%), but a low specificity of 37% (95% CI: 26%, 50%). However, we believe that there is no convincing evidence yet that chest CT achieves such a high sensitivity in diagnosing COVID-19 in clinical practice. Note that the majority of studies that were included in the meta-analysis by Kim et al (1) (58 of 63 studies) only enrolled patients with proven COVID-19 infection whereas patients without the disease were lacking. Strikingly, this is not in line with their exclusion criterion number 3: “lack of extractable data for a two-by-two contingency table.” As a result, these 58 studies only allowed for the calculation of sensitivity, and not specificity. However, the diagnostic value of a test depends on its ability to discriminate between patients with and without disease (2). Sensitivity and specificity are intertwined entities and are both dependent on the threshold value that is applied to discriminate between patients with the disease and those without (2). Generally, creating a high sensitivity by applying a low threshold is at the expense of specificity (2). Multiple studies in the meta-analysis by Dr Kim and colleagues did not report which criteria were used as threshold value (1). The possibility that a low threshold was used remains a realistic scenario. Applying a low threshold in cohorts of patients suspected of having the disease (both with and without an actual COVID-19 infection) may result in virtually all patients classified as having the disease. As a result, sensitivity values in these individual studies and the pooled estimate that was calculated by Dr Kim and colleagues (1) may have been overestimated. It should also be noted that the five studies that did provide a 2 × 2 diagnostic contingency table had numerous methodologic flaws. The lack of high-quality evidence, rather than the mathematical numbers, should have been the main conclusion in the otherwise excellent work by Dr Kim and colleagues (1).

Disclosures of Conflicts of Interest: H.J.A.A. disclosed no relevant relationships. T.C.K. disclosed no relevant relationships. R.M.K. disclosed no relevant relationships.

References

  • 1. Kim H, Hong H, Yoon SH. Diagnostic Performance of CT and Reverse Transcriptase-Polymerase Chain Reaction for Coronavirus Disease 2019: A Meta-Analysis. Radiology 2020 Apr 17:201343 [Epub ahead of print] https://doi.org/10.1148/radiol.2020201343. LinkGoogle Scholar
  • 2. Fletcher R, Fletcher S. Clinical Epidemiology: The Essentials. Baltimore, Md: Lippincott Williams & Wilkins, 2013. Google Scholar

References

References

1. Kim H, Hong H, Yoon SH. Diagnostic Performance of CT and Reverse Transcriptase-Polymerase Chain Reaction for Coronavirus Disease 2019: A Meta-Analysis. Radiology 2020 Apr 17:201343 [Epub ahead of print] https://doi.org/10.1148/radiol.2020201343. LinkGoogle Scholar
2. Fletcher R, Fletcher S. Clinical Epidemiology: The Essentials. Baltimore, Md: Lippincott Williams & Wilkins, 2013. Google Scholar


Response

We thank the authors for their interest in our study (1). We admit that the third exclusion criterion was described insufficiently. To be exact, studies with a lack of extractable data for true-positive cases and disease-positive cases to calculate the sensitivity or true-negative cases and disease-negative cases to calculate the specificity were excluded.

We understand the concern about the potential of overestimation of the sensitivity for chest CT. The sensitivity and specificity are interdependent measures, and thus higher sensitivity may result in lower specificity of a diagnostic test. Given the circumstance that the majority of studies we analyzed reported only the sensitivity, the threshold effect could not be identified. Nevertheless, we performed a subgroup analysis for the five articles that reported both sensitivity and specificity of chest CT (26). In these studies, the pooled sensitivity was 96% (95% CI: 94%, 97%; I2 = 0%), which was similar to that of the primary analysis (94%; 95% CI: 91%, 96%; I2 = 95%). For the five studies, the reported sensitivity ranged from 94% to 100%, and the specificity ranged from 25% to 56%. On the basis of the visual evaluation of the coupled forest plot, there was no decrease in sensitivities according to increase in specificities.

Furthermore, we conducted an additional subgroup analysis for the studies with a low risk of bias for the CT interpretation, which clarified that the image readers were blinded to the clinical information or used radiology reports obtained from the routine clinical practice (2,729). Again, the pooled sensitivity (93%; 95% CI: 86%, 96%; I2 = 96%) was comparable to that of the primary analysis. Although there was a huge heterogeneity in the included studies, we believe our findings would help guide the radiology practice during the outbreak of COVID-19.

Disclosures of Conflicts of Interest: H.K. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: disclosed money to author for research grant from Lunit. Other relationships: disclosed no relevant relationships. H.H. disclosed no relevant relationships. S.H.Y. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: disclosed money to author’s institution for grant from GE Healthcare. Other relationships: disclosed no relevant relationships.

References

1. Kim H, Hong H, Yoon SH. Diagnostic performance of CT and reverse transcriptase-polymerase chain reaction for coronavirus disease 2019: a meta-analysis. Radiology 2020 Apr 17:201343 [Epub ahead of print] https://doi.org/10.1148/radiol.2020201343. LinkGoogle Scholar
2. Ai T, Yang Z, Hou H, et al. Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology 2020 Feb 26:200642 [Epub ahead of print] https://doi.org/10.1148/radiol.2020200642. LinkGoogle Scholar
3. Caruso D, Zerunian M, Polici M, et al. Chest CT Features of COVID-19 in Rome, Italy. Radiology 2020 Apr 3:201237 [Epub ahead of print] https://doi.org/10.1148/radiol.2020201237. LinkGoogle Scholar
4. Cheng Z, Lu Y, Cao Q, et al. Clinical features and chest CT manifestations of coronavirus disease 2019 (COVID-19) in a single-center study in Shanghai, China. AJR Am J Roentgenol 2020 Mar 14:1–6 [Epub ahead of print] https://doi.org/10.2214/AJR.20.22959. CrossrefGoogle Scholar
5. Himoto Y, Sakata A, Kirita M, et al. Diagnostic performance of chest CT to differentiate COVID-19 pneumonia in non-high-epidemic area in Japan. Jpn J Radiol 2020;38(5):400406. https://doi.org/10.1007/s11604-020-00958-w. Crossref, MedlineGoogle Scholar
6. Zhu W, Xie K, Lu H, Xu L, Zhou S, Fang S. Initial clinical features of suspected coronavirus disease 2019 in two emergency departments outside of Hubei, China. J Med Virol 2020 Mar 13 [Epub ahead of print] https://doi.org/10.1002/jmv.25763. CrossrefGoogle Scholar
7. Bai HX, Hsieh B, Xiong Z, et al. Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT. Radiology 2020 Mar 10:200823 [Epub ahead of print] https://doi.org/10.1148/radiol.2020200823. LinkGoogle Scholar
8. Chan JF, Yip CC, To KK, et al. Improved Molecular Diagnosis of COVID-19 by the Novel, Highly Sensitive and Specific COVID-19-RdRp/Hel Real-Time Reverse Transcription-PCR Assay Validated In Vitro and with Clinical Specimens. J Clin Microbiol 2020;58(5):e00310-20 [Epub ahead of print] https://doi.org/10.1128/JCM.00310-20. Crossref, MedlineGoogle Scholar
9. Chan JF, Yuan S, Kok KH, et al. A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster. Lancet 2020;395(10223):514523. Crossref, MedlineGoogle Scholar
10. Guan WJ, Ni ZY, Hu Y, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med 2020;382(18):17081720. Crossref, MedlineGoogle Scholar
11. Hu Z, Song C, Xu C, et al. Clinical characteristics of 24 asymptomatic infections with COVID-19 screened among close contacts in Nanjing, China. Sci China Life Sci 2020;63(5):706711. Crossref, MedlineGoogle Scholar
12. Inui S, Fujikawa A, Jitsu M, et al. Chest CT findings in cases from the cruise ship “Diamond Princess” with coronavirus disease 2019 (COVID-19). Radiol Cardiothorac Imaging 2020;2(2):e200110. https://doi.org/10.1148/ryct.2020200110. LinkGoogle Scholar
13. Li C, Ji F, Wang L, et al. Asymptomatic and human-to-human transmission of SARS-CoV-2 in a 2-family cluster, Xuzhou, China. Emerg Infect Dis 2020;26(7) [Epub ahead of print] https://doi.org/10.3201/eid2607.200718. CrossrefGoogle Scholar
14. Li K, Fang Y, Li W, et al. CT image visual quantitative evaluation and clinical classification of coronavirus disease (COVID-19). Eur Radiol 2020 Mar 25 [Epub ahead of print] https://doi.org/10.1007/s00330-020-06817-6. CrossrefGoogle Scholar
15. Li K, Wu J, Wu F, et al. The clinical and chest CT features associated with severe and critical COVID-19 pneumonia. Invest Radiol 2020 Feb 29 [Epub ahead of print] https://doi.org/10.1097/RLI.0000000000000672. CrossrefGoogle Scholar
16. Li P, Fu JB, Li KF, et al. Transmission of COVID-19 in the terminal stage of incubation period: a familial cluster. Int J Infect Dis 2020 Mar 16 [Epub ahead of print] https://doi.org/10.1016/j.ijid.2020.03.027. CrossrefGoogle Scholar
17. Liu K, Fang YY, Deng Y, et al. Clinical characteristics of novel coronavirus cases in tertiary hospitals in Hubei Province. Chin Med J (Engl) 2020 Feb 7 [Epub ahead of print] https://doi.org/10.1097/CM9.0000000000000744. Google Scholar
18. Lu X, Zhang L, Du H, et al. SARS-CoV-2 infection in children. N Engl J Med 2020;382(17):16631665. Crossref, MedlineGoogle Scholar
19. Qiu H, Wu J, Hong L, Luo Y, Song Q, Chen D. Clinical and epidemiological features of 36 children with coronavirus disease 2019 (COVID-19) in Zhejiang, China: an observational cohort study. Lancet Infect Dis 2020 Mar 25 [Epub ahead of print] https://doi.org/10.1016/S1473-3099(20)30198-5. CrossrefGoogle Scholar
20. Wang D, Hu B, Hu C, et al. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China. JAMA 2020 Feb 7 [Epub ahead of print] https://doi.org/10.1001/jama.2020.1585. CrossrefGoogle Scholar
21. Wang L, Gao YH, Lou LL, Zhang GJ. The clinical dynamics of 18 cases of COVID-19 outside of Wuhan, China. Eur Respir J 2020;55(4):2000398. https://doi.org/10.1183/13993003.00398-2020. Crossref, MedlineGoogle Scholar
22. Wu J, Liu J, Zhao X, et al. Clinical characteristics of imported cases of COVID-19 in Jiangsu province: a multicenter descriptive study. Clin Infect Dis 2020 Feb 29 [Epub ahead of print] https://doi.org/10.1093/cid/ciaa199. Google Scholar
23. Xie C, Jiang L, Huang G, et al. Comparison of different samples for 2019 novel coronavirus detection by nucleic acid amplification tests. Int J Infect Dis 2020;93:264267. Crossref, MedlineGoogle Scholar
24. Xie X, Zhong Z, Zhao W, Zheng C, Wang F, Liu J. Chest CT for typical 2019-nCoV pneumonia: relationship to negative RT-PCR testing. Radiology 2020 Feb 12:200343 [Epub ahead of print] https://doi.org/10.1148/radiol.2020200343. LinkGoogle Scholar
25. Xu XW, Wu XX, Jiang XG, et al. Clinical findings in a group of patients infected with the 2019 novel coronavirus (SARS-Cov-2) outside of Wuhan, China: retrospective case series. BMJ 2020;368:m606 [Published correction appears in BMJ 2020;368:m792.] https://doi.org/10.1136/bmj.m606. Crossref, MedlineGoogle Scholar
26. Zhang J, Wang S, Xue Y. Fecal specimen diagnosis 2019 novel coronavirus-infected pneumonia. J Med Virol 2020;92(6):680682. https://doi.org/10.1002/jmv.25742. Crossref, MedlineGoogle Scholar
27. Zhang JJ, Dong X, Cao YY, et al. Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan, China. Allergy 2020 Feb 19 [Epub ahead of print] https://doi.org/10.1111/all.14238. Google Scholar
28. Zhang MQ, Wang XH, Chen YL, et al. Clinical features of 2019 novel coronavirus pneumonia in the early stage from a fever clinic in Beijing [in Chinese]. Zhonghua Jie He He Hu Xi Za Zhi 2020;43(3):215218. MedlineGoogle Scholar
29. Zhao D, Yao F, Wang L, et al. A comparative study on the clinical features of COVID-19 pneumonia to other pneumonias. Clin Infect Dis 2020 Mar 12 [Epub ahead of print] https://doi.org/10.1093/cid/ciaa247. CrossrefGoogle Scholar

Article History

Published online: Apr 27 2020
Published in print: Oct 2020