Original ResearchFree Access

Computer-aided Detection of Subsolid Nodules at Chest CT: Improved Performance with Deep Learning–based CT Section Thickness Reduction

Abstract

Background

Studies on the optimal CT section thickness for detecting subsolid nodules (SSNs) with computer-aided detection (CAD) are lacking.

Purpose

To assess the effect of CT section thickness on CAD performance in the detection of SSNs and to investigate whether deep learning–based super-resolution algorithms for reducing CT section thickness can improve performance.

Materials and Methods

CT images obtained with 1-, 3-, and 5-mm-thick sections were obtained in patients who underwent surgery between March 2018 and December 2018. Patients with resected synchronous SSNs and those without SSNs (negative controls) were retrospectively evaluated. The SSNs, which ranged from 6 to 30 mm, were labeled ground-truth lesions. A deep learning–based CAD system was applied to SSN detection on CT images of each section thickness and those converted from 3- and 5-mm section thickness into 1-mm section thickness by using the super-resolution algorithm. The CAD performance on each section thickness was evaluated and compared by using the jackknife alternative free response receiver operating characteristic figure of merit.

Results

A total of 308 patients (mean age ± standard deviation, 62 years ± 10; 183 women) with 424 SSNs (310 part-solid and 114 nonsolid nodules) and 182 patients without SSNs (mean age, 65 years ± 10; 97 men) were evaluated. The figures of merit differed across the three section thicknesses (0.92, 0.90, and 0.89 for 1, 3, and 5 mm, respectively; P = .04) and between 1- and 5-mm sections (P = .04). The figures of merit varied for nonsolid nodules (0.78, 0.72, and 0.66 for 1, 3, and 5 mm, respectively; P < .001) but not for part-solid nodules (range, 0.93–0.94; P = .76). The super-resolution algorithm improved CAD sensitivity on 3- and 5-mm-thick sections (P = .02 for 3 mm, P < .001 for 5 mm).

Conclusion

Computer-aided detection (CAD) of subsolid nodules performed better at 1-mm section thickness CT than at 3- and 5-mm section thickness CT, particularly with nonsolid nodules. Application of a super-resolution algorithm improved the sensitivity of CAD at 3- and 5-mm section thickness CT.

© RSNA, 2021

Online supplemental material is available for this article.

See also the editorial by Goo in this issue.

Summary

Deep learning–based reduction in CT section thickness improved the sensitivity of computer-aided detection of subsolid nodules on thick-section images.

Key Results

  • ■ In a retrospective study of 490 patients, computer-aided detection (CAD) performance for subsolid nodules differed across three section thicknesses (figure of merit, 0.92, 0.90, and 0.89 for 1, 3, and 5 mm, respectively; P = .04).

  • ■ CAD performance varied across section thicknesses for nonsolid nodules (figure of merit, 0.78, 0.72, and 0.66 for 1, 3, and 5 mm, respectively; P < .001) but not for part-solid nodules (figure of merit, 0.93–0.94; P = .76).

  • ■ Deep learning–based reduction in CT section thickness improved the sensitivity of CAD on thick-section images (P < .001 to P = .02).

Introduction

A computer-aided detection (CAD) system applied to chest CT can detect actionable nodules as a second reader and help reduce missed nodules. Several studies have demonstrated the effectiveness of CAD, with improved sensitivity for nodule detection even after double-reading (14). Indeed, CAD detected 56%–70% of lung cancers missed by radiologists during lung cancer screening (2). Positive results for lung cancer screening with CT (57) have increased attention on the application of CAD to routine CT screening, particularly in terms of error reduction and work efficiency.

To date, CAD has focused on the detection of solid nodules because they can be an early finding of lung cancer, and the distinct differences between solid nodules and normal lung parenchyma enabled the development of the software. However, subsolid nodules (SSNs) are also clinically important because they show a higher rate of malignancy than solid nodules, with an incidence of SSNs at lung cancer screening of up to 9.4% (811). A limited number of studies have addressed the performance of CAD on SSNs, with various results (1215). Recently, Silva et al (14) found that CAD achieved a high sensitivity (88.4%) in the detection of SSNs (86.8% for nonsolid nodules and 94.1% for part-solid nodules) at low-dose screening CT, with 65.8% of risk-dominant SSNs (102 of 155) being detected only with CAD.

When CAD is applied to SSNs, CT section thickness is an important issue. At thick-section CT, the detection of SSNs can be hindered because of low contrast resolution of SSNs and partial volume averaging effects. Godoy et al (13) reported the benefit of thin-section CT (0.67 or 1 mm for thin-section CT vs 5 mm for thick-section CT) in the detection of SSNs. However, because 3- or 5-mm-thick CT images are usually reviewed in clinical practice, thin-section CT may not be available in every institution. If thin-section CT is required to obtain the best performance from CAD, then it may be worth using a deep learning–based super-resolution algorithm to reduce the CT section thickness, as adopted in a previous study (16). In addition, the actual merit of thin-section CT over 3- or 5-mm CT in the detection of SSNs by recent CAD systems has not yet been sufficiently addressed.

Therefore, the purpose of this study was to assess the effect of CT section thickness on the performance of CAD for detecting SSNs and to investigate whether deep learning–based super-resolution algorithms for reducing CT section thickness can improve performance.

Materials and Methods

Our institutional review board approved this retrospective study and waived the requirement for patient informed consent. Patient overlap with a previous study is described in Appendix E1 (online).

Patients

We retrospectively searched the electronic medical records of our tertiary referral institution (Asan Medical Center) from March 2018 to December 2018 to identify patients who underwent curative resection of lung adenocarcinoma. The patients in the SSN group were enrolled on the basis of the following criteria: (a) contrast-enhanced chest CT reconstructed with section thicknesses of 1, 3, and 5 mm; (b) an SSN at CT; and (c) mean nodule size between 6 mm or greater and 3 cm or less. The nodule types were classified by two radiologists (K.H.D. and S.M.L., with 21 and 11 years of experience in chest radiology, respectively) in consensus. In patients with multiple SSNs, up to seven SSNs were included, regardless of whether they were resected.

The exclusion criteria were recurrent tumor, history of neoadjuvant chemotherapy, severe lung parenchymal abnormalities, and poor CT scan quality. Patients who underwent resection of lung adenocarcinoma during the same period without any eligible SSN were included in a control group.

CT Protocol

Chest CT scans were obtained with multidetector-row CT scanners from one vendor. The acquisition parameters were as follows: 120 kVp, 150–200 mA, pitch of 0.875–1, and collimation of 1–1.25 mm. Images were reconstructed by using a B50f kernel with section thickness of 1 mm and interval of 1 mm, section thickness of 3 mm and interval of 3 mm, section thickness of 3 mm and interval of 2.5 mm, and section thickness of 5 mm and interval of 5 mm. Contrast material enhancement was used in all patients. Images were reconstructed in axial, coronal, and sagittal planes (Appendix E1 [online]).

CAD System for Detection of SSNs

A commercial CAD software system (VUNO Med-LungCT AI, version 1.0.0; VUNO) was used for the detection of SSNs on CT images of each section thickness. The CAD system generated a 6 × 6-mm annotation on axial CT images and rated the probability of the annotated region being a true lesion on a continuous scale between 0 and 1.

A deep learning–based super-resolution algorithm for CT section reduction (16) was applied to the 3- and 5-mm CT images to convert them into 1-mm CT images (super-resolution of 3 mm and 5 mm, respectively). Then, the CAD was applied to the detection of SSNs on the converted images. Because the super-resolution algorithm was designed to generate images with 1-mm-thick sections and 1-mm interval, it could not be applied to CT scans with 3-mm-thick sections and 2.5-mm interval.

Analysis of CAD Results

The initial ground-truth lesions were determined by two radiologists who included patients for this study, referring to the original CT reading and pathology reports. Then, the CAD-marked images were reviewed by one radiologist (W.K., with 8 years of experience in chest radiology) according to the list of nodules with their sizes, locations, and nodule types. The CAD marks were considered true-positive findings when any part of a ground-truth lesion was within the annotation. If the CAD system depicted SSNs within the size criteria that had been overlooked by the radiologists, then they were added as ground-truth lesions. CAD marks other than true-positive results were classified as false-positive findings or lesions to be excluded from the analysis. Because the CAD system in this study was not developed to be specific for SSNs, solid nodules and other pathologic lesions manifesting as consolidation or ground-glass opacity were not considered as a false-positive finding and were therefore excluded.

In addition, image noise was measured and compared between the original and converted images (Appendix E1 [online], Table E1 [online]).

Statistical Analysis

The performance of the CAD system in the detection of SSNs on the three original section thickness CT scans (1, 3, and 5 mm) and the two converted images (super resolution of 3 mm and 5 mm) was evaluated and compared by using jackknife alternative free response receiver operating characteristic analysis (version 4.2.1; http://www.devchakraborty.com) on a per-lesion basis. The figure of merit, which is defined as the probability that a lesion is rated higher than the highest-rated nonlesion on negative control images (17), was calculated for each section thickness and compared with the Dorfman-Berbaum-Metz method with Hillis improvements. The performance of the CAD system was assessed for all SSNs and according to nodule type (part-solid nodules or nonsolid nodules) and size (≤1 cm or >1 cm).

Logistic regression with generalized estimating equations was used to compare sensitivities between the images obtained with different section thicknesses, and Poisson regression with generalized estimating equations was used to compare the false-positive fractions. Pairwise comparisons of the figure of merit, sensitivity, and false-positive values across different section thicknesses were performed with Bonferroni correction by multiplying the P values by the number of comparisons.

Statistical analyses were performed by using R software (version 3.6.1; http://www.r-project.org/). P < .05 was considered to indicate a statistically significant difference.

Results

Patient Characteristics

Of 763 patients who underwent curative resection of lung adenocarcinoma, 334 met the eligibility criteria. Of these 334 patients, 26 were excluded because of recurrent tumor (n = 11), a history of neoadjuvant chemotherapy (n = 2), severe parenchymal abnormalities (n = 9), and poor CT quality (n = 4) (Fig 1). This left 308 patents in the SSN group (mean age ± standard deviation, 62 years ± 10; age range, 33–83 years), consisting of 125 men and 183 women (Table 1). A total of 424 SSNs from these 308 patients were included as ground-truth lesions. Of the 308 patients, 244 had one nodule, 39 had two nodules, 13 had three nodules, five had four nodules, one had five nodules, four had six nodules, and two had seven nodules. The mean nodule size was 13.2 mm ± 5.6 (range, 6–30 mm), and the mean solid portion size was 8.4 mm ± 7.5 (range, 0–28 mm). Of 342 resected SSNs, more than one SSN was resected in 27 patients (two SSNs in 22 patients, three SSNs in four patients, and five SSNs in one patient).

Flow diagram of patient inclusion. SSN = subsolid                        nodule.

Figure 1: Flow diagram of patient inclusion. SSN = subsolid nodule.

Table 1: Baseline Characteristics of Study Sample

Table 1:

The control group (mean age, 65 years ± 10; age range, 29–89 years) consisted of 97 men and 85 women. The mean interval between CT and surgery was 9 days ± 14 (median, 1 day; range, 0–59 days) in the SSN group and 9 days ± 13 (median, 1 day; range, 0–55 days) in the control group.

Performance of the CAD System at CT with 1-, 3-, and 5-mm Section Thickness

On a per-lesion basis, the sensitivity of CAD for detecting SSNs was higher at 1-mm CT, with values of 92.0% (390 of 424 lesions), 86.3% (366 of 424 lesions), and 79.2% (336 of 424 lesions) on 1-, 3-, and 5-mm images, respectively (P < .001 for all pairwise comparisons) (Table 2). On a per-patient basis, CAD showed high sensitivity for detecting the largest resected SSN of each patient on images of all three section thicknesses (97.4% [300 of 308 patients], 95.8% [295 of 308 patients], and 93.2% [287 of 308 patients] for 1-, 3-, and 5-mm images, respectively; P = .01) (Fig 2). The false-positive fractions were higher at thinner-section CT (false-positive fraction, 1.1 [554 lesions in 490 patients], 0.7 [341 lesions in 490 patients], and 0.4 [205 lesions in 490 patients] for 1-, 3-, and 5-mm images, respectively; P < .001 for all pairwise comparisons).

Table 2: Sensitivities of CAD on CT Images with Different Section Thicknesses and Super-Resolution–converted Images

Table 2:
Images in a 57-year-old woman with a pathologically proven invasive                        adenocarcinoma in right upper lobe of lung. Axial contrast-enhanced CT scans                        obtained with (a) 1-mm, (b) 3-mm, and (c) 5-mm section thickness show a                        14-mm well-defined part-solid nodule with an 8-mm solid portion in the right                        upper lobe (arrows). The nodule widely abuts and lies along the                        bronchovascular bundles. Computer-aided detection failed to depict the                        nodule on all three images.

Figure 2a: Images in a 57-year-old woman with a pathologically proven invasive adenocarcinoma in right upper lobe of lung. Axial contrast-enhanced CT scans obtained with (a) 1-mm, (b) 3-mm, and (c) 5-mm section thickness show a 14-mm well-defined part-solid nodule with an 8-mm solid portion in the right upper lobe (arrows). The nodule widely abuts and lies along the bronchovascular bundles. Computer-aided detection failed to depict the nodule on all three images.

Images in a 57-year-old woman with a pathologically proven invasive                        adenocarcinoma in right upper lobe of lung. Axial contrast-enhanced CT scans                        obtained with (a) 1-mm, (b) 3-mm, and (c) 5-mm section thickness show a                        14-mm well-defined part-solid nodule with an 8-mm solid portion in the right                        upper lobe (arrows). The nodule widely abuts and lies along the                        bronchovascular bundles. Computer-aided detection failed to depict the                        nodule on all three images.

Figure 2b: Images in a 57-year-old woman with a pathologically proven invasive adenocarcinoma in right upper lobe of lung. Axial contrast-enhanced CT scans obtained with (a) 1-mm, (b) 3-mm, and (c) 5-mm section thickness show a 14-mm well-defined part-solid nodule with an 8-mm solid portion in the right upper lobe (arrows). The nodule widely abuts and lies along the bronchovascular bundles. Computer-aided detection failed to depict the nodule on all three images.

Images in a 57-year-old woman with a pathologically proven invasive                        adenocarcinoma in right upper lobe of lung. Axial contrast-enhanced CT scans                        obtained with (a) 1-mm, (b) 3-mm, and (c) 5-mm section thickness show a                        14-mm well-defined part-solid nodule with an 8-mm solid portion in the right                        upper lobe (arrows). The nodule widely abuts and lies along the                        bronchovascular bundles. Computer-aided detection failed to depict the                        nodule on all three images.

Figure 2c: Images in a 57-year-old woman with a pathologically proven invasive adenocarcinoma in right upper lobe of lung. Axial contrast-enhanced CT scans obtained with (a) 1-mm, (b) 3-mm, and (c) 5-mm section thickness show a 14-mm well-defined part-solid nodule with an 8-mm solid portion in the right upper lobe (arrows). The nodule widely abuts and lies along the bronchovascular bundles. Computer-aided detection failed to depict the nodule on all three images.

In terms of nodule type, CAD detected most of the part-solid nodules on 1- and 3-mm images (96.8% [300 of 310 lesions] and 95.8% [297 of 310 lesions], respectively; P > .99) but showed lower sensitivity for part-solid nodules on 5-mm images (91.3% [283 of 310 lesions]; P = .005–.01). For nonsolid nodules, more distinct differences were shown across the different section thicknesses, with sensitivities of 78.9% (90 of 114 lesions), 60.5% (69 of 114 lesions), and 46.5% (53 of 114 lesions) for 1, 3, and 5 mm, respectively (P < .001 for all pairwise comparisons). Within the small nodules (≤1 cm), the sensitivity of CAD differed across the three section thicknesses (81.9% [122 of 149 lesions], 67.1% [100 of 149 lesions], and 53.7% [80 of 149 lesions] for 1, 3, and 5 mm, respectively; P < .001 for all pairwise comparisons).

The figures of merit for 1-, 3-, and 5-mm CT images were 0.92 (95% CI: 0.89, 0.94), 0.90 (95% CI: 0.88, 0.92), and 0.89 (95% CI: 0.87, 0.92), respectively, differing across the three image sizes (P = .04) and for the pairwise comparison between 1- and 5-mm images (P = .04) (Table 3, Fig 3). For nonsolid nodules, CAD performed best on 1-mm images, showing higher figures of merit than on 3- or 5-mm images (0.78, 0.72, and 0.66 for 1, 3, and 5 mm, respectively; P = .03 for 1-mm vs 3-mm images; P < .001 for 1-mm vs 5-mm images), whereas the CAD performance for part-solid nodules was consistently high, with almost unvaried figures of merit (range, 0.93–0.94; P = .76). For small nodules (≤1 cm), CAD performed better on 1-mm images than on 5-mm images (figures of merit, 0.81 for 1 mm vs 0.71 for 5 mm; P < .001), although there was no difference between 1- and 3-mm images (figure of merit, 0.77; P = .11).

Table 3: JAFROC Figures of Merit for CAD Software on Differing CT Section Thicknesses

Table 3:
Jackknife alternative free response receiver operating characteristic                        curves for the computer-aided detection (CAD) of subsolid nodules (SSNs) on                        a per-lesion basis. CAD performance in the detection of all SSNs decreased                        as sections got thicker (figure of merit, 0.92 for 1-mm images, 0.90 for                        3-mm images, and 0.89 for 5-mm images), differing across the images with the                        three section thicknesses (P = .04) and between the 1- and 5-mm images                        (P = .04).

Figure 3: Jackknife alternative free response receiver operating characteristic curves for the computer-aided detection (CAD) of subsolid nodules (SSNs) on a per-lesion basis. CAD performance in the detection of all SSNs decreased as sections got thicker (figure of merit, 0.92 for 1-mm images, 0.90 for 3-mm images, and 0.89 for 5-mm images), differing across the images with the three section thicknesses (P = .04) and between the 1- and 5-mm images (P = .04).

Performance of the CAD System on Images Converted with the Deep Learning–based Super-Resolution Algorithm

The super-resolution algorithm could be applied to the 3-mm images in 158 of the 308 patients in the SSN group (226 SSNs; 64 nonsolid nodules and 162 part-solid nodules) and 79 of the 182 patients in the control group, whereas it could be applied to the 5-mm images in all cases except for four in the control group, for which it failed because of technical error.

After application of the super-resolution algorithm to the 3-mm images, the sensitivities improved (range, 86.3% [366 of 424 lesions] to 91.6% [204 of 226 lesions]; P = .02) and were equivalent to that on the original 1-mm images (92.0% [390 of 424 lesions]; P > .99). For the 5-mm images, the sensitivities also improved after application of the super-resolution algorithm (range, 79.2% [326 of 424 lesions] to 88.9% [377 of 424 lesions]; P < .001) (Fig 4), but a difference from the original 1-mm images remained (P = .04). Notably, the false-positive fractions of the converted images were higher after the improvement in sensitivity (from 0.7 [341 lesions in 490 patients] to 1.0 [239 lesions in 237 patients] for super-resolution 3-mm images and from 0.4 [205 lesions in 490 patients] to 0.9 [451 lesions in 486 patients] for super-resolution 5-mm images; P < .001 for both) (Fig 5; Table 4, Table E2 [online]).

Images in a 72-year-old woman with pathologically proven invasive                        adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans                        obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images                        converted from original (c) 3-mm, and (d) 5-mm-thick sections after                        application of a super-resolution algorithm show a 16-mm well-defined                        part-solid nodule with a 5-mm solid portion in the left upper lobe. The                        solid portion of the nodule (arrows) is well defined on the 1-mm image (a)                        and not clearly delineated on the 5-mm image (b). The computer-aided                        detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but                        missed it on the 5-mm image. After application of the super-resolution                        algorithm (c, d), the solid portion was retrieved, and CAD detected the                        nodule on both super-resolution 3- and 5-mm images.

Figure 4a: Images in a 72-year-old woman with pathologically proven invasive adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images converted from original (c) 3-mm, and (d) 5-mm-thick sections after application of a super-resolution algorithm show a 16-mm well-defined part-solid nodule with a 5-mm solid portion in the left upper lobe. The solid portion of the nodule (arrows) is well defined on the 1-mm image (a) and not clearly delineated on the 5-mm image (b). The computer-aided detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but missed it on the 5-mm image. After application of the super-resolution algorithm (c, d), the solid portion was retrieved, and CAD detected the nodule on both super-resolution 3- and 5-mm images.

Images in a 72-year-old woman with pathologically proven invasive                        adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans                        obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images                        converted from original (c) 3-mm, and (d) 5-mm-thick sections after                        application of a super-resolution algorithm show a 16-mm well-defined                        part-solid nodule with a 5-mm solid portion in the left upper lobe. The                        solid portion of the nodule (arrows) is well defined on the 1-mm image (a)                        and not clearly delineated on the 5-mm image (b). The computer-aided                        detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but                        missed it on the 5-mm image. After application of the super-resolution                        algorithm (c, d), the solid portion was retrieved, and CAD detected the                        nodule on both super-resolution 3- and 5-mm images.

Figure 4b: Images in a 72-year-old woman with pathologically proven invasive adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images converted from original (c) 3-mm, and (d) 5-mm-thick sections after application of a super-resolution algorithm show a 16-mm well-defined part-solid nodule with a 5-mm solid portion in the left upper lobe. The solid portion of the nodule (arrows) is well defined on the 1-mm image (a) and not clearly delineated on the 5-mm image (b). The computer-aided detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but missed it on the 5-mm image. After application of the super-resolution algorithm (c, d), the solid portion was retrieved, and CAD detected the nodule on both super-resolution 3- and 5-mm images.

Images in a 72-year-old woman with pathologically proven invasive                        adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans                        obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images                        converted from original (c) 3-mm, and (d) 5-mm-thick sections after                        application of a super-resolution algorithm show a 16-mm well-defined                        part-solid nodule with a 5-mm solid portion in the left upper lobe. The                        solid portion of the nodule (arrows) is well defined on the 1-mm image (a)                        and not clearly delineated on the 5-mm image (b). The computer-aided                        detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but                        missed it on the 5-mm image. After application of the super-resolution                        algorithm (c, d), the solid portion was retrieved, and CAD detected the                        nodule on both super-resolution 3- and 5-mm images.

Figure 4c: Images in a 72-year-old woman with pathologically proven invasive adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images converted from original (c) 3-mm, and (d) 5-mm-thick sections after application of a super-resolution algorithm show a 16-mm well-defined part-solid nodule with a 5-mm solid portion in the left upper lobe. The solid portion of the nodule (arrows) is well defined on the 1-mm image (a) and not clearly delineated on the 5-mm image (b). The computer-aided detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but missed it on the 5-mm image. After application of the super-resolution algorithm (c, d), the solid portion was retrieved, and CAD detected the nodule on both super-resolution 3- and 5-mm images.

Images in a 72-year-old woman with pathologically proven invasive                        adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans                        obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images                        converted from original (c) 3-mm, and (d) 5-mm-thick sections after                        application of a super-resolution algorithm show a 16-mm well-defined                        part-solid nodule with a 5-mm solid portion in the left upper lobe. The                        solid portion of the nodule (arrows) is well defined on the 1-mm image (a)                        and not clearly delineated on the 5-mm image (b). The computer-aided                        detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but                        missed it on the 5-mm image. After application of the super-resolution                        algorithm (c, d), the solid portion was retrieved, and CAD detected the                        nodule on both super-resolution 3- and 5-mm images.

Figure 4d: Images in a 72-year-old woman with pathologically proven invasive adenocarcinoma in left upper lobe of lung. Axial contrast-enhanced CT scans obtained with (a) 1-mm-thick sections, (b) 5-mm-thick sections and images converted from original (c) 3-mm, and (d) 5-mm-thick sections after application of a super-resolution algorithm show a 16-mm well-defined part-solid nodule with a 5-mm solid portion in the left upper lobe. The solid portion of the nodule (arrows) is well defined on the 1-mm image (a) and not clearly delineated on the 5-mm image (b). The computer-aided detection (CAD) algorithm depicted the nodule on 1- and 3-mm images but missed it on the 5-mm image. After application of the super-resolution algorithm (c, d), the solid portion was retrieved, and CAD detected the nodule on both super-resolution 3- and 5-mm images.

Images in a 29-year-old woman with subsegmental atelectasis in the                        left lower lobe of the lung as a false-positive finding detected by                        computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT                        scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c)                        sections show a nodular lesion in the subpleural area of the left lower                        lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the                        lesion on the 1-mm image but not the 5-mm image. (d) Coronal                        contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is                        linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image                        was regarded as a false-positive finding. Of note, the CAD algorithm marked                        the lesion on the super-resolution 5-mm image, as it did on 1-mm                        image.

Figure 5a: Images in a 29-year-old woman with subsegmental atelectasis in the left lower lobe of the lung as a false-positive finding detected by computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c) sections show a nodular lesion in the subpleural area of the left lower lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the lesion on the 1-mm image but not the 5-mm image. (d) Coronal contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image was regarded as a false-positive finding. Of note, the CAD algorithm marked the lesion on the super-resolution 5-mm image, as it did on 1-mm image.

Images in a 29-year-old woman with subsegmental atelectasis in the                        left lower lobe of the lung as a false-positive finding detected by                        computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT                        scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c)                        sections show a nodular lesion in the subpleural area of the left lower                        lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the                        lesion on the 1-mm image but not the 5-mm image. (d) Coronal                        contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is                        linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image                        was regarded as a false-positive finding. Of note, the CAD algorithm marked                        the lesion on the super-resolution 5-mm image, as it did on 1-mm                        image.

Figure 5b: Images in a 29-year-old woman with subsegmental atelectasis in the left lower lobe of the lung as a false-positive finding detected by computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c) sections show a nodular lesion in the subpleural area of the left lower lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the lesion on the 1-mm image but not the 5-mm image. (d) Coronal contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image was regarded as a false-positive finding. Of note, the CAD algorithm marked the lesion on the super-resolution 5-mm image, as it did on 1-mm image.

Images in a 29-year-old woman with subsegmental atelectasis in the                        left lower lobe of the lung as a false-positive finding detected by                        computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT                        scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c)                        sections show a nodular lesion in the subpleural area of the left lower                        lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the                        lesion on the 1-mm image but not the 5-mm image. (d) Coronal                        contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is                        linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image                        was regarded as a false-positive finding. Of note, the CAD algorithm marked                        the lesion on the super-resolution 5-mm image, as it did on 1-mm                        image.

Figure 5c: Images in a 29-year-old woman with subsegmental atelectasis in the left lower lobe of the lung as a false-positive finding detected by computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c) sections show a nodular lesion in the subpleural area of the left lower lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the lesion on the 1-mm image but not the 5-mm image. (d) Coronal contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image was regarded as a false-positive finding. Of note, the CAD algorithm marked the lesion on the super-resolution 5-mm image, as it did on 1-mm image.

Images in a 29-year-old woman with subsegmental atelectasis in the                        left lower lobe of the lung as a false-positive finding detected by                        computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT                        scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c)                        sections show a nodular lesion in the subpleural area of the left lower                        lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the                        lesion on the 1-mm image but not the 5-mm image. (d) Coronal                        contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is                        linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image                        was regarded as a false-positive finding. Of note, the CAD algorithm marked                        the lesion on the super-resolution 5-mm image, as it did on 1-mm                        image.

Figure 5d: Images in a 29-year-old woman with subsegmental atelectasis in the left lower lobe of the lung as a false-positive finding detected by computer-aided detection (CAD). (a–c) Axial contrast-enhanced CT scans obtained with 1-mm (a), 5-mm (b), and super-resolution 5-mm (c) sections show a nodular lesion in the subpleural area of the left lower lobe, mimicking a part-solid nodule (arrow). The CAD algorithm marked the lesion on the 1-mm image but not the 5-mm image. (d) Coronal contrast-enhanced CT scan with 1-mm-thick sections shows that the lesion is linear subsegmental atelectasis (arrow); thus, the mark on the 1-mm image was regarded as a false-positive finding. Of note, the CAD algorithm marked the lesion on the super-resolution 5-mm image, as it did on 1-mm image.

Table 4: False-Positive Fraction of CAD on CT Images of Different Section Thicknesses and Super-Resolution–converted Images

Table 4:

The figure of merit for super-resolution 3-mm images was not higher than that for the original 3-mm images (0.91 [95% CI: 0.89, 0.94] vs 0.89 [95% CI: 0.86, 0.93], respectively; P = .61) but did approach that of the original 1-mm images (0.92 [95% CI: 0.90, 0.95]; P > .99) (Table E3 [online]). More importantly, the figure of merit of the super-resolution 3-mm images (0.79 [95% CI: 0.71, 0.87]) improved to that of the original 1-mm images (0.79 [95% CI: 0.71, 0.87]) in nonsolid nodules, being higher than that of the original 3-mm images (0.69 [95% CI: 0.60, 0.79]; P = .04).

The figure of merit on the super-resolution 5-mm images was higher than that on the original 5-mm images (0.92 [95% CI: 0.90, 0.94] vs 0.89 [95% CI: 0.87, 0.92], respectively; P = .05), being equivalent to that on the original 1-mm images (0.92 [95% CI: 0.89, 0.94]; P > .99) (Table E4 [online]). In the nonsolid nodules, the figure of merit of the super-resolution 5-mm images was higher than that of the original 5-mm images (0.75 [95% CI: 0.68, 0.81] vs 0.66 [95% CI: 0.59, 0.73], respectively; P = .003).

Discussion

Few studies have reported on the optimal CT section thickness for detecting subsolid nodules (SSNs) with computer-aided detection (CAD). In our study, we assessed the effect of CT section thickness on CAD performance in the detection of SSNs and investigated whether deep learning–based super-resolution algorithms for reducing CT section thickness can improve performance. Our results showed that the CAD performance for detecting SSNs was affected by CT section thickness, with the highest performance on 1-mm images (figure of merit, 0.92, 0.90, and 0.89 for 1-, 3-, and 5-mm images, respectively; P = .04). When the deep learning–based super-resolution algorithm was applied to thick-section images, the sensitivity of CAD at 3- and 5-mm CT was improved (P = .02 for 3-mm images; P < .001 for 5-mm images). In detail, the superior performance on 1-mm images in comparison with 3- and 5-mm images was achieved only for nonsolid nodules (figures of merit, 0.78, 0.72, and 0.66 for 1-, 3-, and 5-mm images, respectively; P < .001). CAD of part-solid nodules showed a similar performance (figures of merit, 0.93–0.94; P = .76), regardless of the section thicknesses. Therefore, our analyses indicate that the effect of section thickness was primarily on nonsolid nodules. This result can be explained by the fact that the solid portion within part-solid nodules plays a key role in their detection with CAD, and CT section thickness has less effect on the solid portion than on ground-glass opacity.

Despite the importance of thin-section CT in CAD performance, this modality requires greater storage and higher data traffic. Therefore, it may not always be available. To optimize CAD detection when only thick-section CT is available, we adopted a super-resolution algorithm to convert thick-section images into thin-section images (16). The super-resolution algorithm improved the sensitivity of CAD on 3- and 5-mm images (range, 86.3%–91.6% for 3-mm images [P = .02] and 79.2%–88.9% for 5-mm images [P < .001]), especially for nonsolid nodules (range, 60.5%–78.6% for 3-mm images [P = .006] and 46.5%–69.3% for 5-mm images [P < .001]). Even for part-solid nodules, which CAD detected with high sensitivity on both 1- and 3-mm images (96.8% and 95.8%, respectively), the super-resolution algorithm improved the sensitivity of CAD on 5-mm images (from 91.3% to 96.1%; P = .007). Of note, the false-positive fraction was higher as a result of applying the super-resolution algorithm and reached that of the original 1-mm image (1.0 and 0.9 for super-resolution 3- and 5-mm images, respectively, vs 1.1 for the original 1-mm image). Some of the false-positive marks not generated on the original thick-section images but shown on the super-resolution 3- or 5-mm images were identical to the false-positive marks on the 1-mm images. The reappearance of false-positive marks implies that the algorithm-converted images were similar to the original 1-mm images. Although other aspects of this algorithm remain to be investigated, we believe that it has the potential to overcome some of the limitations of thick-section CT when thin-section CT is not available. Nevertheless, it must be noted that the super-resolution 5-mm images were still suboptimal compared with the 1-mm images in terms of the sensitivity for nonsolid nodules.

Although the deep learning–based CAD in our study demonstrated a high level of performance (sensitivity of 92.0% and false-positive fraction of 1.1 on 1-mm images) compared with previous CAD systems (sensitivity, 80%–88.4%; false-positive fraction, 0.3–3.0) (1315), the false-positive fraction was still not satisfactory. Substantial numbers of false-positive findings were located at the borders between two structures (ie, the border between lung parenchyma and pleura). The false-negative lesions included nodules with considerable size but manifesting as faint ground-glass opacity. In addition, nodules lying along bronchovascular bundles were also prone to be missed by the CAD. These weaknesses of the current CAD system may be reduced by reinforcing the learning of errors.

Our study had several limitations. First, we report only on the performance of the CAD system alone, and uncertainty remains as to how readers and the CAD system will interact in clinical practice. Second, the super-resolution algorithm was not applied to the entire study sample. A prerequisite for applying the algorithm is the same section thickness and interval, which may limit its wider application. In addition, solid nodules were not analyzed. Because SSNs are more affected by partial volume averaging effects, we focused on the detection of SSNs. Considering the high performance for part-solid nodules, we expect that solid nodules will also be reliably detected on the best settings for SSNs. The role of different CT hardware vendors on the utility of the CAD and this deep learning super-resolution method was also not explored.

In conclusion, computer-aided detection (CAD) of subsolid nodules performed better at thin-section CT than at thick-section CT, particularly for nonsolid nodules. The sensitivity of CAD on thick-section CT was improved by applying a deep learning–based algorithm. Refinement of CAD, particularly in the reduction of false-positive findings, and its use in conjunction with an observer are warranted.

Disclosures of Conflicts of Interest: S.P. disclosed no relevant relationships. S.M.L. disclosed no relevant relationships. W.K. disclosed no relevant relationships. H.P. disclosed no relevant relationships. K.H.J. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: is a shareholder in Vuno. Other relationships: disclosed no relevant relationships. K.H.D. disclosed no relevant relationships. J.B.S. disclosed no relevant relationships.

Author Contributions

Author contributions: Guarantors of integrity of entire study, S.M.L., W.K.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; agrees to ensure any questions related to the work are appropriately resolved, all authors; literature research, S.P., S.M.L., W.K., K.H.J., K.H.D.; clinical studies, S.P., S.M.L., W.K., K.H.D.; experimental studies, W.K., H.P., K.H.J.; statistical analysis, S.P., S.M.L., W.K., J.B.S.; and manuscript editing, all authors

1 Current address: Department of Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Va.

Supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (grant NRF-2019R1A2C1087524).

References

  • 1. Zhao Y, de Bock GH, Vliegenthart R, et al. Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume. Eur Radiol 2012;22(10):2076–2084. Crossref, MedlineGoogle Scholar
  • 2. Liang M, Tang W, Xu DM, et al. Low-dose CT screening for lung cancer: computer-aided detection of missed lung cancers. Radiology 2016;281(1):279–288. LinkGoogle Scholar
  • 3. Rubin GD, Lyo JK, Paik DS, et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 2005;234(1):274–283. LinkGoogle Scholar
  • 4. Christe A, Leidolt L, Huber A, et al. Lung cancer screening with CT: evaluation of radiologists and different computer assisted detection software (CAD) as first and second readers for lung nodule detection at different dose levels. Eur J Radiol 2013;82(12):e873–e878. Crossref, MedlineGoogle Scholar
  • 5. National Lung Screening Trial Research Team; Aberle DR, Adams AM, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 2011;365(5):395–409. Crossref, MedlineGoogle Scholar
  • 6. de Koning HJ, van der Aalst CM, de Jong PA, et al. Reduced lung-cancer mortality with volume CT screening in a randomized trial. N Engl J Med 2020;382(6):503–513. Crossref, MedlineGoogle Scholar
  • 7. Pastorino U, Silva M, Sestini S, et al. Prolonged lung cancer screening reduced 10-year mortality in the MILD trial: new confirmation of lung cancer screening efficacy. Ann Oncol 2019;30(7):1162–1169. Crossref, MedlineGoogle Scholar
  • 8. Yankelevitz DF, Yip R, Smith JP, et al. CT screening for lung cancer: nonsolid nodules in baseline and annual repeat rounds. Radiology 2015;277(2):555–564. LinkGoogle Scholar
  • 9. Henschke CI, Yip R, Smith JP, et al. CT screening for lung cancer: part-solid nodules in baseline and annual repeat rounds. AJR Am J Roentgenol 2016;207(6):1176–1184. Crossref, MedlineGoogle Scholar
  • 10. Yip R, Yankelevitz DF, Hu M, et al. Lung cancer deaths in the National Lung Screening Trial attributed to nonsolid nodules. Radiology 2016;281(2):589–596. LinkGoogle Scholar
  • 11. Scholten ET, de Jong PA, de Hoop B, et al. Towards a close computed tomography monitoring approach for screen detected subsolid pulmonary nodules? Eur Respir J 2015;45(3):765–773 [Published correction appears in Eur Respir J 2015;45(5):1517.]. Crossref, MedlineGoogle Scholar
  • 12. Benzakoun J, Bommart S, Coste J, et al. Computer-aided diagnosis (CAD) of subsolid nodules: evaluation of a commercial CAD system. Eur J Radiol 2016;85(10):1728–1734. Crossref, MedlineGoogle Scholar
  • 13. Godoy MC, Kim TJ, White CS, et al. Benefit of computer-aided detection analysis for the detection of subsolid and solid lung nodules on thin- and thick-section CT. AJR Am J Roentgenol 2013;200(1):74–83. Crossref, MedlineGoogle Scholar
  • 14. Silva M, Schaefer-Prokop CM, Jacobs C, et al. Detection of subsolid nodules in lung cancer screening: complementary sensitivity of visual reading and computer-aided diagnosis. Invest Radiol 2018;53(8):441–449. Crossref, MedlineGoogle Scholar
  • 15. Jacobs C, van Rikxoort EM, Twellmann T, et al. Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images. Med Image Anal 2014;18(2):374–384. Crossref, MedlineGoogle Scholar
  • 16. Park S, Lee SM, Do KH, et al. Deep learning algorithm for reducing CT slice thickness: effect on reproducibility of radiomic features in lung cancer. Korean J Radiol 2019;20(10):1431–1440. Crossref, MedlineGoogle Scholar
  • 17. Chakraborty DP. Analysis of location specific observer performance data: validated extensions of the jackknife free-response (JAFROC) method. Acad Radiol 2006;13(10):1187–1193. Crossref, MedlineGoogle Scholar

Article History

Received: Aug 12 2020
Revision requested: Sept 28 2020
Revision received: Nov 19 2020
Accepted: Dec 7 2020
Published online: Feb 09 2021
Published in print: Apr 2021