Added Value of Prereading Screening Mammograms for Breast Cancer by Radiologic Technologists on Early Screening Outcomes
Abstract
Background
In the Dutch breast cancer screening program, mammograms are preread by technologists to identify possible abnormalities, leading to “warning signals” (an audible and visual alert if the technologist observed an abnormality suspicious for cancer) for radiologists. The best moment to present these warning signals is unknown.
Purpose
To determine the effect that blinding of technologists’ warning signals has on radiologists’ early screening outcome measures during interpretation of mammograms.
Materials and Methods
In this prospective study from September 2017 to May 2019, on alternating months, radiologists were either blinded or nonblinded to the warning signals of the technologist when interpreting screening mammograms for breast cancer. All discrepancies between radiologists and technologists were reviewed during quality assurance sessions every 6 weeks, which could result in secondary recalls. The outcome measures of this study were recall rate, cancer detection rate, and positive predictive value of recall. A χ2 test was used to test for differences between the two groups.
Results
During the study period, 109 596 women (mean age, 62 years ± 7 [standard deviation]), including 53 291 in the blinded and 56 305 in the nonblinded groups, participated. The overall recall rate (including secondary recalls) was lower for women in the blinded group than in the nonblinded group (blinded: 1140 of 53 291 women [2.1%], nonblinded: 1372 of 56 305 women [2.4%]; P = .001). There was no evidence of cancer detection rate differences between the groups (blinded: 349 of 53 291 women [6.5 per 1000 screening examinations], nonblinded: 360 of 56 305 women [6.4 per 1000 screening examinations]; P = .75). The blinded group thus had a higher positive predictive value of recall (blinded: 349 of 1140 women [30.6%], nonblinded: 360 of 1372 women [26.2%]; P = .02).
Conclusion
While interpreting screening mammograms for breast cancer, radiologists blinded to technologists’ warning signals had lower recall rates with higher positive predictive values than nonblinded radiologists, yet cancer detection rates seemed to remain unchanged.
See also the editorial by Hofvind and Lee in this issue.
© RSNA, 2021
Summary
While interpreting screening mammograms for breast cancer, radiologists blinded to technologists’ warning signals had lower recall rates with higher positive predictive values than nonblinded radiologists, yet cancer detection rates seemed to remain unchanged.
Key Results
■ In this prospective study of 109 596 women screened for breast cancer with mammography, radiologists blinded to technologists’ warning signals had lower overall recall rates (2.1% vs 2.4%, P = .001) and higher positive predictive values of recall (30.6% vs 26.2%, P = .02) compared with those not blinded to warning signals.
■ Secondary recalls resulted in the detection of 19 additional cancers with blinding and one additional cancer with nonblinding.
■ There was no evidence of cancer detection rate differences between the blinded and nonblinded groups (6.5 per 1000 screening examinations vs 6.4 per 1000 screening examinations, respectively; P = .75).
Introduction
In the Netherlands, a population-based breast cancer screening program was set up in 1989 to reduce breast cancer mortality (1,2). Combined with state-of-the-art treatment, early detection of breast cancer with mammography is still the most effective strategy to achieve a substantial reduction in mortality from this disease (3,4).
Within the Dutch breast cancer screening program, the examinations are obtained by a technologist specializing in mammography. Dutch screening technologists are trained to assess mammograms for possible mammographic abnormalities (prereading) to provide screening radiologists with “warning signals” (an audible and visual alert if the technologist observed an abnormality suspicious for cancer). At the introduction of prereading (in January 2003), the decision was made to present these warning signals to the screening radiologist when opening a screening examination. Involving technologists in reading mammograms is expected to give them an additional challenge in their work, which increases motivation and makes them more aware of the importance of producing high-quality mammograms (5). Furthermore, by looking at the mammograms from the perspective of the radiologist, the technologist is more likely to obtain an additional image to avoid an unnecessary recall. To improve their skills in reading mammograms, technologists attend a quality assurance session every 6 weeks, in which they review a selection of screening examinations with a screening radiologist.
The performance of technologists in prereading or as a second reader in mammography screening has been investigated in several previous studies. These studies, mostly performed during the screen-film mammography era, provided evidence that technologists can learn to read mammograms when given adequate training (5–11). In contrast, our study was designed to investigate the influence of prereading by technologists on the performance of radiologists and the best moment to present the warning signals. We hypothesized that blinding radiologists to warning signals during their interpretation of mammograms would lower the recall rate. To our knowledge, this topic has not been previously explored.
The aim of our study was to determine the effect that blinding of technologists’ warning signals has on radiologist early screening outcome measures during interpretation of mammograms, including recall rate, cancer detection rate, and positive predictive value (PPV) of recall. We also investigated whether prereading by technologists influences the distribution of tumor characteristics and mammographic image features of screening-detected cancers.
Materials and Methods
Study Participants
This prospective study was performed between September 2017 and May 2019 in four units (Eindhoven, Kempen, Den Bosch, Meierij) in a southern region of the Dutch nationwide breast cancer screening program. By participating in screening, women consent to making their data available for evaluation purposes and research unless they choose to opt out explicitly. We did not receive any data for women who objected to the use of their data. This study was performed within the national permit for breast cancer screening issued by the Ministry of Health, Welfare and Sports and did not require additional approval by a local institutional review board.
Standard Mammography Screening Procedure
Details of the Dutch breast cancer screening program have been described previously (12,13). In short, the Dutch screening program offers biennial mammography to women aged 50–75 years, who are invited to attend by a personal letter. Full-field digital mammographic examinations are performed by a technologist specializing in mammography and consist of a mediolateral oblique and a craniocaudal view of each breast. All examinations are performed using a Lorad Selenia mammography system (Hologic).
Immediately after obtaining the mammograms, the technologist checks the images for any abnormalities suspicious for cancer (prereading) in a reading room equipped with a dedicated workstation (Coronis 3MP [MDCG 3120-CB], Barco). Prior screening mammograms are available for comparison in the case of subsequent examinations. For each positive mammogram, the technologist annotates the location and type of abnormality. At the discretion of the technologist, extra views (eg, Cleopatra view) or repeated views can be obtained.
Each mammogram is double read by two certified screening radiologists in a blinded fashion (ie, the second reader is unaware of the first reader’s decision). The mammograms are viewed on a SecurView mammography screening workstation (Hologic) with 5-megapixel monitors (Coronis 5MP Mammo [MFGD 5621 HD], Barco). When opening a new examination, the radiologist receives an audible and visual warning alert if the technologist observed an abnormality suspicious for cancer. Like the technologists, the radiologists have prior screening mammograms available for comparison in the case of subsequent examinations. Radiologists classify each examination according to the Breast Imaging Reporting and Data System (BI-RADS) (14). Women with BI-RADS category 1 or 2 findings are not recalled. Women with BI-RADS category 0, 4, or 5 findings are recalled to a hospital for further assessment (referred to as “primary recall”). The program does not allow a BI-RADS category 3 classification because there is no short-term follow-up available in the screening setting (15). In case of a discordant assessment, which occurs in about 2% of all double readings, arbitration by a third screening radiologist is applied. After arbitration, all examinations with a warning signal and no primary recall are reviewed during quality assurance sessions performed every 6 weeks that are supervised by a screening radiologist (J.N. and W.S.P., with 4 and 7 years of experience, respectively, at the start of the study [September 2017]). For each case, the supervising radiologist decides whether a secondary recall (BI-RADS category 0, 4, or 5) is necessary.
Modified Screening Procedure for This Study
The examinations were performed by 41 technologists. The median years of experience of the technologists was 14.6 (range, 0.5–25.6 years), and they perform and preread a median of 1926 screening mammograms per year. Technologists spend 4 hours on prereading in their initial 5-day screening course. Every 3 years, technologists receive at least 12 hours of supplemental training, including 4 hours on prereading of mammograms. In addition, they attend a quality assurance session every 6 weeks.
The examinations were read by 14 screening radiologists (including W.S.P., J.N., B.K., L.E.M.D). The median years of experience of the screening radiologists was 12 (range, 1–23 years), and they read a median of approximately 9000 screening mammograms per year. Radiologists are obliged to participate in an 8-day initial screening course, read a minimum of 3000 screening examinations per year, and obtain at least 40 continuing medical education credits over 5 years.
During the initial reading, on alternating months, the radiologists were either blinded or nonblinded to the warning signals of the technologist. Nonblinded is the standard procedure. When blinded, the warning signals were not presented at all.
Assessment after Recall and Follow-up
Recalled women underwent further assessment in one of 15 hospitals with a specialized breast unit. During a 1-year follow-up period, all radiology and pathology reports were collected (L.E.M.D.). Data were collected on diagnostic procedures, biopsy results, and breast cancer diagnoses, including TNM classification (16) and tumor characteristics.
Statistical Analysis
The outcome measures were the recall rate (in percentage), cancer detection rate (screening-detected cancers per 1000 screening examinations), and PPV of recall (in percentage). The χ2 test was used to test for differences between the blinded and nonblinded groups in all outcome measures as well as differences in the distribution of tumor characteristics. P < .05 indicated a statistically significant difference. Statistical analyses were performed by one author (T.D.G., with 6 years of experience) using Statistical Package for Social Sciences software (SPSS, version 27.0; IBM SPSS Statistics).
Results
Participant Characteristics
A total of 109 596 women (mean age, 62 years ± 7 [standard deviation]) who underwent screening examinations were included during the study period (Table 1). During reading, the radiologists were blinded to the warning signals of the technologists for 53 291 screening examinations and nonblinded for 56 305 (Fig 1). Of the 53 291 screening examinations in the blinded group, 4695 (8.81%) had extra views. Although the technologists gave warning signals for 1339 of the 53 291 screening examinations (2.5%), they were not presented to the radiologists during reading (radiologists were blinded); after resolution of discordant readings, 566 were classified as BI-RADS category 0 (with the technologist giving a warning signal in 161 screening examinations [28.4%]), 455 as BI-RADS category 4 (with the technologist giving a warning signal in 220 screening examinations [48.4%]), 68 as BI-RADS category 5 (with the technologist giving a warning signal in 58 screening examinations [85%]), and 52 202 as BI-RADS category 1 or 2 (with the technologist giving a warning signal in 900 screening examinations [1.7%]). The 900 BI-RADS category 1 or 2 scores for which the technologists had given a warning signal were reviewed during a quality assurance session. Of the 56 305 screening examinations in the nonblinded group, 4977 (8.84%) had extra views (repeated views for technical reasons and additional views). There were warning signals in 1408 of the 56 305 screening examinations (2.5%) in the nonblinded group. After resolution of discordant readings, 724 of the 56 305 screening examinations in the nonblinded group were classified as BI-RADS category 0 (with warning signals presented in 290 screening examinations [40.1%]), 571 as BI-RADS category 4 (with warning signals presented in 276 screening examinations [48.3%]), 75 as BI-RADS category 5 (with warning signals presented in 61 screening examinations [81%]), and 54 935 as BI-RADS category 1 or 2 (with warning signals presented in 781 screening examinations [1.4%]). The 781 BI-RADS category 1 or 2 screening examinations with a warning signal were reviewed during a quality assurance session.
 |
Figure 1: Flowchart of the study participants divided into a blinded group (no warning signal presented to the radiologist) and a nonblinded group (warning signal presented to the radiologist). BI-RADS = Breast Imaging and Reporting Data System.
Screening Results of the Blinded Group versus the Nonblinded Group, Excluding Secondary Recalls
The primary recall rate was lower for women in the blinded group than for those in the nonblinded group (blinded: 1089 of 53 291 screening examinations [2.0%], nonblinded: 1370 of 56 305 screening examinations [2.4%]; P < .001) (Table 2). There was no evidence of cancer detection rate differences between the groups (blinded: 330 of 53 291 screening examinations [6.2 per 1000 screening examinations], nonblinded: 359 of 56 305 screening examinations [6.4 per 1000 screening examinations]; P = .70), yielding a higher PPV of recall for the blinded group (blinded: 330 of 1089 screening examinations [30.3%], nonblinded: 359 of 1370 screening examinations [26.2%]; P = .03).
 |
Screening Results of the Blinded Group versus the Nonblinded Group, Including Secondary Recalls
At the quality assurance sessions where the technologists and one of the two supervising screening radiologists were present, more women were recalled at the second instance in the blinded group compared with the nonblinded group (51 secondary recalls in the blinded group vs two secondary recalls in the nonblinded group) (Fig 1). Nevertheless, the overall recall rate including these secondary recalls (blinded: 1140 of 53 291 screening examinations [2.1%], nonblinded: 1372 of 56 305 screening examinations [2.4%]; P = .001) was lower in the blinded group than in the nonblinded group (Table 2). The secondary recalls resulted in the additional detection of 19 cancers in the blinded group and one cancer in the nonblinded group (Fig 1). However, we did not find evidence that these additional screening-detected cancers resulted in a higher cancer detection rate, not even in the blinded group (detection rate of primary recalls in blinded group: 330 of 53 291 screening examinations [6.2 per 1000 screening examinations], detection rate of overall recalls in blinded group: 349 of 53 291 screening examinations [6.5 per 1000 screening examinations]; P = .46). We found no evidence of cancer detection rate differences, including the cancers detected at secondary recalls, between the blinded and nonblinded groups (blinded: 349 of 53 291 screening examinations [6.5 per 1000 screening examinations], nonblinded: 360 of 56 305 screening examinations [6.4 per 1000 screening examinations]; P = .75). The overall PPV of recall including the secondary recalls was higher for the blinded group than for the nonblinded group (349 of 1140 screening examinations [30.6%] vs 360 of 1372 screening examinations [26.2%], respectively; P = .02). The PPV of recall of only the secondary recalls was 37% (19 of 51 screening examinations) for the blinded group and 50% (one of two screening examinations) for the nonblinded group (P = .62).
Tumor Characteristics of the Detected Cancers in the Blinded and Nonblinded Groups
Table 3 shows the tumor characteristics of all screening-detected breast cancers in both the blinded and nonblinded groups. We found no evidence of a difference in tumor characteristics between the blinded and nonblinded groups (P value range, .25–.93). In the blinded group, the proportion of ductal carcinoma in situ among detected cancers following secondary recalls was higher than that observed in primary recalls (primary recalls: 61 of 330 cancers [18.5%], secondary recalls: nine of 19 cancers [47%]; P = .002).
 |
Mammographic Features of Technologists’ Warning Signals
In the blinded group, the technologists annotated a suspicious abnormality in 1339 of 53 291 screening examinations (2.5%) at prereading (Figs 2, 3). Table 4 presents the different mammographic features of those abnormalities. The mammographic abnormalities were most often masses (767 of 1339 screening examinations [57.3%]), followed by calcifications (237 of 1339 screening examinations [17.7%]) and asymmetries (153 of 1339 screening examinations [11.4%]). In the blinded group, the warning signals were not presented to the radiologists during interpretation of the screening examinations but were reviewed at the quality assurance sessions, which resulted in 51 secondary recalls. The majority of these secondary recalls were related to calcifications and asymmetries (both 18 of 51 screening examinations [35%]). Of the 19 breast cancers detected through secondary recall, 11 (58%) showed suspicious calcifications on the screening mammogram. The substantial number of warning signals for masses (767 of 1339 [57.3%]) resulted in only six secondary recalls, yielding one breast cancer. Most false-positive secondary recalls were based on asymmetries (14 of 32 screening examinations [44%]). The PPV of recall was highest for calcifications (11 of 18 screening examinations [61%]), followed by architectural distortions (three of five screening examinations [60%]), and was lowest for masses with or without calcifications (0 of four screening examinations [0%] and one of six screening examinations [17%], respectively).
Figure 2: (A) Left mediolateral oblique view and (B) Left craniocaudal view in 74-year-old woman who underwent two-view screening mammography in 2018. Images show an architectural distortion (arrow), classified by the technologist as Breast Imaging Reporting and Data System category 4. Initially, the woman was not recalled after the radiologist’s double reading, but when technologists gave a warning signal for this screening examination, the woman was recalled at second instance after the quality assurance session. US-guided core-needle biopsy revealed invasive lobular cancer (Bloom-Richardson grade 2, estrogen and progesterone receptor positive), without signs of axillary metastasis. Because of tumor type and discrepancy in size between mammography and US (3 cm vs 2 cm, respectively), an additional MRI scan was obtained. The tumor size determined at MRI was 2.5 cm.
Figure 3: (A) Left craniocaudal view and (B) magnified image in 58-year-old woman who underwent two-view screening mammography in 2018 show new (compared with prior mammograms in 2016) grouped, fine pleomorphic calcifications (arrow). The technologist classified this finding as Breast Imaging Reporting and Data System category 4. Initially, the woman was not recalled after the radiologist’s double reading but was recalled at second instance after the quality assurance session. (C) Left craniocaudal view and (D) magnified image obtained in the hospital in 2018 for assessment after recall confirm calcifications (arrow in C). Magnification view shows more details, with calcifications spread over an area of 40 mm. Stereotactic vacuum-assisted biopsy yielded grade 2 ductal carcinoma in situ. Mastectomy was performed at the patient’s request, with a final diagnosis of a 70-mm grade 2 ductal carcinoma in situ.
 |
Discussion
In the Dutch breast cancer screening program, technologists preread mammograms to identify possible abnormalities, leading to warning signals for radiologists. The best moment to present these warning signals is unknown. In this prospective study, we evaluated the effect that blinding of technologists’ prereading findings has on radiologist early screening outcomes. We showed that blinding radiologists to the prereading findings resulted in a lower overall recall rate (blinded group: 2.1%, nonblinded group: 2.4%; P = .001) and a higher positive predictive value of recall (blinded group: 30.6%, nonblinded group: 26.2%; P = .02) when compared with nonblinding. However, we found no evidence of a difference in the cancer detection rate (blinded group: 6.5 per 1000 screening examinations, nonblinded group: 6.4 per 1000 screening examinations; P = .75).
In a breast cancer screening program, one continuously strives to find an optimal balance between recall rate and cancer detection rate. The recall rate in the Netherlands is one of the lowest worldwide (17,18). Low recall rates may result in more missed subtle cancers. However, high recall rates result in unnecessary further assessment, patient anxiety, and increased costs. Different factors, such as national health policy issues (eg, malpractice concerns) and characteristics of the screened population, may strongly influence recall rates in different screening programs (19,20).
Our study shows that it is not effective to expose radiologists to technologists’ warning signals during their initial interpretation of the screening mammograms. In general, radiologists are getting more and more signals or markers to support them in detecting lesions on medical images. Due to the introduction of artificial intelligence, new computer-aided detection systems are becoming available for different modalities (21)—for example, for mammography or tomosynthesis in breast screening and for CT in lung screening. When implementing such tools in practice, careful evaluation is needed to decide whether these markers should be presented during or after reading images.
In a recent study performed within our screening region, Coolen et al (22) found a limited role for quality assurance sessions in additional cancer detection. Less than 1% of the screening-detected cancers were additionally detected through these sessions. In that study, the radiologists were not blinded to the technologists’ interpretation. We found no evidence of a difference in cancer detection rate between the blinded and nonblinded group, but looking at the characteristics of the additional cancers, these appear to be clinically relevant findings, including grade 2 or 3 ductal carcinoma in situ and grade 2 or 3 invasive cancers.
More than half of the warning signals (767 of 1339 [57.3%]) given by the technologists concerned masses, resulting in only one additional cancer after secondary recall. Most cancers diagnosed through secondary recall (11 of 19 [58%])showed suspicious calcifications. The proportion of ductal carcinoma in situ in these additional cancers was higher than that observed after primary recall, which is in accordance with previous studies (23,24). Our results further show that warning signals concerning calcifications have the highest cancer probability, while masses have the lowest. These findings correspond with those reported by Coolen et al (24) and Wivell et al (25). There is no obvious explanation, but we hypothesize that technologists may spend more time looking for calcifications and, in contrast to the radiologists, do not read mammograms in batches.
Our study has a few limitations. First, no data are available on interval cancers—that is, cancers diagnosed between two consecutive screening rounds following a negative screening examination. Therefore, program sensitivity and specificity could not be calculated as outcome measures and we did not have information on interval cancers for which the technologists had given a warning signal. Second, due to the very low number of cancers in a screening population, we could not identify small differences in cancer detection rate. However, we were mainly concerned with the effect on the recall rate. Third, when not blinded to warning signals, all radiologists made their own decisions on how to apply those signals. In contrast, in the blinded group, only two radiologists (J.N., W.S.P.) saw the warning signals during the quality assurance sessions and decided whether secondary recall was necessary. Thus, the study design may have influenced the number of secondary recalls.
In conclusion, our results showed that radiologists interpreting screening mammograms for breast cancer who were blinded to technologists’ warning signals had lower recall rates with higher positive predictive values than nonblinded radiologists, yet cancer detection rates seemed to remain unchanged. Thus, we suggest that radiologists should be blinded to warning signals of technologists while interpreting screening mammograms. The technologists’ findings could be presented immediately after interpretation to give the radiologists the opportunity to change their recall decision. Additional cancers may be detected through review at quality assurance sessions. Future research is necessary to investigate whether and to what extent such a new procedure increases the performance of radiologists and to ensure the effect of the quality assurance sessions. In addition, we will investigate technologists’ additional views and whether these views prevent unnecessary recalls.
Acknowledgments
The authors thank the entire team of screening radiologists, technologists, data analysts, and secretarial support of our screening region.
Author Contributions
Author contributions: Guarantors of integrity of entire study, T.D.G., E.T., L.E.M.D.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; agrees to ensure any questions related to the work are appropriately resolved, all authors; literature research, T.D.G., W.S.P., B.K., R.M.P., L.E.M.D.; clinical studies, J.N., B.K., E.T., R.M.P., L.E.M.D.; statistical analysis, T.D.G., D.v.d.W., B.K., M.J.M.B., L.E.M.D.; and manuscript editing, all authors
References
- 1. . Reduction of breast cancer mortality through mass screening with modern mammography. First results of the Nijmegen project, 1975-1981. Lancet 1984;1(8388):1222–1224.
- 2. . Evaluation of screening for breast cancer in a non-randomised study (the DOM project) by means of a case-control study. Lancet 1984;1(8388):1224–1226.
- 3. . Worldwide Review and Meta-Analysis of Cohort Studies Measuring the Effect of Mammography Screening Programmes on Incidence-Based Breast Cancer Mortality. Cancers (Basel) 2020;12(4):976–991.
- 4. . The benefits and harms of breast cancer screening: an independent review. Br J Cancer 2013;108(11):2205–2240.
- 5. . Medical radiologic technologist review: effects on a population-based breast cancer screening program. Radiology 1999;211(2):529–533.
- 6. . Radiographers supporting radiologists in the interpretation of screening mammography: a viable strategy to meet the shortage in the number of radiologists. BMC Cancer 2015;15(1):410.
- 7. . An observational study to evaluate the performance of units using two radiographers to read screening mammograms. Clin Radiol 2012;67(2):114–121.
- 8. . A retrospective study of the performance of radiographers in interpreting screening mammograms. Radiography 2011;17(2):126–131.
- 9. . Introduction of additional double reading of mammograms by radiographers: effects on a biennial screening programme outcome. Eur J Cancer 2008;44(9):1223–1228.
- 10. . Prescreening mammography by technologists: a preliminary assessment. AJR Am J Roentgenol 2003;180(1):253–256.
- 11. . Performance of radiographers in mammogram interpretation: a systematic review. Breast 2008;17(1):85–90.
- 12. . The effect of population-based mammography screening in Dutch municipalities on breast cancer mortality: 20 years of follow-up. Int J Cancer 2017;141(4):671–677.
- 13. . Likelihood of early detection of breast cancer in relation to false-positive risk in life-time mammographic screening: population-based cohort study. Ann Oncol 2013;24(10):2501–2506.
- 14. . ACR BI-RADS Mammography. In: ACR BI-RADS Atlas, Breast Imaging Reporting and Data System.Reston, Va:American College of Radiology,2013.
- 15. . The Breast Imaging Reporting and Data System (BI-RADS) in the Dutch breast cancer screening programme: its role as an assessment and stratification tool. Eur Radiol 2012;22(8):1717–1723.
- 16. . AJCC Cancer Staging Manual. American Joint Commission on Cancer. 8th ed. Basel, Switzerland:Springer International,2017.
- 17. . Effect of recall rate on earlier screen detection of breast cancers based on the Dutch performance indicators. J Natl Cancer Inst 2005;97(10):748–754.
- 18. . Cancer screening in the European Union. Report on the implementation of the council recommendation on cancer screening. Second report.Brussels, Belgium:European Commission,2017.
- 19. . International variation in screening mammography interpretations in community-based programs. J Natl Cancer Inst 2003;95(18):1384–1393.
- 20. . Is the false-positive rate in mammography in North America too high? Br J Radiol 2016;89(1065):20160045.
- 21. . Current Clinical Applications of Artificial Intelligence in Radiology and Their Best Supporting Evidence. J Am Coll Radiol 2020;17(11):1371–1381.
- 22. . Additional breast cancer detection at digital screening mammography through quality assurance sessions between technologists and radiologists. Radiology 2020;294(3):509–517.
- 23. . Additional double reading of screening mammograms by radiologic technologists: impact on screening performance parameters. J Natl Cancer Inst 2007;99(15):1162–1170.
- 24. . Incorporation of the technologist’s opinion for arbitration of discrepant assessments among radiologists at screening mammography. Breast Cancer Res Treat 2018;171(1):143–149.
- 25. . Can radiographers read screening mammograms? Clin Radiol 2003;58(1):63–67.
Article History
Received: Mar 19 2021Revision requested: May 14 2021
Revision received: Aug 13 2021
Accepted: Sept 1 2021
Published online: Nov 09 2021
Published in print: Feb 2022
