FREE NEJM E-TOC    HOME   |   SUBSCRIBE   |   CURRENT ISSUE   |   PAST ISSUES   |   COLLECTIONS   |   Search Term  Advanced Search

Sign in | Get NEJM's E-Mail Table of Contents — Free | Subscribe

Published at www.nejm.org October 1, 2008 (10.1056/NEJMoa0803545)

Abstract PDF Add to Personal Archive Add to Citation Manager Notify a Friend E-mail When Cited PubMed Citation

ABSTRACT

Background The sensitivity of screening mammography for the detection of small breast cancers is higher when the mammogram is read by two readers rather than by a single reader. We conducted a trial to determine whether the performance of a single reader using a computer-aided detection system would match the performance achieved by two readers.

Methods The trial was designed as an equivalence trial, with matched-pair comparisons between the cancer-detection rates achieved by single reading with computer-aided detection and those achieved by double reading. We randomly assigned 31,057 women undergoing routine screening by film mammography at three centers in England to double reading, single reading with computer-aided detection, or both double reading and single reading with computer-aided detection, at a ratio of 1:1:28. The primary outcome measures were the proportion of cancers detected according to regimen and the recall rates within the group receiving both reading regimens.

Results The proportion of cancers detected was 199 of 227 (87.7%) for double reading and 198 of 227 (87.2%) for single reading with computer-aided detection (P=0.89). The overall recall rates were 3.4% for double reading and 3.9% for single reading with computer-aided detection; the difference between the rates was small but significant (P<0.001). The estimated sensitivity, specificity, and positive predictive value for single reading with computer-aided detection were 87.2%, 96.9%, and 18.0%, respectively. The corresponding values for double reading were 87.7%, 97.4%, and 21.1%. There were no significant differences between the pathological attributes of tumors detected by single reading with computer-aided detection alone and those of tumors detected by double reading alone.

Conclusions Single reading with computer-aided detection could be an alternative to double reading and could improve the rate of detection of cancer from screening mammograms read by a single reader. (ClinicalTrials.gov number, NCT00450359 [ClinicalTrials.gov] .)

Computer-aided detection systems use computer algorithms to analyze digital mammographic images. They identify and mark potentially suspicious regions to attract the reader's attention to features that might have been overlooked or dismissed as normal.^{6,11,12,13,14,15,16,17,18,19,20,21,22,23,24} In the United States, where single reading is standard practice, commercial computer-aided detection systems have been widely adopted to improve reader performance^{25,26,27,28,29,30,31,32,33,34,35,36} and are used in 25 to 30% of all mammogram readings.³⁷ However, the benefit of computer-aided detection in screening mammography was recently questioned in an observational study by Fenton et al.³⁵ and remains controversial.³⁸

There have been calls for robust evidence from multicenter, randomized trials involving large numbers of participants and readers to inform screening programs that are considering an investment in computer-aided detection.^{36,39,40,41,42} We conducted the prospective, randomized Computer-Aided Detection Evaluation Trial II (CADET II), in which single reading with computer-aided detection was compared with double reading during routine screening mammography in the United Kingdom National Health Service Breast Screening Programme (NHSBSP). Arbitration was used in cases of disagreement between the two readers, since this is United Kingdom practice and has been shown to reduce recall rates while maintaining high cancer-detection rates.^43,44,45

Methods

Study Design

The study was designed as an equivalence trial, with matched comparisons between the sensitivity of single reading with computer-aided detection and the sensitivity of double reading. A total of 31,057 women were recruited between September 2006 and August 2007 from the static and mobile units of three NHSBSP centers in England (Manchester; Nottingham; and Warwickshire, Solihull, and Coventry). Information about the trial was posted 3 to 6 weeks before their appointments to women who were invited to undergo routine screening by two-view film mammography. Written informed consent was obtained at the mammography appointment. The study was approved by the South East Multi-Centre Research Ethics Committee.

Randomization

Batches of films from each screening session were randomly assigned, at a ratio of 1:1:28, to one of three reading regimens. Group A, with a planned number of 1000 subjects, was assigned to receive a double reading only; group B, with a planned number of 1000 subjects, was assigned to receive a single reading with computer-aided detection only; group C, with a planned number of 28,000 subjects, was assigned to receive both a single reading with computer-aided detection and a double reading. Assignments were made with the aid of a random-number generator and provided to each center in sequentially numbered, sealed envelopes.

Film Digitization and Computer-Aided Detection Output

Trial mammograms were digitized and analyzed by the software detection algorithms of an ImageChecker DMax computer-aided detection system, version 8.1 (Hologic/R2 Technology). The prompting markers were viewed on a CheckMate flat-panel display screen superimposed on an image of the corresponding mammogram at a reduced resolution. The PeerView facility provided the reader with a high-resolution image of any region of interest that was identified by the computer-aided detection algorithm.⁴⁶ Markers were generated for masses (asterisks) and microcalcifications (solid triangles). If both a mass and microcalcification were detected, a four-pointed star was displayed. The sizes of the markers were related to the likelihood of cancer, as determined by the algorithms. In this study, the software-detection algorithm thresholds were set to operate at a detection sensitivity of approximately 88% for masses and 95% for calcifications, with corresponding false marker rates of 1.5 and 1.0, respectively, per four-film examination.⁴⁶

Film Readers

The film readers were 17 radiologists, 2 breast-cancer clinicians, and 8 trained film-reading technologists (radiographers). All film readers met the requirements of the NHSBSP quality-assurance guidelines by reading at least 5000 screening mammograms per year and participating in the annual PERFORMS self-assessment test.⁴⁷ Readers acting as single readers with computer-aided detection or as the first of the two readers in a double reading had a median of 6 years of mammography experience (interquartile range, 4 to 14 years); the second readers had slightly less experience, with a median of 5 years (interquartile range, 2 to 10 years). Before the study, the readers completed a training course in the use of computer-aided detection involving 300 to 400 subjects.^42,48

Film Reading

Films were independently double read or single read with computer-aided detection in separate reading sessions by different readers with no access to the decision of the recorded outcome from the other reading regimen. A reader who was one of the two double readers was not permitted to read the same subject's films with computer-aided detection. Mammograms from a previous round were mounted for viewing.

The readers recorded the results for each subject as "recall for further assessment" or "return to routine screening." Discordant results from double readings were arbitrated by a third reader or, in Manchester, by another pair of readers. In single reading with computer-aided detection, the readers viewed the mammogram and then accessed the computer image with prompts and reviewed any areas marked by the computer-aided detection system before recording in the National Breast Screening System their overall recommendation for "recall for further assessment" or "return to routine screening."

Outcome Measures

The primary outcome measures were the proportion of cancers detected by the two reading regimens and the recall rate. These were analyzed in a matched-pair comparison.

Statistical Analysis

The trial was designed as an equivalence study,^49,50 with a matched-pair comparison of the two regimens in the group that received both single reading with computer-aided detection and double reading. Equivalence was defined as a 95% confidence interval that ruled out a difference of more than 10% in either direction in the rate of detection of cancers. The 10% equivalence criterion is more rigorous than the 15% criterion often used in pharmaceutical equivalence studies.^51,52 Within the group receiving both regimens, the differences between recall rates for all subjects and for those with cancer were compared with the use of McNemar's method.⁵³ This method takes into account the fact that the decisions resulting from single reading with computer-aided detection and from double reading are made on the same subjects. For the primary end point, detection of cancers, the 95% confidence interval for the difference between detection rates was calculated. If the 95% confidence interval did not extend beyond ±10%, it was considered to be equivalence.

In 2004, the rate of detection of invasive and in situ breast cancers in the United Kingdom NHSBSP with the use of double reading was 6.91 cancers per 1000 women screened.⁵⁴ Increasing this rate by 11% because each mammogram was read according to two regimens in our study would result in a detection rate of 7.67 per 1000. Power calculations on the assumption of a 20% disagreement rate between the two regimens indicated a requirement of 215 cancers to establish equivalence according to our criterion of 10%. Thus, 28,000 mammograms would have to be read according to both regimens to achieve a power of 90%. In addition, to minimize reader bias, so that a reader using one of the regimens would not know whether the mammogram would be read according to the other regimen, a further 2000 mammograms, randomly assigned to either double reading or single reading with computer-aided detection, were included.⁵⁵

Results

Figure 1 shows a flow diagram of the trial. In the group receiving both single reading with computer-aided detection and double reading, 227 cancers were detected among 28,204 subjects, for an overall screening-detection rate of 8.0 per 1000. The corresponding numbers for the group receiving double reading only and the group receiving single reading with computer-aided detection only were 12 cancers among 1152 subjects (10.4 per 1000) and 8 cancers among 1182 subjects (6.8 per 1000), respectively. Table 1 shows the distribution of subjects in the group receiving both single reading with computer-aided detection and double reading according to age, type of screening (prevalence or incidence screening), and center.

View larger version (24K): [in this window] [in a new window]   Figure 1. Enrollment, Randomization, and Assessment of the Subjects. CAD denotes computer-aided detection.

 

View this table: [in this window] [in a new window]   Table 1. Characteristics of Subjects in the Group Receiving Both Single Reading with Computer-Aided Detection and Double Reading.

 
Table 2 shows the recall decisions after a single reading with computer-aided detection and after a double reading for all subjects and for all subjects with cancer in the group of women whose mammograms were read according to both regimens. The concordance between the two reading systems was 74.9% (170 of 227). The cancer-detection rates after single reading with computer-aided detection and after double reading were similar (7.02 and 7.06 per 1000, respectively; P=0.89), as were the proportions of cancers detected (87.2% and 87.7%, respectively; P=0.89). The 95% confidence interval for the difference between 87.2% and 87.7% is –7.4% to +6.6%, which is within the 10% definition of equivalence.

View this table: [in this window] [in a new window]   Table 2. Recall Decisions for All Subjects and for Those with Cancer among Women Whose Mammograms Were Read According to Both Regimens.

 
The recall rates were 3.9% for single reading with computer-aided detection and 3.4% for double reading (P<0.001). The absolute difference in recall rates was 0.5 percentage point, with a 95% confidence interval of 0.3 to 0.8 percentage point corresponding to a relative difference of 15%. The estimated sensitivity, specificity, and positive predictive value for single reading with computer-aided detection were 87.2%, 96.9%, and 18.0%, respectively. The corresponding figures for double reading were 87.7%, 97.4%, and 21.1%. The sensitivity values are overestimates, because data on interval cancers (cancers detected within 1 year after a negative mammography result) were not yet available for all subjects. The same interval cancers were missed by both regimens, and hence the difference in sensitivity will remain the same for both regimens.

Table 3 shows the recall rates according to center and screening regimen for subjects participating in the trial and for women not participating in the trial who were undergoing routine contemporaneous screening, with double reading of their mammograms. The recall rates were similar for trial subjects whose mammograms were read by a single reader with computer-aided detection and for nontrial subjects at the Nottingham center and the Warwickshire, Solihull, and Coventry center. However, at Manchester the recall rate among trial subjects undergoing single reading with computer-aided detection was 1.4 percentage points higher than the recall rate among trial subjects undergoing double reading and 1.2 percentage points higher than the recall rate for nontrial subjects. Overall, the recall rate was 0.3 percentage point higher for mammograms that were given a single reading with computer-aided detection than for routine screening mammograms of nonparticipating women. The recall decision was arbitrated for 365 of the 28,723 women (1.3%) whose mammograms received a single reading with computer-aided detection; these women included 6 of the 227 with cancer (2.6%).

View this table: [in this window] [in a new window]   Table 3. Recall Rates of Trial Subjects Whose Mammograms Were Read According to Both Regimens and Nontrial Subjects.

 
Table 4 lists the pathological features of the 57 cancers with discordant recall decisions according to screening regimen. The proportion of in situ tumors was higher for tumors detected only by single reading with computer-aided detection than for tumors detected only by double reading, but the difference was not significant (P=0.33). Among the invasive cancers, those detected only by single reading with computer-aided detection had similar node status (P=0.89) to those detected only by double reading; they were smaller (P=0.35) and less likely to be grade 1 and more likely to be grade 3 (P=0.07) than those detected only by double reading, but these differences were not significant. Thus, there were no significant differences in pathological attributes between tumors detected by single reading with computer-aided detection alone and those detected by double reading alone.

View this table: [in this window] [in a new window]   Table 4. Pathological Features of Tumors with Discordant Recall Decisions.

 
Discussion

The trial demonstrated that the cancer-detection rates of a single reader using computer-aided detection and of two readers are similar (7.02 per 1000 and 7.06 per 1000, respectively). However, the recall rate for mammograms given a single reading with computer-aided detection was 0.5 percentage point higher than that for mammograms given a double reading (3.9% and 3.4%, respectively), a relative difference of 15%.

There has been much recent debate regarding the use of computer-aided detection in clinical practice. Eight studies making the comparison between single reading with computer-aided detection and double reading were reviewed by Bennett et al.,⁵⁶ but methodologic differences precluded any definitive conclusions. Most of the studies were small, had a high proportion of subjects with cancer, and involved a limited number of readers with variable training in the use of computer-aided detection.²³ Taylor and Potts used pooled estimates of effect sizes from two meta-analyses⁷: one included 10 U.S. studies in which the comparison was made between single reading alone and single reading with computer-aided detection, and the second included 17 studies comparing single reading with double reading. The authors concluded that there was no significant difference in cancer-detection rates between single reading with computer-aided detection and double reading with arbitration in cases of disagreement, but they found that the recall rate with double reading was significantly lower than the rate with single reading with computer-aided detection. However, differences between the screening programs of the United States and the United Kingdom make these evaluations difficult to interpret.⁵⁷ Our large, prospective study helps to clarify some of these issues. The small increase in the number of women recalled for assessment in the group given a single reading with computer-aided detection as compared with the group given a double reading is consistent with the results of previously published studies.^26,30,31,58 Furthermore, our results are in line with the pooled estimates predicted by Taylor and Potts⁷ in a comparison between double reading with arbitration in cases of disagreement and single reading with computer-aided detection.

The strengths of this study are that it was a multicenter, prospective trial with more than 30,000 women randomly assigned to one of three reading groups and that the readers were unaware of the reading assignments. Reflecting practice in the United Kingdom, the readers were all high-volume readers (reading more than 5000 mammograms per year) and were breast radiologists, breast-cancer clinicians, or film-reading technologists. The single readers using computer-aided detection and the first readers of mammograms given double readings had almost identical experience in reading mammograms, with a median of 6 years of experience (interquartile range, 4 to 14 years) for both groups of readers. Thus, our study was not biased by a difference in the amount of experience of the two groups of readers, but the conclusion that the two reading regimens are equivalent applies only if the single reader with computer-aided detection has experience similar to that of a first reader in a double reading. We used double reading with arbitration in cases of disagreement, which is routine practice in the United Kingdom, because it has been shown to reduce the recall rate.^{1,2,3,4,5,6,7} In the group assigned to single reading with computer-aided detection, 1.3% of the cases were arbitrated by another reader (or pair of readers in Manchester). The recall rates at all sites were within the limits acceptable to the United Kingdom NHSBSP (<7% for prevalence screenings and <5% for incidence screenings)⁴⁷ and other European countries.⁸ The recall rates for trial subjects assigned to double reading (and at two centers, for those assigned to single reading with computer-aided detection) were similar to those for nontrial subjects assigned to double reading, a result indicating that the behavior of readers participating in double reading was not substantially altered by participation in the trial.

The main study design was a matched comparison of the results of the application of the two reading regimens to the same subject. Some interval cancers were missed by both reading regimens (i.e., the number of false negatives was the same for both regimens), but the failure to detect these cancers did not alter the absolute difference in sensitivity between the two regimens. The advantage of the matched design is that it enables a valid comparison of sensitivities to be performed before the interval cancers come to light. In the mammograms in which breast cancer was detected after discordance between the recall decisions of the two reading regimens, we observed no significant differences between the pathological attributes of the tumors detected by the two regimens.

The film-reading and arbitration procedures were not standardized among the three centers but reflected current practice at each center and were within national guidelines. Similarly, each center applied a pragmatic approach to the protocol for single reading with computer-aided detection. In two centers, there was no significant difference in recall rates between mammograms given a single reading with computer-aided detection and those given a double reading; however, the Manchester center had a significant increase of 1.4 percentage points in the rate of recalls with single reading with computer-aided detection as compared with double reading. This difference was not associated with the amount of experience or the staff category of the readers or whether the mammograms were obtained at a prevalence or an incidence screening. However, of the three centers, Manchester also had both the highest recall rates for double reading and the highest cancer-detection rates in the trial. For Warwickshire, Solihull, and Coventry; Manchester; and Nottingham, the overall cancer-detection rates within the trial were 7.1, 10.0, and 7.0 per 1000 screened, respectively (P=0.03), a result suggesting that the increased recall rate with single reading with computer-aided detection in Manchester is due to a more cautious recall policy in general in that center. During the period of the trial, the time between mammography screenings was greater than 3 years in Manchester, an interval that might have permitted more cancers to become available for detection than a shorter period might have done.

Our results suggest that single reading with computer-aided detection is an alternative to double reading; whether to adopt this technology is a question of cost-effectiveness. The additional costs of the computer-aided–detection equipment and the costs associated with an increase in recall must be balanced against the potential savings in reader time. Clearly, the cost-effectiveness of computer-aided detection requires investigation.^6,23,37 Furthermore, comparison between the performance of computer-aided detection in full-field digital mammography and its performance in film mammography (which was used in this study) will be required.⁵⁹

Double reading, which is recognized as the best method for the detection of small invasive cancers,⁶⁰ is often difficult to achieve in practice because of costs and the need for two readers. The results of this study are applicable to programs in which double reading is standard practice.⁸ Where single reading is standard practice, computer-aided detection has the potential to improve cancer-detection rates to the level achieved by double reading.

Supported by a grant from Cancer Research UK (CRUK/03/010) and by the National Health Service Cancer Screening Programme.

Professor Gilbert reports being vice-chairman of the Royal College of Radiologists Breast Group, which advises on mammography standards in the United Kingdom; Dr. Wallis reports being president of the European Society for Breast Imaging and a member of the Royal College of Radiologists Breast Group, the National Health Service Breast Screening Programme (NHSBSP) Radiology Quality Assurance Group, the NHSBSP Evaluation Group, and the NHSBSP Association of Breast Surgeons at the British Association of Surgical Oncology Audit Group, all of which advise on and set standards for breast screening and breast imaging; Dr. Astley and Dr. Boggis report receiving fees from VuComp for digitizing and transmitting data, and consulting with Hologic with no remuneration; Dr. Astley reports being a past member of the former NHSBSP Digital Steering Group; and Drs. Gilbert, Astley, and Boggis report being loaned two CAD systems by R2 Technology (now Hologic) for 12 months in 2003 for the retrospective CADET I study. No other potential conflict of interest relevant to this article was reported.

We thank E. Fawcett-Gough, P. Shann, S. Berry, and S. Ferguson at Manchester and M. Wheaton and S. Wright at Warwickshire, Solihull, and Coventry for local assistance and data management; J. Matthews and L. Checkley at Nottingham, and the radiographers at all three centers for assistance with recruitment; the screening office managers and the radiographic, clerical, and secretarial staff at all three centers for their assistance with data collection; and the women at all three centers who consented to their mammograms being used in this study.

^* The members of the Computer-Aided Detection Evaluation Trial II (CADET II) group are listed in the Appendix.

Source Information

From the Aberdeen Biomedical Imaging Centre, University of Aberdeen, Aberdeen (F.J.G., M.G.C.G.); the Department of Imaging Science and Biomedical Engineering, University of Manchester, Manchester (S.M.A.); the Department of Epidemiology, Mathematics, and Statistics, Wolfson Institute of Preventive Medicine, London (O.F.A., S.W.D.); the Cambridge Breast Unit, Addenbrookes Hospital, Cambridge (M.G.W.); the Nottingham Breast Institute, Nottingham City Hospital, Nottingham (J.J.); and the Nightingale Breast Screening Unit, Wythenshawe Hospital, Manchester (C.R.M.B.) — all in the United Kingdom.

This article (10.1056/NEJMoa0803545) was published at www.nejm.org on October 1, 2008. It will appear in the October 16 issue of the Journal.

Address reprint requests to Dr. Gilbert at the Aberdeen Biomedical Imaging Centre, University of Aberdeen, Lilian Sutton Bldg., Foresterhill, Aberdeen AB25 2ZD, Scotland, United Kingdom, or at f.j.gilbert{at}abdn.ac.uk.

References

1. Anttinen I, Pamilo M, Soiva M, Roiha M. Double reading of mammography screening films -- one radiologist or two? Clin Radiol 1993;48:414-421. [CrossRef][Web of Science][Medline]
2. Deans HE, Everington D, Cordiner C, et al. Scottish experience of double reading in the National Breast Screening Programme. Breast 1998;7:75-79. [CrossRef][Web of Science]
3. Blanks RG, Wallis MG, Moss SM. A comparison of cancer detection rates achieved by breast cancer screening programmes by number of readers, for one and two view mammography: results from the UK National Health Service Breast Screening Programme. J Med Screen 1998;5:195-201. [Free Full Text]
4. Liston JC, Dall BJ. Can the NHS Breast Screening Programme afford not to double read screening mammograms? Clin Radiol 2003;58:474-477. [CrossRef][Web of Science][Medline]
5. Ciatto S, Ambrogetti D, Risso G, et al. The role of arbitration of discordant reports at double reading of screening mammograms. J Med Screen 2005;12:125-127. [Free Full Text]
6. Helvie M. Improving mammographic interpretation: double reading and computer-aided diagnosis. Radiol Clin North Am 2007;45:801-811. [CrossRef][Web of Science][Medline]
7. Taylor P, Potts HWW. Computer aids and human second reading as interventions in screening mammography: two systematic reviews to compare effects on cancer detection and recall rate. Eur J Cancer 2008;44:798-807. [CrossRef][Web of Science][Medline]
8. Yankaskas BC, Klabunde CN, Ancelle-Park R, et al. International comparison of performance measures for screening mammography: can it be done? J Med Screen 2004;11:187-193. [Free Full Text]
9. Working Party of the Radiologists Quality Assurance Coordinating Group. Computer aided detection in mammography. Sheffield, United Kingdom: NHSBSP, 2001. (Publication no. 48.)
10. Karssemeijer N, Otten JD, Verbeek AL, et al. Computer-aided detection versus independent double reading of masses on mammograms. Radiology 2003;227:192-200. [Free Full Text]
11. Thurfjell E, Thurfjell MG, Egge E, Bjurstam N. Sensitivity and specificity of computer-assisted breast cancer detection in mammography screening. Acta Radiol 1998;39:384-388. [Web of Science][Medline]
12. Garvican L, Field S. A pilot evaluation of the R2 image checker system and users' response in the detection of interval breast cancers on previous screening films. Clin Radiol 2001;56:833-837. [CrossRef][Web of Science][Medline]
13. Moberg K, Bjurstam N, Wilczek B, Rostgard L, Egge E, Muren C. Computed assisted detection of interval breast cancers. Eur J Radiol 2001;39:104-110. [CrossRef][Web of Science][Medline]
14. Brem RF, Schoonjans JM. Radiologist detection of microcalcifications with and without computer-aided detection: a comparative study. Clin Radiol 2001;56:150-154. [CrossRef][Web of Science][Medline]
15. Birdwell RL, Ikeda DM, O'Shaughnessy KF, Sickles EA. Mammographic characteristics of 115 missed cancers later detected with screening mammography and the potential utility of computer-aided detection. Radiology 2001;219:192-202. [Free Full Text]
16. Huo Z, Giger ML, Vyborny CJ, Metz CE. Breast cancer: effectiveness of computer-aided diagnosis observer study with independent database of mammograms. Radiology 2002;224:560-568. [Free Full Text]
17. Ciatto S, Del Turco MR, Risso G, et al. Comparison of standard reading and computer aided detection (CAD) on a national proficiency test of screening mammography. Eur J Radiol 2003;45:135-138. [CrossRef][Web of Science][Medline]
18. Brem RF, Baum J, Lechner M, et al. Improvement in sensitivity of screening mammography with computer-aided detection: a multiinstitutional trial. AJR Am J Roentgenol 2003;181:687-693. [Free Full Text]
19. Ikeda DM, Birdwell RL, O'Shaughnessy KF, Sickles EA, Brenner RJ. Computer-aided detection output on 172 subtle findings on normal mammograms previously obtained in women with breast cancer detected at follow-up screening mammography. Radiology 2004;230:811-819. [Free Full Text]
20. Astley SM, Gilbert FJ. Computer-aided detection in mammography. Clin Radiol 2004;59:390-399. [CrossRef][Web of Science][Medline]
21. Castellino RA. Computer aided detection (CAD): an overview. Cancer Imaging 2005;5:17-19. [CrossRef][Medline]
22. Nishikawa RM. Current status and future directions of computer-aided diagnosis in mammography. Comput Med Imaging Graph 2007;31:224-235. [CrossRef][Web of Science][Medline]
23. Malich A, Fischer DR, Bottcher J. CAD for mammography: the technique, results, current role and further developments. Eur Radiol 2006;16:1449-1460. [CrossRef][Web of Science][Medline]
24. Computer aided detection (CAD) in mammography. Chicago: BlueCross BlueShield Association Technology Evaluation Center Assessment Program, December 2002.
25. Warren Burhenne LJ, Wood SA, D'Orsi CJ, et al. Potential contribution of computer-aided detection to the sensitivity of screening mammography. Radiology 2000;215:554-562. [Free Full Text]
26. Freer TW, Ulissey MJ. Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology 2001;220:781-786. [Free Full Text]
27. Gur D, Sumkin JH, Rockette HE, et al. Changes in breast cancer detection and mammography recall rates after the introduction of a computer-aided detection system. J Natl Cancer Inst 2004;96:185-190. [Free Full Text]
28. Destounis SV, DiNitto P, Logan-Young W, Bonaccio E, Zuley ML, Willison KM. Can computer-aided detection with double reading of screening mammograms help decrease the false-negative rate? Initial experience. Radiology 2004;232:578-584. [Free Full Text]
29. Birdwell RL, Bandodkar P, Ikeda DM. Computer-aided detection with screening mammography in a university hospital setting. Radiology 2005;236:451-457. [Free Full Text]
30. Cupples TE, Cunningham JE, Reynolds JC. Impact of computer-aided detection in a regional screening mammography program. AJR Am J Roentgenol 2005;185:944-950. [Free Full Text]
31. Morton MM, Whaley DH, Brandt KR, Amrami KK. Screening mammograms: interpretation with computer-aided detection-prospective evaluation. Radiology 2006;239:375-383. [Free Full Text]
32. Dean JC, Ilvento CC. Improved cancer detection using computer-aided detection with diagnostic and screening mammography: prospective study of 104 cancers. AJR Am J Roentgenol 2006;187:20-28. [Free Full Text]
33. Ko JM, Nicholas MJ, Mendel JB, Slanetz PJ. Prospective assessment of computer-aided detection in interpretation of screening mammography. AJR Am J Roentgenol 2006;187:1483-1491. [Free Full Text]
34. Georgian-Smith D, Moore RH, Halpern E, et al. Blinded comparison of computer-aided detection with human second reading in screening mammography. AJR Am J Roentgenol 2007;189:1135-1141. [Free Full Text]
35. Fenton JJ, Taplin SH, Carney PA, et al. Influence of computer-aided detection on performance of screening mammography. N Engl J Med 2007;356:1399-1409. [Free Full Text]
36. Gromet M. Comparison of computer-aided detection to double reading of screening mammograms: review of 231,221 mammograms. AJR Am J Roentgenol 2008;190:854-859. [Free Full Text]
37. Lindfors KK, McGahan MC, Rosenquist CJ, Hurlock GS. Computer-aided detection of breast cancer: a cost-effectiveness study. Radiology 2006;239:710-717. [Free Full Text]
38. Ciatto S, Houssami N. Computer-aided screening mammography. N Engl J Med 2007;357:83-83. [Free Full Text]
39. Houssami N, Given-Wilson R. Incorporating new technologies into clinical practice without evidence of effectiveness in prospective studies: computer-aided detection (CAD) in breast screening reinforces the need for better initial evaluation. Breast 2007;16:219-221. [CrossRef][Web of Science][Medline]
40. Brem RF. Blinded comparison of computer-aided detection with human second reading in screening mammography: the importance of the question and the critical numbers game. AJR Am J Roentgenol 2007;189:1142-1144. [Free Full Text]
41. Jiang Y, Miglioretti DL, Metz CE, Schmidt RA. Breast cancer detection rate: designing imaging trials to demonstrate improvements. Radiology 2007;243:360-367. [Free Full Text]
42. Gilbert FJ, Astley SM, McGee MA, et al. Single reading with computer-aided detection and double reading of screening mammograms in the United Kingdom National Breast Screening Program. Radiology 2006;241:47-53. [Free Full Text]
43. Duijm LE, Groenewoud JH, Hendriks JH, de Koning HJ. Independent double reading of screening mammograms in The Netherlands: effect of arbitration following reader disagreements. Radiology 2004;231:564-570. [Free Full Text]
44. Cornford EJ, Evans AJ, James JJ, Burrell HC, Pinder SE, Wilson AR. The pathological and radiological features of screen-detected breast cancers diagnosed following arbitration of discordant double reading opinions. Clin Radiol 2005;60:1182-1187. [CrossRef][Web of Science][Medline]
45. Mucci B, Athey G, Scarisbrick G. Double reading of screening mammograms: the use of a third reader to arbitrate on disagreements. Breast 1999;8:37-39. [Medline]
46. Hologic mammography. Santa Clara, CA: R2 Technology, 2007. (Accessed September 22, 2008, at http://www.hologic.com/imagechecker.)
47. Quality assurance guidelines for breast cancer screening radiology. Sheffield, United Kingdom: National Health Service Breast Screening Programme, 2005. (Publication no. 59.)
48. Astley S, Quarterman C, Al Nuaimi Y, et al. Computer-aided detection in screening mammography: the impact of training on reader performance. In: Proceedings of IWDM [International Workshop of Digital Mammography] 2004, Durham, NC, June 18–21, 2004:231-6.
49. Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. BMJ 1996;313:36-39. [Erratum, BMJ 1996;313:550.] [Free Full Text]
50. Senn SJ. Statistical issues in drug development. Chichester, England: John Wiley, 1997.
51. Lara LF, Cisneros G, Gurney M, et al. One-day quadruple therapy compared with 7-day triple therapy for Helicobacter pylori infection. Arch Intern Med 2003;163:2079-2084. [Free Full Text]
52. Gallant JE, Staszewski S, Pozniak AL, et al. Efficacy and safety of tenofovir DF vs stavudine in combination therapy in antiretroviral-naive patients: a 3-year randomized trial. JAMA 2004;292:191-201. [Free Full Text]
53. McNemar Q. Note on the sampling error of the difference between correleated proportions or percentages. Psychometrika 1947;12:153-157. [CrossRef]
54. National Health Service Breast Screening Programme. Annual review. Sheffield, United Kingdom: NHSBSP, 2004.
55. Wald NJ, Murphy P, Major P, Parkes C, Townsend J, Frost C. UKCCCR multicentre randomised controlled trial of one and two view mammography in breast cancer screening. BMJ 1995;311:1189-1193. [Free Full Text]
56. Bennett RL, Blanks RG, Moss SM. Does the accuracy of single reading with CAD (computer-aided detection) compare with that of double reading? A review of the literature. Clin Radiol 2006;61:1023-1028. [CrossRef][Web of Science][Medline]
57. Smith-Bindman R, Chu PW, Miglioretti DL, et al. Comparison of screening mammography in the United States and the United Kingdom. JAMA 2003;290:2129-2137. [Free Full Text]
58. Khoo LAL, Taylor P, Given-Wilson RM. Computer-aided detection in the United Kingdom National Breast Screening Programme: prospective study. Radiology 2005;237:444-449. [Free Full Text]
59. Del Turco MR, Ciatto S, Lazzari B, Zatelli G. New imaging techniques: the role of digital mammography. Eur J Cancer 2002;38:Suppl:S31-S32. 
60. Blanks RG, Wallis MG, Given-Wilson RM. Observer variability in cancer detection during routine repeat (incident) mammographic screening in a study of two versus one view mammography. J Med Screen 1999;6:152-158. [Free Full Text]

The members of the CADET II group were as follows: Analysis and Writing Committee: F.J. Gilbert, S.M. Astley, M.G.C. Gillan, O.F. Agbaje, M.G. Wallis, J. James, C.R.M. Boggis, S.W. Duffy; Trial Management and Site Coordination Group: F.J. Gilbert, M.G.C. Gillan, S.W. Duffy, O.F. Agbaje, S.M. Astley, M.G. Wallis, J. James, C.R.M. Boggis, H. Flight, K. Heer, J. Cooper; Statistical Analysis: S.W. Duffy, O.F. Agbaje; Film readers at participating NHSBSP centers (Warwickshire, Solihull, and Coventry; Manchester; and Nottingham): M.G. Wallis, C.R.M. Boggis, J. James, R.R. Ramachandra, M. Wilson, H.C. Burrell, C. Lewis, U.M. Beetles, E.J. Cornford, S. Garnett, E. Hurley, L.J. Hamilton, A.A. Duncan, A.K. Jain, A.J. Evans, V. Gaur, N.B. Barr, R. Wilson, R. Walker, M. Griffiths, A. Luck, L. Chapman, J. Johnson, J. Patel, J. Hackney, R. Roberts, S. Bundred.

Abstract PDF Add to Personal Archive Add to Citation Manager Notify a Friend E-mail When Cited PubMed Citation

This article has been cited by other articles:

• Karssemeijer, N., Bluekens, A. M., Beijerinck, D., Deurenberg, J. J., Beekman, M., Visser, R., van Engen, R., Bartels-Kortland, A., Broeders, M. J. (2009). Breast Cancer Screening Results 5 Years after Introduction of Digital Mammography in a Population-based Screening Program. Radiology 253: 353-358 [Abstract] [Full Text]  
• Birdwell, R. L. (2009). The Preponderance of Evidence Supports Computer-aided Detection for Screening Mammography. Radiology 253: 9-16 [Full Text]  
• Philpotts, L. E. (2009). Can Computer-aided Detection Be Detrimental to Mammographic Interpretation?. Radiology 253: 17-22 [Full Text]  
• Hall, F. M. (2009). Computer-Aided Mammography Screening. NEJM 360: 836-836 [Full Text]  
• (2008). All you need to read in the other general journals. BMJ 337: a1954-a1954 [Full Text]