BACKGROUND. Computer programs for the interpretation of electrocardiograms (ECGs) are now widely used. However, a systematic assessment of various computer programs for the interpretation of ECGs has not been performed. METHODS. We undertook a large international study to compare the performance of nine electrocardiographic computer programs with that of eight cardiologists in interpreting ECGs in 1220 clinically validated cases of various cardiac disorders. ECGs from the following groups were included in the sample: control patients (n = 382); patients with left ventricular hypertrophy (n = 183), right ventricular hypertrophy (n = 55), or biventricular hypertrophy (n = 53); patients with anterior myocardial infarction (n = 170), inferior myocardial infarction (n = 273), or combined myocardial infarction (n = 73); and patients with combined infarction and hypertrophy (n = 31). The interpretations of the computer programs and the cardiologists were compared with the clinical diagnoses made independently of the ECGs, and the computer interpretations were compared with those of the cardiologists. RESULTS. The percentage of ECGs correctly classified by the computer programs (median, 91.3 percent) was lower than that of the cardiologists (median, 96.0 percent; P less than 0.01). The median sensitivity of the computer programs was also significantly lower than that of the cardiologists in diagnosing left ventricular hypertrophy (56.6 percent vs. 63.9 percent, P less than 0.02), right ventricular hypertrophy (31.8 percent vs. 46.6 percent, P less than 0.01), anterior myocardial infarction (77.1 percent vs. 84.9 percent, P less than 0.001), and inferior myocardial infarction (58.8 percent vs. 71.7 percent, P less than 0.0001). The median total accuracy level (the percentage of correct classifications) was 6.6 percent lower for the computer programs (69.7 percent) than for the cardiologists (76.3 percent; P less than 0.001). However, the performance of the best programs nearly matched that of the most accurate cardiologists. CONCLUSIONS. Our study shows that some but not all computer programs for the interpretation of ECGs perform almost as well as cardiologists in identifying seven major cardiac disorders.
Source Information
Division of Medical Informatics, University of Leuven, Belgium.
This article has been cited by other articles:
Solomons, L., Treloar, A., Noronha, R.
(2008). Competence of psychiatric clinicians in interpreting electrocardiograms and QT intervals: can they do this? Does it matter?. Psychiatr. Bull.
32: 291-294
[Abstract][Full Text]
van Noord, C., van der Deure, W. M, Sturkenboom, M. C J M, Straus, S. M J M, Hofman, A., Visser, T. J, Kors, J. A, Witteman, J. C M, Stricker, B. H C.
(2008). High free thyroxine levels are associated with QTc prolongation in males. J Endocrinol
198: 253-260
[Abstract][Full Text]
Ikram, M. A., van Oijen, M., de Jong, F. J., Kors, J. A., Koudstaal, P. J., Hofman, A., Witteman, J. C.M., Breteler, M. M.B.
(2008). Unrecognized Myocardial Infarction in Relation to Risk of Dementia and Cerebral Small Vessel Disease. Stroke
39: 1421-1426
[Abstract][Full Text]
Aarnoudse, A.-J. L.H.J., Newton-Cheh, C., de Bakker, P. I.W., Straus, S. M.J.M., Kors, J. A., Hofman, A., Uitterlinden, A. G., Witteman, J. C.M., Stricker, B. H.C.
(2007). Common NOS1AP Variants Are Associated With a Prolonged QTc Interval in the Rotterdam Study. Circulation
116: 10-16
[Abstract][Full Text]
Kligfield, P., Gettes, L. S., Bailey, J. J., Childers, R., Deal, B. J., Hancock, E. W., van Herpen, G., Kors, J. A., Macfarlane, P., Mirvis, D. M., Pahlm, O., Rautaharju, P., Wagner, G. S.
(2007). Recommendations for the Standardization and Interpretation of the Electrocardiogram: Part I: The Electrocardiogram and Its Technology A Scientific Statement From the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society Endorsed by the International Society for Computerized Electrocardiology. J Am Coll Cardiol
49: 1109-1127
[Abstract][Full Text]
Kligfield, P., Gettes, L. S., Bailey, J. J., Childers, R., Deal, B. J., Hancock, E. W., van Herpen, G., Kors, J. A., Macfarlane, P., Mirvis, D. M., Pahlm, O., Rautaharju, P., Wagner, G. S.
(2007). Recommendations for the Standardization and Interpretation of the Electrocardiogram: Part I: The Electrocardiogram and Its Technology: A Scientific Statement From the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society Endorsed by the International Society for Computerized Electrocardiology. Circulation
115: 1306-1324
[Abstract][Full Text]
Heeringa, J., van der Kuip, D. A. M., Hofman, A., Kors, J. A., van Rooij, F. J. A., Lip, G. Y. H., Witteman, J. C. M.
(2007). Subclinical Atherosclerosis and Risk of Atrial Fibrillation: The Rotterdam Study. Arch Intern Med
167: 382-387
[Abstract][Full Text]
Akyol, M., Jalilzadeh, S., Sinner, M. F., Perz, S., Beckmann, B. M., Gieger, C., Illig, T., Wichmann, H.-E., Meitinger, T., Kaab, S., Pfeufer, A.
(2007). The common non-synonymous variant G38S of the KCNE1-(minK)-gene is not associated to QT interval in Central European Caucasians: results from the KORA study. Eur Heart J
28: 305-309
[Abstract][Full Text]
Darpo, B., Agin, M., Kazierad, D. J., Layton, G., Muirhead, G., Gray, P., Jorkasky, D. K.
(2006). Man Versus Machine: Is There an Optimal Method for QT Measurements in Thorough QT Studies?. J Clin Pharmacol
46: 598-612
[Abstract][Full Text]
van der Hooft, C. S., Heeringa, J., Brusselle, G. G., Hofman, A., Witteman, J. C. M., Kingma, J. H., Sturkenboom, M. C. J. M., Stricker, B. H. Ch.
(2006). Corticosteroids and the risk of atrial fibrillation.. Arch Intern Med
166: 1016-1020
[Abstract][Full Text]
Heeringa, J., van der Kuip, D. A.M., Hofman, A., Kors, J. A., van Herpen, G., Stricker, B. H.Ch., Stijnen, T., Lip, G. Y.H., Witteman, J. C.M.
(2006). Prevalence, incidence and lifetime risk of atrial fibrillation: the Rotterdam study. Eur Heart J
27: 949-953
[Abstract][Full Text]
Nilsson, G., Hedberg, P., Jonasson, T., Lonnberg, I., Ohrvik, J.
(2006). QTc interval and survival in 75-year-old men and women from the general population.. Europace
8: 233-240
[Abstract][Full Text]
Straus, S. M.J.M., Kors, J. A., De Bruin, M. L., van der Hooft, C. S., Hofman, A., Heeringa, J., Deckers, J. W., Kingma, J. H., Sturkenboom, M. C.J.M., Stricker, B. H. Ch., Witteman, J. C.M.
(2006). Prolonged QTc Interval and Risk of Sudden Cardiac Death in a Population of Older Adults. J Am Coll Cardiol
47: 362-367
[Abstract][Full Text]
Pfeufer, A., Jalilzadeh, S., Perz, S., Mueller, J. C., Hinterseer, M., Illig, T., Akyol, M., Huth, C., Schopfer-Wendels, A., Kuch, B., Steinbeck, G., Holle, R., Nabauer, M., Wichmann, H. -E., Meitinger, T., Kaab, S.
(2005). Common Variants in Myocardial Ion Channel Genes Modify the QT Interval in the General Population: Results From the KORA Study. Circ. Res.
96: 693-701
[Abstract][Full Text]
Goldman, L., Kirtane, A. J.
(2003). Triage of Patients with Acute Chest Pain and Possible Cardiac Ischemia: The Elusive Search for Diagnostic Perfection. ANN INTERN MED
139: 987-995
[Abstract][Full Text]
Tsai, T. L., Fridsma, D. B., Gatti, G.
(2003). Computer Decision Support as a Source of Interpretation Error: The Case of Electrocardiograms. J. Am. Med. Inform. Assoc.
10: 478-483
[Abstract][Full Text]
Kardys, I., Kors, J. A, van der Meer, I. M, Hofman, A., van der Kuip, D. A.M, C.M. Witteman, J.
(2003). Spatial QRS-T angle predicts cardiac death in a general population. Eur Heart J
24: 1357-1364
[Abstract][Full Text]
Fraser, H. S. F., Long, W. J., Naimi, S.
(2003). Evaluation of a Cardiac Diagnostic Program in a Typical Clinical Setting. J. Am. Med. Inform. Assoc.
10: 373-381
[Abstract][Full Text]
van der Meer, I. M., de Maat, M. P. M., Kiliaan, A. J., van der Kuip, D. A. M., Hofman, A., Witteman, J. C. M.
(2003). The Value of C-Reactive Protein in Cardiovascular Risk Prediction: The Rotterdam Study. Arch Intern Med
163: 1323-1328
[Abstract][Full Text]
Salerno, S. M., Alguire, P. C., Waxman, H. S.
(2003). Competency in Interpretation of 12-Lead Electrocardiograms: A Summary and Appraisal of Published Evidence. ANN INTERN MED
138: 751-760
[Abstract][Full Text]
van der Bom, J. G., Bots, M. L., Haverkate, F., Kluft, C., Grobbee, D. E.
(2003). The 4G5G polymorphism in the gene for PAI-1 and the circadian oscillation of plasma PAI-1. Blood
101: 1841-1844
[Abstract][Full Text]
Vliegenthart, R., Oudkerk, M., Song, B., van der Kuip, D.A.M., Hofman, A., Witteman, J.C.M.
(2002). Coronary calcification detected by electron-beam computed tomography and myocardial infarction. The Rotterdam Coronary Calcification Study. Eur Heart J
23: 1596-1603
[Abstract][Full Text]
Bots, M L, Elwood, P C, Nikitin, Y, Salonen, J T, Freire de Concalves, A, Inzitari, D, Sivenius, J, Trichopoulou, A, Tuomilehto, J, Koudstaal, P J, Grobbee, D E
(2002). The EUROSTROKE cohorts: a short description and data analytical approach. J. Epidemiol. Community Health
56: i2-7
[Abstract][Full Text]
Bots, M L, Nikitin, Y, Salonen, J T, Elwood, P C, Malyutina, S, Freire de Concalves, A, Sivenius, J, Di Carlo, A, Lagiou, P, Tuomilehto, J, Koudstaal, P J, Grobbee, D E
(2002). Left ventricular hypertrophy and risk of fatal and non-fatal stroke. EUROSTROKE: a collaborative study among research centres in Europe. J. Epidemiol. Community Health
56: i8-13
[Abstract][Full Text]
Bots, M L, Elwood, P C, Salonen, J T, Freire de Concalves, A, Sivenius, J, Di Carlo, A, Nikitin, Y, Benetou, V, Tuomilehto, J, Koudstaal, P J, Grobbee, D E
(2002). Level of fibrinogen and risk of fatal and non-fatal stroke. EUROSTROKE: a collaborative study among research centres in Europe. J. Epidemiol. Community Health
56: i14-18
[Abstract][Full Text]
Bots, M L, Elwood, P C, Nikitin, Y, Salonen, J T, Freire de Concalves, A, Inzitari, D, Sivenius, J, Benetou, V, Tuomilehto, J, Koudstaal, P J, Grobbee, D E
(2002). Total and HDL cholesterol and risk of stroke. EUROSTROKE: a collaborative study among research centres in Europe. J. Epidemiol. Community Health
56: i19-24
[Abstract][Full Text]
Bots, M L, Salonen, J T, Elwood, P C, Nikitin, Y, Freire de Concalves, A, Inzitari, D, Sivenius, J, Trichopoulou, A, Tuomilehto, J, Koudstaal, P J, Grobbee, D E
(2002). {gamma}-Glutamyltransferase and risk of stroke: the EUROSTROKE project. J. Epidemiol. Community Health
56: i25-29
[Abstract][Full Text]
Moons, K G M, Bots, M L, Salonen, J T, Elwood, P C, Freire de Concalves, A, Nikitin, Y, Sivenius, J, Inzitari, D, Benetou, V, Tuomilehto, J, Koudstaal, P J, Grobbee, D E
(2002). Prediction of stroke in the general population in Europe (EUROSTROKE): Is there a role for fibrinogen and electrocardiography?. J. Epidemiol. Community Health
56: i30-36
[Abstract][Full Text]
Hripcsak, G., Wilcox, A.
(2002). Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance. J. Am. Med. Inform. Assoc.
9: 1-15
[Abstract][Full Text]
Kadish, A. H., Buxton, A. E., Kennedy, H. L., Knight, B. P., Mason, J. W., Schuger, C. D., Tracy, C. M., Winters, W. L. Jr, Boone, A. W., Elnicki, M., Hirshfeld, J. W. Jr, Lorell, B. H., Rodgers, G. P., Tracy, C. M., Weitz, H. H.
(2001). ACC/AHA Clinical Competence Statement on Electrocardiography and Ambulatory Electrocardiography: A Report of the ACC/AHA/ACP-ASIM Task Force on Clinical Competence (ACC/AHA Committee to Develop a Clinical Competence Statement on Electrocardiography and Ambulatory Electrocardiography) Endorsed by the International Society for Holter and Noninvasive Electrocardiology. Circulation
104: 3169-3178
[Full Text]
Kadish, A. H., Buxton, A. E., Kennedy, H. L., Knight, B. P., Mason, J. W., Schuger, C. D., Tracy, C. M., Winters, W. L. Jr, Boone, A. W., Elnicki, M., Hirshfeld, J. W. Jr, Lorell, B. H., Rodgers, G. P., Tracy, C. M., Weitz, H. H.
(2001). ACC/AHA clinical competence statement on electrocardiography and ambulatory electrocardiography: A report of the ACC/AHA/ACP-ASIM Task Force on Clinical Competence (ACC/AHA Committee to Develop a Clinical Competence Statement on Electrocardiography and Ambulatory Electrocardiography) Endorsed by the International Society for Holter and Noninvasive Electrocardiology. J Am Coll Cardiol
38: 2091-2100
[Full Text]
Hak, A. E., Pols, H. A. P., Stehouwer, C. D. A., Meijer, J., Kiliaan, A. J., Hofman, A., Breteler, M. M. B., Witteman, J. C. M.
(2001). Markers of Inflammation and Cellular Adhesion Molecules in Relation to Insulin Resistance in Nondiabetic Elderly: The Rotterdam Study. J. Clin. Endocrinol. Metab.
86: 4398-4405
[Abstract][Full Text]
Mosterd, A, Cost, B, Hoes, A.W, de Bruijne, M.C, Deckers, J.W, Hofman, A, Grobbee, D.E
(2001). The prognosis of heart failure in the general population. The Rotterdam Study. Eur Heart J
22: 1318-1327
[Abstract]
Goodacre, S, Webster, A, Morris, F
(2001). Do computer generated ECG reports improve interpretation by accident and emergency senior house officers?. Postgrad. Med. J.
77: 455-457
[Abstract][Full Text]
Diercks, G.F.H, van Boven, A.J, Hillege, H.L, Janssen, W.M.T, Kors, J.A, de Jong, P.E, Grobbee, D.E, Crijns, H.J.G.M, van Gilst, W.H
(2000). Microalbuminuria is independently associated with ischaemic electrocardiographic abnormalities in a large non-diabetic population. The PREVEND (Prevention of REnal and Vascular ENdstage Disease) study. Eur Heart J
21: 1922-1927
[Abstract]
de Leeuw, F.-E., de Groot, J. C., Oudkerk, M., Kors, J. A., Hofman, A., van Gijn, J., Breteler, M. M. B.
(2000). Atrial fibrillation and the risk of cerebral white matter lesions. Neurology
54: 1795-1801
[Abstract][Full Text]
Hak, A. E., Pols, H. A.P., Visser, T. J., Drexhage, H. A., Hofman, A., Witteman, J. C.M.
(2000). Subclinical Hypothyroidism Is an Independent Risk Factor for Atherosclerosis and Myocardial Infarction in Elderly Women: The Rotterdam Study. ANN INTERN MED
132: 270-278
[Abstract][Full Text]
Kors, J. A., van Herpen, G., van Bemmel, J. H.
(1999). QT Dispersion as an Attribute of T-Loop Morphology. Circulation
99: 1458-1463
[Abstract][Full Text]
Mosterd, A., Hoes, A.W., de Bruyne, M.C., Deckers, J.W., Linker, D.T., Hofman, A., Grobbee, D.E.
(1999). Prevalence of heart failure and left ventricular dysfunction in the general population; The Rotterdam Study. Eur Heart J
20: 447-455
[Abstract]
de Bruyne, M.C., Hoes, A.W., Kors, J.A., Hofman, A., van Bemmel, J.H., Grobbee, D.E.
(1999). Prolonged QT interval predicts cardiac and all-cause mortality in the elderly: The Rotterdam Study. Eur Heart J
20: 278-284
[Abstract]
Macfarlane, P. W., McLaughlin, S. C., Rodger, J. C.
(1998). Influence of Lead Selection and Population on Automated Measurement of QT Dispersion. Circulation
98: 2160-2167
[Abstract][Full Text]
Kors, J A, van Herpen, G
(1998). Measurement error as a source of QT dispersion: a computerised analysis. Heart
80: 453-458
[Abstract][Full Text]
Panju, A. A., Hemmelgarn, B. R., Guyatt, G. H., Simel, D. L.
(1998). Is This Patient Having a Myocardial Infarction?. JAMA
280: 1256-1263
[Abstract][Full Text]
Coumel, P., Maison-Blanche, P., Badilini, F.
(1998). Dispersion of Ventricular Repolarization : Reality? Illusion? Significance?. Circulation
97: 2491-2493
[Full Text]
van der Bom, J. G., de Maat, M. P. M., Bots, M. L., Haverkate, F., de Jong, P. T. V. M., Hofman, A., Kluft, C., Grobbee, D. E.
(1998). Elevated Plasma Fibrinogen : Cause or Consequence of Cardiovascular Disease?. Arterioscler. Thromb. Vasc. Bio.
18: 621-625
[Abstract][Full Text]
de Bruyne, M. C., Hoes, A. W., Kors, J. A., Hofman, A., van Bemmel, J. H., Grobbee, D. E.
(1998). QTc Dispersion Predicts Cardiac Mortality in the Elderly : The Rotterdam Study. Circulation
97: 467-472
[Abstract][Full Text]
Meijer, W. T., Hoes, A. W., Rutgers, D., Bots, M. L., Hofman, A., Grobbee, D. E.
(1998). Peripheral Arterial Disease in the Elderly : The Rotterdam Study. Arterioscler. Thromb. Vasc. Bio.
18: 185-192
[Abstract][Full Text]
Janssen, J. A. M. J. L., Stolk, R. P., Pols, H. A. P., Grobbee, D. E., Lamberts, S. W. J.
(1998). Serum Total IGF-I, Free IGF-I, and IGFBP-1 Levels in an Elderly Population : Relation to Cardiovascular Risk Factors and Disease. Arterioscler. Thromb. Vasc. Bio.
18: 277-282
[Abstract][Full Text]
RuDusky, B. M., RuDusky, B. M.
(1997). Errors of Computer Electrocardiography. ANGIOLOGY
48: 1045-1050
[Abstract]
Miller, R. A., Gardner, R. M.
(1997). Recommendations for Responsible Monitoring and Regulation of Clinical Software Systems. J. Am. Med. Inform. Assoc.
4: 442-457
[Abstract][Full Text]
van der Bom, J. G., de Knijff, P., Haverkate, F., Bots, M. L., Meijer, P., de Jong, P. T. V. M., Hofman, A., Kluft, C., Grobbee, D. E.
(1997). Tissue Plasminogen Activator and Risk of Myocardial Infarction: The Rotterdam Study. Circulation
95: 2623-2627
[Abstract][Full Text]
Bots, M. L., van der Wilk, E. C., Koudstaal, P. J., Hofman, A., Grobbee, D. E.
(1997). Transient Neurological Attacks in the General Population : Prevalence, Risk Factors, and Clinical Relevance. Stroke
28: 768-773
[Abstract][Full Text]
Brailer, D. J., Kroch, E., Pauly, M. V.
(1997). The impact of Computer-assisted Test Interpretation on Physician Decision Making: The Case of Electrocardiograms. Med Decis Making
17: 80-86
[Abstract]
Ott, A., Breteler, M. M.B., de Bruyne, M. C., van Harskamp, F., Grobbee, D. E., Hofman, A.
(1997). Atrial Fibrillation and Dementia in a Population-Based Study: The Rotterdam Study. Stroke
28: 316-321
[Abstract][Full Text]
Hillson, S. D., Connelly, D. P., Yuanli Liu,
(1995). The Effects of Computer-assisted Electrocardiographic Interpretation on Physicians' Diagnostic Decisions. Med Decis Making
15: 107-112
[Abstract]
(1991). DO COMPUTERS RELIABLY INTERPRET ECGs. JWatch General
1991: 4-4
[Full Text]
Previous
