Internist-I is an experimental computer program capable of making multiple and complete diagnoses in internal medicine. It differs from most other programs for computer-assisted diagnosis in the generality of its approach and the size and diversity of its knowledge base. To document the strengths and weaknesses of the program we performed a systematic evaluation of the capabilities of INTERNIST-I. Its performance on a series of 19 clinicopathological exercises (Case Records of the Massachusetts General Hospital) published in the Journal appeared qualitatively similar to that of the hospital clinicians but inferior to that of the case discussants. The evaluation demonstrated that the present form of the program is not sufficiently reliable for clinical applications. Specific deficiencies that must be overcome include the program's inability to reason anatomically or temporally, its inability to construct differential diagnoses spanning multiple areas, its occasional attribution of findings to improper causes, and its inability to explain its "thinking".
This article has been cited by other articles:
Rosenbloom, S. T., Miller, R. A., Johnson, K. B., Elkin, P. L., Brown, S. H.
(2006). Interface Terminologies: Facilitating Direct Entry of Clinical Data into Electronic Health Record Systems. J. Am. Med. Inform. Assoc.
13: 277-288
[Abstract][Full Text]
Rosenbloom, S. T., Geissbuhler, A. J., Dupont, W. D., Giuse, D. A., Talbert, D. A., Tierney, W. M., Plummer, W. D., Stead, W. W., Miller, R. A.
(2005). Effect of CPOE User Interface Design on User-Initiated Access to Educational and Patient Information during Clinical Care. J. Am. Med. Inform. Assoc.
12: 458-473
[Abstract][Full Text]
Moldoveanu, M. C., Bauer, R. M.
(2004). On the Relationship Between Organizational Complexity and Organizational Structuration. Organization Science
15: 98-118
[Abstract]
Ramnarayan, P., Kapoor, R. R., Coren, M., Nanduri, V., Tomlinson, A. L., Taylor, P. M., Wyatt, J. C., Britto, J. F.
(2003). Measuring the Impact of Diagnostic Decision Support on the Quality of Clinical Decision Making: Development of a Reliable and Valid Composite Score. J. Am. Med. Inform. Assoc.
10: 563-572
[Abstract][Full Text]
Fraser, H. S. F., Long, W. J., Naimi, S.
(2003). Evaluation of a Cardiac Diagnostic Program in a Typical Clinical Setting. J. Am. Med. Inform. Assoc.
10: 373-381
[Abstract][Full Text]
Pan, K.-H., Lih, C.-J., Cohen, S. N.
(2002). Analysis of DNA microarrays using algorithms that employ rule-based expert knowledge. Proc. Natl. Acad. Sci. USA
99: 2118-2123
[Abstract][Full Text]
Hripcsak, G., Wilcox, A.
(2002). Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance. J. Am. Med. Inform. Assoc.
9: 1-15
[Abstract][Full Text]
Friedman, C. P., Elstein, A. S., Wolf, F. M., Murphy, G. C., Franz, T. M., Heckerling, P. S., Fine, P. L., Miller, T. M., Abraham, V.
(1999). Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation: A Multisite Study of 2 Systems. JAMA
282: 1851-1856
[Abstract][Full Text]
Berner, E. S., Maisiak, R. S., Cobbs, C. G., Taunton, O. D.
(1999). Effects of a Decision Support System on Physicians' Diagnostic Performance. J. Am. Med. Inform. Assoc.
6: 420-427
[Abstract][Full Text]
Berner, E. S., Maisiak, R. S.
(1999). Influence of Case and Physician Characteristics on Perceptions of Decision Support Systems. J. Am. Med. Inform. Assoc.
6: 428-434
[Abstract][Full Text]
Lemaire, J. B., Schaefer, J. P., Martin, L. A., Faris, P., Ainslie, M. D., Hull, R. D.
(1999). Effectiveness of the Quick Medical Reference as a diagnostic tool. CMAJ
161: 725-728
[Abstract][Full Text]
Miller, R. A., Gardner, R. M.
(1997). Recommendations for Responsible Monitoring and Regulation of Clinical Software Systems. J. Am. Med. Inform. Assoc.
4: 442-457
[Abstract][Full Text]
Saint, S., Go, A. S., Frances, C., Tierney, L. M.
(1995). Case Records of the Massachusetts General Hospital -- A Home-Court Advantage?. NEJM
333: 883-884
[Full Text]
Berner, E. S., Webster, G. D., Shugerman, A. A., Jackson, J. R., Algina, J., Baker, A. L., Ball, E. V., Cobbs, C. G., Dennis, V. W., Frenkel, E. P., Hudson, L. D., Mancall, E. L., Rackley, C. E., Taunton, O. D.
(1994). Performance of Four Computer-Based Diagnostic Systems. NEJM
330: 1792-1796
[Abstract][Full Text]
Sonnenberg, F. A., Hagerty, C. G., Kulikowski, C. A.
(1994). An Architecture for Knowledge-based Construction of Decision Models. Med Decis Making
14: 27-39
[Abstract]
Christensen, C., Larson, J. R. JR
(1993). Collaborative Medical Decision Making. Med Decis Making
13: 339-346
[Abstract]
Liebhart, J., Krusinska, E.
(1993). Classic uersus Sequential Diagnostic Support for Chronic Nonspecific Respiratory Diseases. Med Decis Making
13: 103-113
[Abstract]
Meredith, J. W., Selen, W.J.
(1987). A database medical diagnostic support system using standardized medical data: a pilot study. Journal of Information Science
13: 353-360
[Abstract]
Greenes, R. A., Cain, K. C., Begg, C. B.
(1984). Patient-Oriented Performance Measures of Diagnostic Tests: 1. Tools for Prospective Evaluation of Test Order Decisions. Med Decis Making
4: 7-15
Duda, R., Shortliffe, E.
(1983). Expert Systems Research. Science
220: 261-268
[Abstract]
Previous
