The New England Journal of Medicine
e-mail icon  FREE NEJM E-TOC    HOME   |   SUBSCRIBE   |   CURRENT ISSUE   |   PAST ISSUES   |   COLLECTIONS   |    Advanced Search
Sign in | Get NEJM's E-Mail Table of Contents — Free | Subscribe
 
Special Article
Patient Safety
PreviousPrevious
Volume 348:2526-2534 June 19, 2003 Number 25
NextNext

Improving Safety with Information Technology
David W. Bates, M.D., and Atul A. Gawande, M.D., M.P.H.

 

This Article
- PDF
-PDA Full Text

Commentary
-Editorial
by Berwick, D. M.

Tools and Services
-Add to Personal Archive
-Add to Citation Manager
-Notify a Friend
-E-mail When Cited
-E-mail When Letters Appear

More Information
-PubMed Citation
Health care is growing increasingly complex, and most clinical research focuses on new approaches to diagnosis and treatment. In contrast, relatively little effort has been targeted at the perfection of operational systems, which are partly responsible for the well-documented problems with medical safety.1 If medicine is to achieve major gains in quality, it must be transformed, and information technology will play a key part,2 especially with respect to safety.

In other industries, information technology has made possible what has been called "mass customization" — the efficient and reliable production of goods and services according to the highly personalized needs of individual customers.2 Computer retailers, for example, now use their Web sites to allow people to purchase computers built to their exact specifications, which can be shipped within two days. Medical care is, of course, orders of magnitude more complex than selling personal computers, and clinicians have always strived to provide carefully individualized care. However, safe care now requires a degree of individualization that is becoming unimaginable without computerized decision support. For example, computer systems can instantaneously identify interactions among a patient's medications. Even today, more than 600 drugs require adjustment of doses for multiple levels of renal dysfunction, a task that is poorly performed by human prescribers without assistance but can be done accurately by computers.3 Multiple studies now demonstrate that computer-based decision support can improve physicians' performance and, in some instances, patient outcomes.3,4,5,6

In the past decade, the risk of harm caused by medical care has received increasing scrutiny.1 The growing sophistication of computers and software should allow information technology to play a vital part in reducing that risk — by streamlining care, catching and correcting errors, assisting with decisions, and providing feedback on performance. Given the large potential risks and benefits as well as the costs involved, in this article we analyze what is known about the role and effect of information technology with respect to safety and consider the implications for medical care, research, and policy.

Ways That Information Technology Can Reduce Errors

Information technology can reduce the rate of errors in three ways: by preventing errors and adverse events, by facilitating a more rapid response after an adverse event has occurred, and by tracking and providing feedback about adverse events. Data now show that information technology can reduce the frequency of errors of different types and probably the frequency of associated adverse events.7,8,9,10,11,12,13,14,15,16,17,18 The main classes of strategies for preventing errors and adverse events include tools that can improve communication, make knowledge more readily accessible, require key pieces of information (such as the dose of a drug), assist with calculations, perform checks in real time, assist with monitoring, and provide decision support.

Improving Communication

Failures of communication, particularly those that result from inadequate "handoffs" between clinicians, remain among the most common factors contributing to the occurrence of adverse events.19,20,21 In one study, cross-coverage of medical inpatients was associated with an increase by a factor of 5.2 in the risk of an adverse event.22 A new generation of technology — including computerized coverage systems for signing out, hand-held personal digital assistants (Figure 1), and wireless access to electronic medical records — may improve the exchange of information, especially if links between various applications and a common clinical data base are in place, since many errors result from inadequate access to clinical data. In the study mentioned above, the implementation of a "coverage list" application, which standardized the information exchanged among clinicians, eliminated the excess risk resulting from cross-coverage.16


View larger version (39K):
[in this window]
[in a new window]
  		Figure 1. Notification about a Critical Laboratory Result.

This is an example of the combination of a hand-held device and a cellular telephone (Sprint) to allow rapid communication about an important abnormality (in this case, a potassium level of 2.5 mg per deciliter) and to offer the clinician the option to take one of several actions immediately.

 
Also, many serious laboratory abnormalities — for example, hypokalemia and a decreasing hematocrit — require urgent action but occur relatively infrequently, often when a clinician is not at hand, and such results can be buried amid less critical data. Information systems can identify and rapidly communicate these problems to clinicians automatically (Figure 1), unlike traditional systems in which such results are communicated to a clerk for the unit.12,13,14,15 In one controlled trial, this approach reduced the time to the administration of appropriate treatment by 11 percent and reduced the duration of dangerous conditions in patients by 29 percent.23

Providing Access to Information

Another key to improving safety will be improving access to reference information. A wide range of textbooks, references on drugs, and tools for managing infectious disease, as well as access to the Medline data base, are already available for desktop and even hand-held computers (e.g., through http://www.epocrates.com and http://www.unboundmedicine.com). Ease and rapidity of use at the point of care were initially problematic but appear to be improving, and hand-held devices are now widely used, especially for drug-reference information.24

Requiring Information and Assisting with Calculations

One of the main benefits of using computers for clinical tasks that is often overlooked is that it makes it possible to implement "forcing functions" — features that restrict the way in which tasks may be performed. For example, prescriptions written on a computer can be forced to be legible and complete. Similarly, applications can require constraints on clinicians' choices regarding the dose or route of administration of a potentially dangerous medication. Thus, a dose that is 10 times as large as it should be will be ordered much less frequently if it is not one of the options on a menu (Figure 2). Indeed, forcing functions have been found to be one of the primary ways in which computerized order entry by physicians reduces the rate of errors.26 The usefulness of forcing functions may also apply to other types of information technology. For example, bar-coded patient-identification bracelets designed to prevent accidents, such as the performance in one patient of a procedure intended for another patient, function in this way.27 Similarly, many actions imply that another should be taken; these dependent actions have been termed "corollary orders" by Overhage et al.28 For example, prescribing bed rest for a patient would trigger the suggestion that the physician consider initiating prophylaxis against deep venous thrombosis. This approach — which essentially targets errors of omission — has resulted in a change in behavior in 46 percent of cases in the intervention group, as compared with 22 percent of cases in the control group, with regard to a broad range of actions.28


View larger version (6K):
[in this window]
[in a new window]
  		Figure 2. Percentage of Medication Orders with Doses Exceeding the Maximum.

Data are the percentage of orders for doses exceeding the medication-specific recommended maximal dose according to year, after the implementation of a computerized system for order entry by physicians.25 The application suggested a default dose and displayed only potentially appropriate options, but it did not check for overly high doses. Even so, the percentage of orders exceeding the recommended safe maximum fell by more than 80 percent over a three-year period.

 
The use of computers can also reduce the frequency of errors of calculation, a common human failing.29 Such tools can be used on demand — for example, by a nurse in the calculation of an infusion rate.

Monitoring

Monitoring is inherently boring and is not performed well by humans. Moreover, so many data are collected now that it can be hard to sift through them to detect problems. However, if the monitoring of information is computerized, applications can perform this task, looking for relations and trends and highlighting them, which can permit clinicians to intervene before an adverse outcome occurs. For example, "smart" monitors can look for and highlight signals that suggest the occurrence of decompensation in a patient — signals that a human observer would often fail to detect (Figure 3).30


View larger version (27K):
[in this window]
[in a new window]
  		Figure 3. "Smart" Monitoring in an Intensive Care Unit.

This screen highlights physiological changes that are occurring (in this case, a rapid pulse and a trend toward increasing pulse and decreasing blood pressure [BP]); such monitoring can help clinicians to detect and respond to such changes before an adverse event occurs. The heart-rate (HR) limit alert is triggered when the heart rate crosses a high (H) or low (L) limit, which are determined according to the patient's active medical conditions. Patient 5 (thick arrow) has had surgery and is at risk for perioperative coronary events. The limit value is given in brackets, followed by the patient's current value. The heart-rate or blood-pressure trend alert is triggered if the heart rate or blood pressure changes substantially over a period of several hours. Patient 4 (thin arrows) has an increasing heart rate and a decreasing blood pressure; on evaluation, this patient was found to have hypovolemia. The base-line value is given in brackets, followed by the current value. Screen courtesy of Michael Breslow, M.D., Visicu, Baltimore.

 
A related approach that appears to be beneficial on the basis of early data is technology-enabled remote monitoring of intensive care. In one study, remote monitoring in a 10-bed intensive care unit was associated with a reduction in mortality of 68 percent and 46 percent as compared with two different base-line periods, and the average length of stay in the intensive care unit and related costs each decreased by about a third.17 Such monitoring is especially attractive in the intensive care unit because there is a national shortage of intensivists.

Decision Support

Information systems can assist in the flow of care in many important ways by making available such key information on patients as laboratory values, by calculating weight-based doses of medications, or by red-flagging patients for whom an order for imaging with intravenous contrast material may be inappropriate. A longer-term benefit will occur as more sophisticated tools — such as computerized algorithms and neural networks — become integrated with the provision of health care. Neural-network decision aids allow many factors to be considered simultaneously in order to predict a specific outcome. These tools have been developed in order to reduce diagnostic and treatment errors in numerous clinical settings, including the assessment of abdominal pain, chest pain, and psychiatric emergencies and the interpretation of radiologic images and tissue specimens.31 Controlled trials have demonstrated improvement in clinical accuracy with the use of such technical tools, including their use in the diagnosis of myocardial infarction,32,33 the detection of breast cancer on screening mammograms,34 and the finding of cervical neoplasia on Papanicolaou smears.35 However, of these practices, only neural-network–assisted cervical screening has had substantial use, and little of that use has been in the United States.31,36 Nonetheless, more widespread use of electronic medical records could lead to an expanded role for these applications and make it easier to integrate them into routine care.

Rapid Response to and Tracking of Adverse Events

Computerized tools can also be used with electronic medical records to identify, intervene early in, and track the frequency of adverse events — a major gap in the current safety-related armamentarium — since, to improve processes, it is important to be able to measure outcomes.37 Classen et al. pioneered an approach for combing clinical data bases to detect signals that suggest the presence of an adverse drug event in hospitalized patients, such as the use of an antidote; this approach identified 81 times as many events as did spontaneous reporting, which is the standard technique used today.38 Others have built applications that allow the detection of nosocomial infections in inpatients39 and adverse drug events in outpatients.40

Such tools may be useful both for the improvement of care and for research. Together with Indiana University, we are conducting a controlled trial to evaluate computerized prescribing for outpatients. In the first year of this study, we built a computerized monitor for adverse drug events, which goes through the electronic medical record to detect signals (such as high serum drug levels) that suggest that an adverse drug event may have occurred (Table 1). This approach inexpensively identifies large numbers of adverse drug events that are not routinely detected. We are now using the rates of events to assess the effect of computerized prescribing, first with simple and then with more advanced decision support.

View this table:
[in this window]
[in a new window]
  		Table 1. Results of Screening for Drug-Related Adverse Events with the Use of Electronic Medical Records for Outpatients.

 
Electronic tools designed to identify a broad array of adverse events in a variety of settings seem promising.41 Often, these signals may permit earlier intervention; for example, Raschke et al. found that 44 percent of the alerts generated by a tool that they built had not been identified by the team of clinicians.5

Medication Safety and the Prevention of Errors

After anesthesia, medication safety has perhaps been the most closely studied domain in patient safety. Efforts to reduce the rate of medication errors have involved all the strategies discussed above. Nearly half of serious medication errors have been found to result from the fact that clinicians have insufficient information about the patient and the drug. Other common factors include a failure to provide sufficient specificity in an order, illegibility of handwritten orders, errors of calculation, and errors in transcription.7 In one controlled trial involving inpatients, the implementation of a computerized application for order entry by physicians — which improves communication, makes knowledge accessible, includes appropriate constraints on choices of drugs, routes, frequencies, and doses, helps with calculations, performs real-time checks, and assists with monitoring — resulted in a 55 percent reduction in serious medication-related errors.8 In a further study, which evaluated serial improvements to this application with the addition of higher levels of support for clinical decisions (e.g., more comprehensive checking for drug allergies and drug–drug interactions), there was an 83 percent reduction in the overall rate of medication errors.9 The use of decision support for clinical decisions can also result in major reductions in the rate of complications associated with antibiotics, and can decrease costs and the rate of nosocomial infections.10 Other technological tools with substantial potential but less solid evidence of effectiveness include the bar coding of medications and the use of automated drug-delivery devices for both oral and intravenous medications.11

Summary of Approaches to Prevention

To date, studies have generally been conducted only in individual facilities and rarely in the outpatient setting; moreover, only a few types of technology have been well tested. However, the large benefits found in the improvement of fundamental aspects of patient care8,12,13,16,17,18 indicate that information technology can be an important tool for improving safety in many clinical settings.

Tools that can improve communication, make knowledge more accessible, require key information, and assist with calculations and clinical decision making are available today and should provide substantial benefit. More research is needed on such questions as how best to perform checks, how best to assist in monitoring, and especially, how to provide decision support most effectively in complex situations. In today's systems, many important warnings are ignored,42 and there are too many unimportant warnings. Approaches have been developed to highlight more serious warnings — for instance, by displaying a skull and crossbones — when a clinician tries to order a drug that has previously caused an anaphylactic reaction in the patient (Figure 4). However, many efforts directed at complex targets such as the management of hypertension44 or congestive heart failure45 have failed. Overcoming these difficulties will require bringing cognitive engineers and techniques for assessing and accommodating human factors, such as usability testing, into the design of medical processes.


View larger version (36K):
[in this window]
[in a new window]
  		Figure 4. Warning Displayed for a Drug Allergy.

When warnings are displayed in current systems, even important messages are often overridden, most likely because too many unimportant warnings are displayed. Principles of design that take into account human factors suggest that it is important to make warnings that are more serious look different from those that are less serious,43 as in this case, in which the screen displays a skull and crossbones to warn that the patient has previously had anaphylaxis. Whether or not such a design would result in increased attention to important warnings has not been tested.

 
Barriers and Directions for Improvement

Despite the substantial opportunities for improvement in patient safety, the development, testing, and adoption of information technology remain limited. Numerous barriers exist, although some approaches to overcoming them are at hand.

Financial Barriers

The development of medical applications of information technology has largely been commercially funded, and reimbursement has rewarded excellent billing rather than outstanding clinical care. As a result, the focus has been more on products to improve the "back-office" functions related to clinical practice than on those that might improve clinical practice itself. Since they depend on new capital, research and development efforts for clinical tools have had relatively limited funding. When companies have produced useful technological tools, their spending on clinical testing has been negligible, particularly in comparison with what is spent on the testing of medical devices or drugs.46 Furthermore, even for proven applications, such as computerized order entry for physicians, vendors do not have ready-made products.47 For clinicians and institutions seeking to adopt technological tools, the investment costs can be high,48 and the quality of the decision support that comes along with these applications remains highly variable.49

Progress on this front is unlikely to occur without considerable investment — particularly public investment — in clinical information technology. Incentives could make an important difference. To increase capital investment, legislation has been introduced in the U.S. Senate to provide nearly $1 billion over a period of 10 years to hospitals and Medicare-supported nursing homes that implement technology that improves medication safety.50 Of concern, however, are measures that mandate the adoption of such technology without providing the funding for doing so. California, for example, has passed a law requiring, as a condition of licensure, that all nonrural hospitals implement technology such as, but not limited to, computerized order entry for physicians by January 1, 2005.51 Neither an increase in reimbursement nor capital grants were provided to help hospitals to meet this requirement. A piece of national legislation in this area — the Patient Safety Improvement Act of 2003 (H.R. 877) — was passed by the House of Representatives on March 12, 2003. This bill would provide $50 million in grants over a two-year period to institutions that implement information technology intended to improve patient safety. Forms of technology that are named include electronic communication of patient data, computerized order entry by physicians, bar coding, and data support technology. Although this is a positive development, these incentives are sufficiently limited that their effect would most likely be small.52

Lack of Standards

We lack a single standard in the United States today for representation of most types of key clinical data, including conditions, procedures, medications, and laboratory data.53 The result has been that most applications do not communicate well, even within organizations, and the costs of interfaces are high. Another highly charged issue is that standards for some important types of data are privately held. Privately held standards are standards that are in general use but are licensed by a company or organization. Examples of privately held standards are diagnosis codes that are licensed by the College of American Pathologists and procedure codes that are licensed by the American Medical Association.

However, there are both short-term and longer-term opportunities in this area. The National Committee on Vital and Health Statistics recently released a report54 endorsing national standards for electronic data for key domains. The adoption of the Consolidated Health Informatics standards by the federal government on March 21, 2003, represents a major step forward.55 This initial set includes standards for messaging, images, and clinical laboratory tests. Such standardization will encourage innovation and the adoption of applications with relatively little cost to the government. Although standards are not fully developed for every important type of information, the identification of this area as a major priority should make it possible to do the additional work required, especially if federal funding to support it is provided. An important, open question is whether any organization should be able to hold a national standard privately. We believe that one appropriate approach would be to require organizations to sell such classification systems for a fair price.

Cultural Barriers

There is also a tendency for clinicians and policymakers to see information technology as relatively unimportant for either research efforts or incorporation into medical practice. Academic centers are more apt to seek and reward faculty members who pursue research on a drug or a device that might lead to a reduction of 0.5 percent in the rate of death from myocardial infarction than those who develop a decision-support system that could result in a far greater reduction. Furthermore, clinicians have been reluctant to adopt information technology even when it has been shown to be effective.

This reluctance appears to have a number of causes. It is still a new concept in medicine that computerized tools can have powerful benefits in practice. When errors occur, physicians are no less likely than the public to see the clinicians involved, rather than the system, as the central problem.2 In addition, many physicians are still uncomfortable with computers. Some are concerned about depending on them, particularly for clinical decision making. With regard to certain technological tools, such as e-mail between physicians and patients and electronic medical records, clinicians are also concerned about legal issues, including privacy.

Not only the government but clinicians too, in their practices and relationships with colleagues and health care facilities, must recognize that most preventable adverse events result from failures of systems, not individual failures. Investment in and adoption of new forms of information technology must be understood as being as vital to good patient care as the adoption of new technological tools for diagnosis and treatment.

Current Situation

Overall, few of the types of information technology that may improve safety are widely implemented. For example, few hospitals have adopted computerized order entry for physicians. However, the Leapfrog Group — a coalition of some of the nation's largest employers, such as General Electric and General Motors — has identified it as one of three changes that they believe would most improve safety,56 and many hospitals are beginning on this path. Use of computer-assisted decision making in diagnosis and the planning of treatment remains rare. Furthermore, the quality of the clinical software applications that are currently being developed remains unclear. Especially given the absence of widely used standards, organizations have been reluctant to make large financial commitments, fearing that they will select a dead-end solution. Another pivotal issue is that information technology has been seen by many health care organizations as a commodity, like plumbing, rather than as a strategic resource that is vitally important to the delivery of care. Exceptions are institutions such as the health systems of the Department of Veterans Affairs and Kaiser, and reported data suggest these strategies have been successful.57,58,59

Conclusions

The fundamental difficulty in modern medical care is execution. Providing reliable, efficient, individualized care requires a degree of mastery of data and coordination that will be achievable only with the increased use of information technology. Information technology can substantially improve the safety of medical care by structuring actions, catching errors, and bringing evidence-based, patient-centered decision support to the point of care to allow necessary customization. New approaches that improve customization and gather and sift through reams of data to identify key changes in status and then notify key persons should prove to be especially important.

Supported in part by a grant (PO1 HS11534) from the Agency for Healthcare Research and Quality (to Dr. Bates).

Dr. Bates reports having served as a paid lecturer for Eclipsys and as a consultant for MedManagement and Alaris.

We are indebted to Amar Desai for comments on previous versions of this manuscript and to Anne Kittler for assistance with the preparation of the manuscript.


Source Information

From the Division of General Medicine and Primary Care, Department of Medicine (D.W.B.), and the Department of Surgery (A.A.G.), Brigham and Women's Hospital; the Center for Applied Medical Information Systems, Partners HealthCare System (D.W.B.); and Harvard Medical School (D.W.B., A.A.G.) — all in Boston.

Address reprint requests to Dr. Bates at the Division of General Medicine and Primary Care, Brigham and Women's Hospital, 75 Francis St., Boston, MA 02115, or at dbates{at}partners.org.

References

  1. Kohn LT, Corrigan JM, Donaldson MS, eds. To err is human: building a safer health system. Washington, D.C.: National Academy Press, 2000. 
  2. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, D.C.: National Academy Press, 2001.
  3. Chertow GM, Lee J, Kuperman GJ, et al. Guided medication dosing for inpatients with renal insufficiency. JAMA 2001;286:2839-2844. [Free Full Text]
  4. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998;280:1339-1346. [Free Full Text]
  5. Raschke RA, Gollihare B, Wunderlich TA, et al. A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital. JAMA 1998;280:1317-1320. [Erratum, JAMA 1999;281:420.] [Free Full Text]
  6. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med 1994;120:135-142. [Free Full Text]
  7. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. JAMA 1995;274:35-43. [Abstract]
  8. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280:1311-1316. [Free Full Text]
  9. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. Arch Intern Med 1999;159:2553-2560. [Free Full Text]
  10. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med 1998;338:232-238. [Free Full Text]
  11. Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ 2000;320:788-791. [Free Full Text]
  12. Tate KE, Gardner RM, Scherting K. Nurses, pagers, and patient-specific criteria: three keys to improved critical value reporting. Proc Annu Symp Comput Appl Med Care 1995;19:164-168. 
  13. Rind DM, Safran C, Phillips RS, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med 1994;154:1511-1517. [Abstract]
  14. Tate KE, Gardner RM, Weaver LK. A computerized laboratory alerting system. MD Comput 1990;7:296-301. [Medline]
  15. Shabot MM, LoBue M, Chen J. Wireless clinical alerts for physiologic, laboratory and medication data. In: Proceedings of the American Medical Informatics Association 2000 annual symposium. Philadelphia: Hanley & Belfus, 2000:789-93.
  16. Petersen LA, Orav EJ, Teich JM, O'Neil AC, Brennan TA. Using a computerized sign-out program to improve continuity of in-patient care and prevent adverse events. Jt Comm J Qual Improv 1998;24:77-87. [Medline]
  17. Rosenfeld BA, Dorman T, Breslow MJ, et al. Intensive care unit telemedicine: alternate paradigm for providing continuous intensivist care. Crit Care Med 2000;28:3925-3931. [CrossRef][ISI][Medline]
  18. Kuperman G, Teich J, Bates DW. Improving care with computerized alerts and reminders. Assoc Health Serv Res 1997;14:224-5. abstract.
  19. Wanlass RL, Reutter SL, Kline AE. Communication among rehabilitation staff: "mild," "moderate," or "severe" deficits? Arch Phys Med Rehabil 1992;73:477-481. [ISI][Medline]
  20. Greenlaw J. Legally speaking: the deadly toll of communication failure. RN (For Managers) 1982;45:81-84. [Medline]
  21. Schmidt IK, Svarstad BL. Nurse-physician communication and quality of drug use in Swedish nursing homes. Soc Sci Med 2002;54:1767-1777. [Medline]
  22. Petersen LA, Brennan TA, O'Neil AC, Cook EF, Lee TH. Does housestaff discontinuity of care increase the risk for preventable adverse events? Ann Intern Med 1994;121:866-872. [Free Full Text]
  23. Kuperman GJ, Teich JM, Tanasijevic MJ, et al. Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc 1999;6:512-522. [Free Full Text]
  24. Rothschild JM, Lee TH, Bae T, Bates DW. Clinician use of a palmtop drug reference guide. J Am Med Inform Assoc 2002;9:223-229. [Free Full Text]
  25. Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr C, Bates DW. Effects of computerized physician order entry on prescribing practices. Arch Intern Med 2000;160:2741-2747. [Free Full Text]
  26. Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc 1994;1:108-123. [Free Full Text]
  27. Chassin MR, Becher EC. The wrong patient. Ann Intern Med 2002;136:826-833. [Free Full Text]
  28. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc 1997;4:364-375. [Free Full Text]
  29. Bates DW, Cullen D, Laird N, Petersen LA, Small S, Servi D, et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA 1995;274:29-34. [Abstract]
  30. Celi LA, Hassan E, Marquardt C, Breslow M, Rosenfeld B. The eICU: it's not just telemedicine. Crit Care Med 2001;29:Suppl:N183-N189. [CrossRef][ISI][Medline]
  31. Baxt WG. Application of artificial neural networks to clinical medicine. Lancet 1995;346:1135-1138. [CrossRef][ISI][Medline]
  32. Baxt WG, Skora J. Prospective validation of artificial neural network trained to identify acute myocardial infarction. Lancet 1996;347:12-15. [CrossRef][ISI][Medline]
  33. Heden B, Ohlin H, Rittner R, Edenbrandt L. Acute myocardial infarction detected in the 12-lead ECG by artificial neural networks. Circulation 1997;96:1798-1802. [Free Full Text]
  34. Petrick N, Sahiner B, Chan HP, Helvie MA, Paquerault S, Hadjiiski LM. Breast cancer detection: evaluation of a mass-detection algorithm for computer-aided diagnosis -- experience in 263 patients. Radiology 2002;224:217-224. [Free Full Text]
  35. Kok MR, Boon ME. Consequences of neural network technology for cervical screening: increase in diagnostic consistency and positive scores. Cancer 1996;78:112-117. [CrossRef][Medline]
  36. Frable WJ. Cervical screening by neural networks. Cancer 2001;93:171-172. [Medline]
  37. Bates DW, Cohen M, Leape LL, Overhage JM, Shabot MM, Sheridan T. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc 2001;8:299-308. [Free Full Text]
  38. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA 1991;266:2847-2851. [Erratum, JAMA 1992;267:1922.] [Abstract]
  39. Evans RS, Larsen RA, Burke JP, et al. Computer surveillance of hospital-acquired infections and antibiotic use. JAMA 1986;256:1007-1011. [Abstract]
  40. Honigman B, Lee J, Rothschild J, et al. Using computerized data to identify adverse drug events in outpatients. J Am Med Inform Assoc 2001;8:254-266. [Free Full Text]
  41. Bates DW, Evans RS, Murff HJ, Stetson PD, Pizziferri L, Hripcsak G. Detecting adverse events using information technology. J Am Med Inform Assoc 2003;10:115-128. [Free Full Text]
  42. Rosebraugh CJ, Honig PK, Yasuda SU, Pezzullo JC, Flockhart DA, Woosley RL. Formal education about medication errors in internal medicine clerkships. JAMA 2001;286:1019-1020. [Free Full Text]
  43. Edworthy J, Adams AS. Warning design: a research prospective. London: Taylor & Francis, 1996.
  44. Montgomery AA, Fahey T, Peters TJ, MacIntosh C, Sharp DJ. Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ 2000;320:686-690. [Free Full Text]
  45. Tierney WM, Overhage JM, Takesue BY, et al. Computerizing guidelines to improve care and patient outcomes: the example of heart failure. J Am Med Inform Assoc 1995;2:316-322. [Free Full Text]
  46. Gawande AA, Bates DW. The use of information technology in improving medical performance. II. Physician-support tools. MedGenMedicine 2000:E13.
  47. The quality initiative: computerized physician order entry. Oakland: California Healthcare Foundation, September 2000. (Accessed May 27, 2003, at http://www.chcf.org/documents/quality/CPOEfactsheet.pdf.)
  48. Doolan DF, Bates DW. Computerized physician order entry systems in hospitals: mandates and incentives. Health Aff (Millwood) 2002;21:180-188. [Free Full Text]
  49. Metzger J, Turisco F. Computerized physician order entry: a look at the vendor marketplace and getting started. Washington, D.C.: Leapfrog Group, December 2001. (Accessed May 27, 2003, at http://www.leapfroggroup.org/CPOE/CPOE%20Guide.pdf.)
  50. Medication Errors Reduction Act of 2001, S. 824, 107th Cong., 1st Sess. (2001).
  51. S.B. 1875 Senate bill — chaptered. (Accessed May 27, 2003, at http://info.sen.ca.gov/pub/99-00/bill/sen/sb_1851-1900/sb_1875_bill_20000928_chaptered.html.)
  52. H.R. 877. Patient Safety Improvement Act of 2003. (Accessed May 27, 2003, at http://thomas.loc.gov.)
  53. Information for health: a strategy for building the National Health Information Infrastructure: report and recommendations from the National Committee on Vital and Health Statistics. Washington, D.C.: National Committee on Vital and Health Statistics, November 15, 2002. (Accessed May 27, 2003, at http://www.ncvhs.hhs.gov/nhiilayo.pdf.)
  54. Letter to the Secretary: recommendations for the first set of PMRI standards. Washington, D.C.: National Committee on Vital and Health Statistics, February 27, 2002. (Accessed May 27, 2003, at http://www.ncvhs.hhs.gov/reptrecs.htm.)
  55. Federal government announces first federal e-gov health information exchange standards. (Accessed May 27, 2003, at http://www.whitehouse.gov/omb/egov/press/chi_march.htm.)
  56. Shapiro JP. Industry preaches safety in Pittsburgh. U.S. News & World Report. July 17, 2000:56.
  57. Weeks WB, Bagian JP. Developing a culture of safety in the Veterans Health Administration. Eff Clin Pract 2000;6:270-276.
  58. Weitzel R. The introduction of bar coding technology at Kaiser Permanente. Healthc Inform 1991;8:24, 26-24, 26. 
  59. Johnston B, Wheeler L, Deuser J, Sousa KH. Outcomes of the Kaiser Permanente Tele-Home Health Research Project. Arch Fam Med 2000;9:40-45. [Free Full Text]

 

This Article
- PDF
-PDA Full Text

Commentary
-Editorial
by Berwick, D. M.

Tools and Services
-Add to Personal Archive
-Add to Citation Manager
-Notify a Friend
-E-mail When Cited
-E-mail When Letters Appear

More Information
-PubMed Citation

This article has been cited by other articles:


HOME  |  SUBSCRIBE  |  SEARCH  |  CURRENT ISSUE  |  PAST ISSUES  |  COLLECTIONS  |  PRIVACY  |  HELP  |  beta.nejm.org

Comments and questions? Please contact us.

The New England Journal of Medicine is owned, published, and copyrighted © 2008 Massachusetts Medical Society. All rights reserved.