FREE NEJM E-TOC    HOME   |   SUBSCRIBE   |   CURRENT ISSUE   |   PAST ISSUES   |   COLLECTIONS   |   Search Term  Advanced Search

Sign in | Get NEJM's E-Mail Table of Contents — Free | Subscribe

Previous Volume 345:965-970 September 27, 2001 Number 13 Next

Abstract PDF Editorial by James, B. C. Add to Personal Archive Add to Citation Manager Notify a Friend E-mail When Cited PubMed Citation

ABSTRACT

Background Although they are effective in outpatient settings, computerized reminders have not been proved to increase preventive care in inpatient settings.

Methods We conducted a randomized, controlled trial to determine the effects of computerized reminders on the rates at which four preventive therapies were ordered for inpatients. During an 18-month study period, a computerized system processed on-line information for all 6371 patients admitted to a general-medicine service (for a total of 10,065 hospitalizations), generating preventive care reminders as appropriate. Physicians who were in the intervention group viewed these reminders when they were using a computerized order-entry system for inpatients.

Results The reminder system identified 3416 patients (53.6 percent) as eligible for preventive measures that had not been ordered by the admitting physician. For patients with at least one indication, computerized reminders resulted in higher adjusted ordering rates for pneumococcal vaccination (35.8 percent of the patients in the intervention group vs. 0.8 percent of those in the control group, P<0.001), influenza vaccination (51.4 percent vs. 1.0 percent, P<0.001), prophylactic heparin (32.2 percent vs. 18.9 percent, P<0.001), and prophylactic aspirin at discharge (36.4 percent vs. 27.6 percent, P<0.001).

Conclusions A majority of hospitalized patients in this study were eligible for preventive measures, and computerized reminders significantly increased the rate of delivery of such therapies.

Although multiple randomized trials have confirmed that computerized reminders increase the use of preventive care in the outpatient setting,^{13,14,15,16,17} a trial among hospitalized patients failed to demonstrate an increase in the use of preventive care measures with this intervention.^14,18 Nevertheless, hospitalization represents an opportunity to target persons who are particularly likely to benefit from preventive care. For example, 1 future hospitalization might be avoided by the simple administration of pneumococcal vaccine to 100 appropriate hospitalized patients.¹⁹ We hypothesized that a computerized reminder system could increase the use of preventive care among hospitalized patients.

Methods

Setting and Eligibility

We obtained approval for this study from the institutional review board of the Indiana University Medical Center. The board did not require informed consent from patients, since computer reminders have been considered routine components of care in the outpatient setting. We included all patients admitted to the general-medicine service of Wishard Memorial Hospital, an urban public teaching hospital in Indianapolis, between May 1, 1997, and October 31, 1998. The organization of the teams of the general-medicine ward has been described previously¹⁸; at present there are eight independent teams whose staff members (physicians and students) rotate approximately monthly.

Randomization and Outcomes

Using a blinded system of coin randomization, one of the investigators randomly designated four of the general-medicine teams as intervention teams and four as control teams. All physicians, medical students, and patients associated with a team were assigned that team's intervention status. The same investigator also randomly assigned physicians to teams insofar as practical constraints allowed (e.g., avoiding assignments that might lead to two consecutive nights of overnight on-call duty). When physicians returned for multiple rotations during the study period, we attempted to maintain their original intervention status by assigning them to teams with the same status; most medical students had only one rotation at the hospital during the study period. Patients were admitted to the general-medicine wards with the use of a system that distributed admissions equally among the teams, solely on the basis of the order in which patients required hospitalization. Patients automatically assumed the intervention status of the team to which they were assigned on admission and retained that status for the duration of their hospitalization. Previous studies involving inpatients have shown no material differences in clinical status among patients assigned to different teams.²⁰

The primary outcomes of interest were the rates at which the various preventive therapies were ordered. These rates were obtained from routinely stored data.

Computerized Order Entry and Clinical-Decision Support

Using personal-computer–based order-entry workstations,^18,20,21,22 resident physicians and medical students on the general-medicine teams wrote all orders (except for do-not-resuscitate orders); these persons were the targets of the intervention. During the order-entry process, the system provided clinical-decision support to physicians and medical students by means of rule-based reminders, which we called Care rules.^18,23 The Care rules generated the reminders in this study as prewritten orders with explanatory text. Physicians could accept or reject the reminders with one or two keystrokes on the computer.

Generation and Display of Computer Reminders

Using the Care rules, we implemented national guidelines that were current at the time of the study regarding the use of pneumococcal vaccination,³ influenza vaccination,²⁴ prophylactic enteric-coated aspirin for cardiovascular disease,^25,26,27 and prophylactic subcutaneous heparin to reduce the risk of thromboembolic events in patients with certain medical conditions.²⁸ Computer-generated reminders for all but one of these therapies (subcutaneous heparin) have been a routine part of care in the outpatient setting for more than 15 years. The computer was programmed to suggest influenza vaccination only during the "flu-shot season" from October 1, 1997, through January 31, 1998. Before the study, we tested the appropriateness of the reminders generated by the Care rules (available from the National Auxiliary Publications Service [^*]) by having the practitioners review the rules directly to judge whether they were reasonable and by running the rule-based reminder program on a sample of more than 300 patient records and manually validating the recommendations by reference to the rules and the content of the computerized medical charts.

The Care rules relied on multiple sources of such routinely collected data as demographic characteristics of the patient, lists of medical problems, diagnoses at previous hospital discharge, vital signs, active inpatient orders, previous pharmacy records, and coded radiologic results. In addition, we obtained information from patients on their vaccination status by means of a standard admission questionnaire completed by a research assistant or the admitting nurse. The Care rules generated reminders when the patient's electronic medical record included at least one indication for one of the selected preventive therapies, no evident contraindication to the therapy, no active orders for the therapy, and in the case of the two vaccinations, no record of previous administration within an appropriate time frame.

The Care rules program was used for all study patients (in both the intervention group and the control group) when the physician initiated an order-entry session for daily care, for the transfer of the patient, or for discharge. The program was not run and no reminders were generated when orders were entered at the time of admission; thus, the clinician had one opportunity to order the preventive therapies without receiving a reminder. The computer program logged the following types of data: the names of all the Care rules that were applied, all the resultant messages and orders that were generated, the dates and times of all messages and orders, the physician's identification number and intervention status, and the patient's hospital identification number and intervention status.

In most cases, the physician and the patient had the same intervention status; when physicians were on call or during emergencies, they might write orders for a patient with a different intervention status. When the physician and the patient were both assigned to the intervention group, the computer displayed to the physician all reminders generated by the program. When either the physician or the patient was assigned to the control group, the computer logged the reminders but did not display them. Patients whose data triggered a reminder were considered to be eligible for the specified therapy.

A sample computerized reminder message, as it would appear to a physician assigned to the intervention group, is shown in Figure 1. Physicians indicated their acceptance or rejection of each recommended therapy by choosing "order" or "omit." For the three therapies with few risks (influenza vaccination, pneumococcal vaccination, and subcutaneous heparin), we set the default to "order," so that the physician could accept the item simply by pressing the "enter" key. In the case of daily aspirin prophylaxis against coronary artery disease, we set the default to "omit," thereby requiring a more deliberate effort by the physician to change the status to "order." As a means of capturing the physician's attention briefly, we disabled the "escape" key and presented the reminders in a color scheme different from that used for physician-initiated orders. If a physician ordered the targeted therapy, no more reminders related to that therapy appeared. No reminders were displayed between the fifth hospital day and the time discharge orders were entered, at which time vaccination and prophylactic daily aspirin were suggested again if they were indicated.

View larger version (50K): [in this window] [in a new window]   Figure 1. Example of a Computerized Reminder as Displayed to Physicians on Intervention Teams.

 
Statistical Analysis

We compared the demographic characteristics of the patients in the intervention and control groups by means of the chi-square test and Student's t-test. The demographic data, including race, used for these comparisons were obtained as part of the routine procedures for hospital registration. The unit of analysis for all models was the individual hospital admission. We used a generalized-estimating-equation method²⁹ for all estimates of effect. We used a compound-symmetry structure, which assumes that patients are independent of each other and that hospitalizations for a particular patient have a fixed correlation that does not vary over time. Our primary analyses included all hospital admissions, but we also performed analyses that dealt with the effects of crossover by excluding patients and physicians who were part of both the intervention and the control group at different times. We limited these analyses to patients' first hospitalizations during the study period and excluded physicians who were in both the intervention group and the control group during the course of the study.

We modeled the effect of the explanatory variables (intervention status and demographic characteristics) on the binary response variable (therapy ordered or not ordered) with the use of logistic regression, and we considered a P value of 0.05 or less to indicate statistical significance. To account for possible similarities among the ordering rates of physicians working on the same team, we included in the models a variable indicating medical team nested within the intervention status. We created two models for each of the four preventive therapies — one including only the hospitalizations during which a reminder was generated by the computer, and the other including all hospitalizations.

Results

Study Subjects

A total of 6371 patients accounted for 10,065 admissions during the study period; patients were assigned to intervention teams for 4995 hospitalizations and to control teams for 5070 hospitalizations. Twenty-eight percent of the patients were hospitalized more than once. The mean age of the hospitalized patients was 53.2 years, 50 percent were women, and 51 percent were black. There were no significant differences in mean age, race, or sex between the intervention and control groups.

During the 18 months of the study, a total of 202 physicians rotated an average of 2.1 times onto the inpatient service: 96 of these physicians (47.5 percent) were assigned only to intervention teams, 78 (38.6 percent) were assigned only to control teams, and 28 (13.9 percent) were assigned at different times to intervention teams and control teams.

Computer Reminders and Ordering Rates for Preventive Therapies

Overall, 3416 patients (53.6 percent) were eligible for at least one preventive therapy. The mean number of times that a reminder was displayed to a physician on an intervention team ranged from 3.4 to 5.2 for each hospitalization of an eligible patient in the intervention group.

The display of computer-generated reminders had a significant effect on the use of each of the four preventive therapies (Table 1). The use of the reminders led to a higher ordering rate for pneumococcal vaccination (35.8 percent among eligible patients in the intervention group, as compared with 0.8 percent among eligible patients in the control group) and a higher ordering rate for influenza vaccination (51.4 percent as compared with 1.0 percent) (Table 1). When the total number of hospitalizations was used as the denominator for calculating the ordering rates for each of the preventive therapies, the rates were also significantly higher among patients in the intervention group than among those in the control group (Table 2). There were no important differences in our results when we limited our analyses to the patients' first hospitalizations and excluded the physicians who took part in both intervention and control teams during the study period.

View this table: [in this window] [in a new window]   Table 1. Adjusted Ordering Rates for Preventive Therapies for Eligible Patients.

 

View this table: [in this window] [in a new window]   Table 2. Adjusted Ordering Rates for Preventive Therapies for All 6371 Hospitalized Patients during 10,065 Hospitalizations.

 
Physicians varied in the frequency with which they accepted these reminders. The distribution of ordering rates among 114 physicians randomly assigned to the intervention group who received computerized reminders for at least 10 patients (range, 10 to 82) is shown in Figure 2.

View larger version (5K): [in this window] [in a new window]   Figure 2. Distribution of Ordering Rates for Preventive Therapies among the Physicians in the Intervention Group. Data are limited to the physicians who had at least 10 patients for whom reminders were displayed.

 
The rates of physicians' orders for each of the four preventive therapies were positively correlated with the age of the patients (P<0.002 for all comparisons). Physicians also ordered prophylactic aspirin at the time of discharge more frequently for patients who were hospitalized for acute myocardial infarction and unstable angina than for those with other indications (cerebrovascular disease, peripheral vascular disease, and coronary risk factors) (P<0.001 for all comparisons).

Discussion

The results of this trial provide compelling evidence that computerized reminders can increase the delivery of preventive care to hospitalized patients. For all four preventive therapies, the use of computerized reminders resulted in absolute ordering rates for both eligible patients and all hospitalized patients that were significantly higher than the rates in the control group. In particular, the use of reminders increased the use of pneumococcal and influenza vaccination from practically zero to approximately 35 percent and 50 percent, respectively. (Analyses of data from the 15 months after the completion of the controlled trial revealed that, with continuing reminders, these rates had increased to 50 percent for pneumococcal vaccination and 57 percent for influenza vaccination.) The easy sustainability of computer-based reminder systems contrasts with the weaknesses of such approaches as manual reviewing of charts,^30,31 patient-directed interventions,^32,33 and physician-directed continuing medical education.³⁰

Previous studies of computer-generated reminders for preventive care have focused almost exclusively on the outpatient setting.¹⁴ Yet we found that 54 percent of patients hospitalized in general-medicine wards were eligible for preventive care interventions and that physicians' rates of compliance with the reminders were similar to those achieved in outpatient settings.¹⁵ These findings alone would justify focusing greater attention on preventive care for inpatients, but there are other reasons to do so as well. For one, patients hospitalized because of severe underlying diseases may benefit more from preventive measures (e.g., pneumococcal vaccination^19,34) than patients who do not require hospitalization. Also, among patients who do not have regular outpatient follow-up, hospitalization may offer the only good case-finding opportunity to provide such care.³⁵ Other preventive therapies, such as subcutaneous heparin, are specifically indicated in the hospital setting.

The findings of this study contrast with the results of a previous study performed at our institution that did not demonstrate an effect of computerized reminders.¹⁸ We attribute our recent success to relatively small changes in the presentation of these reminders. In our previous study, we relied on physicians to make a deliberate choice to view the reminders and notified them only by means of a banner at the bottom of the screen stating that "there are suggested orders for this patient." By contrast, in the current study, the computer immediately displayed the reminders to the physicians as full, prewritten orders. In this study, we also highlighted the suggested reminders with a distinctive color scheme, disabled the "escape" key, repeated the individual reminders approximately four times per hospitalization, on average, and set the default to "order" for three of the preventive therapies (allowing the physician to accept the item simply by pressing the "enter" key).

We do not have data on the reasons for noncompliance by physicians in the intervention group. In some cases, physicians may have had good reasons for ignoring a reminder; they may have known something that the computer did not — for example, the patient may have told the physician that aspirin had caused bleeding in the past. Furthermore, the compliance rates of physicians in some cases were related to the strength of the indication. For example, physicians complied with reminders for aspirin prophylaxis in 79 percent of the cases in which a patient was admitted for an acute myocardial infarction but only 22 percent of the cases in which indications were an age of more than 50 years and two or more coronary risk factors.

As in previous studies,³⁶ there was a great deal of individual variation among physicians in their acceptance of the reminders. This variation suggests that some portion of the overall rate of noncompliance was attributable to very low compliance among some physicians who generally ignored the reminders. The extremely low rate of use of vaccinations in the control group also suggests that some of the noncompliance with vaccination reminders could be attributed to long-established habits of vaccinating patients only in the outpatient setting.

Although the reminders in this study were based on a physician order-entry system and a rich repository of clinical data, neither of these features is necessarily a prerequisite to improving hospital-based preventive care. In the outpatient setting, reminders written on paper encounter forms are effective in increasing the use of preventive care.¹⁶ The daily patient census reports produced for physicians in many hospitals could be as effective a mechanism for delivering reminders in the hospital as an order-entry system. Furthermore, the majority of the reminders generated in our study were triggered by information that is routinely available in hospital information systems. For example, an age of at least 65 years is a valid indication for pneumoccal and influenza vaccination. The addition of the diagnosis on admission, past diagnoses, and the service to which the patient was admitted would have been sufficient to identify the majority of patients with indications for vaccinations and many of those with indications for aspirin or subcutaneous heparin. It is likely that with a small amount of computer programming and the creation of nursing or other protocols to check for possible contraindications and previous vaccinations, many hospitals could implement programs that provide simple reminders and thereby improve preventive care for inpatients.

Supported by a grant (HS07719) from the Agency for Healthcare Research and Quality and a grant (N01-LM-6-3546) from the National Library of Medicine.

We are indebted to the physicians, nurses, and administrators at Wishard Memorial Hospital for their support of the physician order-entry process and this study.

^* See NAPS document no. 05605 for 8 pages of supplementary material. To order, contact NAPS, c/o Microfiche Publications, 248 Hempstead Tpke., West Hempstead, NY 11552.

Source Information

From the Department of Medicine, Indiana University School of Medicine (P.R.D., S.P., J.M.O., K.M., R.B.K., C.J.M.); the Regenstrief Institute for Health Care (P.R.D., S.P., J.M.O., C.J.M.); and the Richard L. Roudebush Veterans Affairs Medical Center (P.R.D., R.B.K.) — all in Indianapolis.

Address reprint requests to Dr. McDonald at the Regenstrief Institute for Health Care, 1050 Wishard Blvd., Indianapolis, IN 46202, or at clem{at}regen.rg.iupui.edu.

References

1. Sisk JE, Moskowitz AJ, Whang W, et al. Cost-effectiveness of vaccination against pneumococcal bacteremia among elderly people. JAMA 1997;278:1333-1339. [Abstract]
2. Fedson DS. The clinical effectiveness of pneumococcal vaccination: a brief review. Vaccine 1999;17:Suppl 1:S85-S90.
3. Prevention of pneumococcal disease: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morb Mortal Wkly Rep 1997;46:1-24. [Medline]
4. Influenza and pneumococcal vaccination levels among adults aged 65 years -- United States, 1997. MMWR Morb Mortal Wkly Rep 1998;47:797-802. [Medline]
5. Gross PA, Hermogenes AW, Sacks HS, Lau J, Levandowski RA. The efficacy of influenza vaccine in elderly persons: a meta-analysis and review of the literature. Ann Intern Med 1995;123:518-527. [Free Full Text]
6. Nichol KL, Baken L, Nelson A. Relation between influenza vaccination and outpatient visits, hospitalization, and mortality in elderly persons with chronic lung disease. Ann Intern Med 1999;130:397-403. [Free Full Text]
7. McDonald CJ, Hui SL, Tierney WM. Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput 1992;9:304-312. [Medline]
8. Collaborative overview of randomised trials of antiplatelet therapy. I. Prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients: Antiplatelet Trialists' Collaboration. BMJ 1994;308:81-106. [Free Full Text]
9. O'Connor GT, Quinton HB, Traven ND, et al. Geographic variation in the treatment of acute myocardial infarction: the Cooperative Cardiovascular Project. JAMA 1999;281:627-633. [Free Full Text]
10. Geerts WH, Heit JA, Clagett GP, et al. Prevention of venous thromboembolism. Chest 2001;119:Suppl:132S-175S. [Free Full Text]
11. Anderson FA Jr, Wheeler HB, Goldberg RJ, Hosmer DW, Forcier A, Patwardhan NA. Physician practices in the prevention of venous thromboembolism. Ann Intern Med 1991;115:591-595.
12. Keane MG, Ingenito EP, Goldhaber SZ. Utilization of venous thromboembolism prophylaxis in the medical intensive care unit. Chest 1994;106:13-14. [Free Full Text]
13. Fedson DS. Adult immunization: summary of the National Vaccine Advisory Committee Report. JAMA 1994;272:1133-1137. [Abstract]
14. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998;280:1339-1346. [Free Full Text]
15. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc 1996;3:399-409. [Free Full Text]
16. McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record: a two-year randomized trial. Ann Intern Med 1984;100:130-138.
17. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med 1976;295:1351-1355. [Abstract]
18. Overhage JM, Tierney WM, McDonald CJ. Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med 1996;156:1551-1556. [Abstract]
19. Fedson DS, Harward MP, Reid RA, Kaiser DL. Hospital-based pneumococcal immunization: epidemiologic rationale from the Shenandoah study. JAMA 1990;264:1117-1122. [Abstract]
20. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations: effects on resource utilization. JAMA 1993;269:379-383. [Abstract]
21. McDonald CJ, Tierney WM. The Medical Gopher -- a microcomputer system to help find, organize and decide about patient data. West J Med 1986;145:823-829. [ISI][Medline]
22. McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Inf 1999;54:225-253. [CrossRef][ISI][Medline]
23. Overhage JM, Mamlin B, Warvel J, Warvel J, Tierney W, McDonald CJ. A tool for provider interaction during patient care: G-CARE. Proc Annu Symp Comput Appl Med Care 1995:178-82.
24. Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morb Mortal Wkly Rep 1997;46:1-25.
25. Cairns JA, Lewis HD Jr, Meade TW, Sutton GC, Theroux P. Antithrombotic agents in coronary artery disease. Chest 1995;108:Suppl:380S-400S. [Medline]
26. Sherman DG, Dyken ML Jr, Gent M, Harrison JG, Hart RG, Mohr JP. Antithrombotic therapy for cerebrovascular disorders: an update. Chest 1995;108:Suppl:444S-456S. [Medline]
27. Clagett GP, Krupski WC. Antithrombotic therapy in peripheral arterial occlusive disease. Chest 1995;108:Suppl:431S-443S. [Medline]
28. Clagett GP, Anderson FA Jr, Heit J, Levine MN, Wheeler HB. Prevention of venous thromboembolism. Chest 1995;108:Suppl:312S-334S. [Medline]
29. Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics 1986;42:121-130. [CrossRef][ISI][Medline]
30. LeBaron CW, Chaney M, Baughman AL, et al. Impact of measurement and feedback on vaccination coverage in public clinics, 1988-1994. JAMA 1997;277:631-635. [Abstract]
31. Morrow RW, Gooding AD, Clark C. Improving physicians' preventive health care behavior through peer review and financial incentives. Arch Fam Med 1995;4:165-169. [Abstract]
32. Jacobson TA, Thomas DM, Morton FJ, Offutt G, Shevlin J, Ray S. Use of a low-literacy patient education tool to enhance pneumococcal vaccination rates: a randomized controlled trial. JAMA 1999;282:646-650. [Free Full Text]
33. Smith DM, Zhou XH, Weinberger M, Smith F, McDonald RC. Mailed reminders for area-wide influenza immunization: a randomized controlled trial. J Am Geriatr Soc 1999;47:1-5. [ISI][Medline]
34. Lipsky BA, Boyko EJ, Inui TS, Koepsell TD. Risk factors for acquiring pneumococcal infections. Arch Intern Med 1986;146:2179-2185. [Abstract]
35. Spitzer WO, Mann KV. The public's health is too important to be left to public health workers: a commentary on Guide to Clinical Preventive Services. Ann Intern Med 1989;111:939-942.
36. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc 1997;4:364-375. [Free Full Text]

Abstract PDF Editorial by James, B. C. Add to Personal Archive Add to Citation Manager Notify a Friend E-mail When Cited PubMed Citation

This article has been cited by other articles:

• Kilbridge, P. M., Classen, D. C. (2008). The Informatics Opportunities at the Intersection of Patient Safety and Clinical Informatics. J. Am. Med. Inform. Assoc. 15: 397-407 [Abstract] [Full Text]  
• Coady, M. H., Galea, S., Blaney, S., Ompad, D. C., Sisco, S., Vlahov, D., for the Project Viva Intervention Working Group, (2008). Project VIVA: A Multilevel Community-Based Intervention to Increase Influenza Vaccination Rates Among Hard-to-Reach Populations in New York City. Am. J. Public Health 98: 1314-1321 [Abstract] [Full Text]  
• Dexheimer, J. W., Talbot, T. R., Sanders, D. L., Rosenbloom, S. T., Aronsky, D. (2008). Prompting Clinicians about Preventive Care Measures: A Systematic Review of Randomized Controlled Trials. J. Am. Med. Inform. Assoc. 15: 311-320 [Abstract] [Full Text]  
• Kooij, F. O., Klok, T., Hollmann, M. W., Kal, J. E. (2008). Decision Support Increases Guideline Adherence for Prescribing Postoperative Nausea and Vomiting Prophylaxis. Anesth. Analg. 106: 893-898 [Abstract] [Full Text]  
• Ferraris, V. A., Edwards, F. H., Shahian, D. M., Ferraris, S. P. (2008). Risk Stratification and Comorbidity. Card Surg Adult 3: 199-246 [Full Text]  
• Craig, T. J., Perlin, J. B., Fleming, B. B. (2007). Self-Reported Performance Improvement Strategies of Highly Successful Veterans Health Administration Facilities. American Journal of Medical Quality 22: 438-444 [Abstract]  
• Jha, A. K., Wright, S. M., Perlin, J. B. (2007). Performance Measures, Vaccinations, and Pneumonia Rates Among High-Risk Patients in Veterans Administration Health Care. Am. J. Public Health 97: 2167-2172 [Abstract] [Full Text]  
• Lo, H. G., Newmark, L. P., Yoon, C., Volk, L. A., Carlson, V. L., Kittler, A. F., Lippincott, M., Wang, T., Bates, D. W. (2007). Electronic Health Records in Specialty Care: A Time-Motion Study. J. Am. Med. Inform. Assoc. 14: 609-615 [Abstract] [Full Text]  
• Haberman, S., Rotas, M., Perlman, K., Feldman, J. G. (2007). Variations in Compliance With Documentation Using Computerized Obstetric Records. Obstet Gynecol 110: 141-145 [Abstract] [Full Text]  
• Wright, A., Goldberg, H., Hongsermeier, T., Middleton, B. (2007). A Description and Functional Taxonomy of Rule-based Decision Support Content at a Large Integrated Delivery Network. J. Am. Med. Inform. Assoc. 14: 489-496 [Abstract] [Full Text]  
• Ash, J. S., Sittig, D. F., Poon, E. G., Guappone, K., Campbell, E., Dykstra, R. H. (2007). The Extent and Importance of Unintended Consequences Related to Computerized Provider Order Entry. J. Am. Med. Inform. Assoc. 14: 415-423 [Abstract] [Full Text]  
• Gibbons, R. J. (2007). Leading the Elephant Out of the Corner: The Future of Health Care: Presidential Address at the American Heart Association 2006 Scientific Sessions. Circulation 115: 2221-2230 [Full Text]  
• Francis, C. W. (2007). Prophylaxis for Thromboembolism in Hospitalized Medical Patients. NEJM 356: 1438-1444 [Full Text]  
• Chaudhry, R., Scheitel, S. M., McMurtry, E. K., Leutink, D. J., Cabanela, R. L., Naessens, J. M., Rahman, A. S., Davis, L. A., Stroebel, R. J. (2007). Web-Based Proactive System to Improve Breast Cancer Screening: A Randomized Controlled Trial. Arch Intern Med 167: 606-611 [Abstract] [Full Text]  
• Greist, J. H., Jefferson, J. W. (2007). Obsessive-Compulsive Disorder. Focus 5: 283-298 [Abstract] [Full Text]  
• Callen, J. L., Westbrook, J. I., Braithwaite, J. (2006). The Effect of Physicians' Long-term Use of CPOE on Their Test Management Work Practices. J. Am. Med. Inform. Assoc. 13: 643-652 [Abstract] [Full Text]  
• Ompad, D. C., Galea, S., Vlahov, D. (2006). Distribution of Influenza Vaccine to High-Risk Groups. Epidemiol Rev 28: 54-70 [Abstract] [Full Text]  
• Stafford, R. S., Monti, V., Furberg, C. D., Ma, J. (2006). Long-Term and Short-Term Changes in Antihypertensive Prescribing by Office-Based Physicians in the United States. Hypertension 48: 213-218 [Abstract] [Full Text]  
• Sidorov, J. (2006). It Ain't Necessarily So: The Electronic Health Record And The Unlikely Prospect Of Reducing Health Care Costs.. Health Aff (Millwood) 25: 1079-1085 [Abstract] [Full Text]  
• Kaushal, R., Jha, A. K., Franz, C., Glaser, J., Shetty, K. D., Jaggi, T., Middleton, B., Kuperman, G. J., Khorasani, R., Tanasijevic, M., Bates, D. W., Brigham and Women's Hospital CPOE Working Group, (2006). Return on Investment for a Computerized Physician Order Entry System. J. Am. Med. Inform. Assoc. 13: 261-266 [Abstract] [Full Text]  
• Vreeman, D. J (2006). Clinical Prediction Rules. ptjournal 86: 761-762 [Full Text]  
• McDonald, C. J. (2006). Computerization Can Create Safety Hazards: A Bar-Coding Near Miss. ANN INTERN MED 144: 510-516 [Abstract] [Full Text]  
• Ozdas, A., Speroff, T., Waitman, L. R., Ozbolt, J., Butler, J., Miller, R. A. (2006). Integrating "Best of Care" Protocols into Clinicians' Workflow via Care Provider Order Entry: Impact on Quality-of-Care Indicators for Acute Myocardial Infarction. J. Am. Med. Inform. Assoc. 13: 188-196 [Abstract] [Full Text]  
• Vreeman, D. J, Taggard, S. L, Rhine, M. D, Worrell, T. W (2006). Evidence for Electronic Health Record Systems in Physical Therapy. ptjournal 86: 434-446 [Abstract] [Full Text]  
• Vreeman, D. J, Taggard, S. L, Rhine, M. D, Worrell, T. W (2006). Author Response. ptjournal 86: 448-449 [Full Text]  
• Park, R. W., Shin, S. S., Choi, Y. I., Ahn, J. O., Hwang, S. C. (2005). Computerized Physician Order Entry and Electronic Medical Record Systems in Korean Teaching and General Hospitals: Results of a 2004 Survey. J. Am. Med. Inform. Assoc. 12: 642-647 [Abstract] [Full Text]  
• Rosenbloom, S. T., Chiu, K.-W., Byrne, D. W., Talbert, D. A., Neilson, E. G., Miller, R. A. (2005). Interventions to Regulate Ordering of Serum Magnesium Levels: Report of an Unintended Consequence of Decision Support. J. Am. Med. Inform. Assoc. 12: 546-553 [Abstract] [Full Text]  
• Shortell, S. M., Schmittdiel, J., Wang, M. C., Li, R., Gillies, R. R., Casalino, L. P., Bodenheimer, T., Rundall, T. G. (2005). An Empirical Assessment of High-Performing Medical Groups: Results from a National Study. Med Care Res Rev 62: 407-434 [Abstract]  
• Pham, H. H., Schrag, D., Hargraves, J. L., Bach, P. B. (2005). Delivery of Preventive Services to Older Adults by Primary Care Physicians. JAMA 294: 473-481 [Abstract] [Full Text]  
• Rosenbloom, S. T., Geissbuhler, A. J., Dupont, W. D., Giuse, D. A., Talbert, D. A., Tierney, W. M., Plummer, W. D., Stead, W. W., Miller, R. A. (2005). Effect of CPOE User Interface Design on User-Initiated Access to Educational and Patient Information during Clinical Care. J. Am. Med. Inform. Assoc. 12: 458-473 [Abstract] [Full Text]  
• Coffin, C. S., Saunders, C., Thomas, C. M., Loewen, A. H.S., Ghali, W. A., Campbell, N. R.C. (2005). Validity of ICD-9-CM Administrative Data for Determining Eligibility for Pneumococcal Vaccination Triggers. American Journal of Medical Quality 20: 158-163 [Abstract]  
• Galanter, W. L., Didomenico, R. J., Polikaitis, A. (2005). A Trial of Automated Decision Support Alerts for Contraindicated Medications Using Computerized Physician Order Entry. J. Am. Med. Inform. Assoc. 12: 269-274 [Abstract] [Full Text]  
• Weiner, M, Stump, T E, Callahan, C M, Lewis, J N, McDonald, C J (2005). Pursuing integration of performance measures into electronic medical records: beta-adrenergic receptor antagonist medications. Qual Saf Health Care 14: 99-106 [Abstract] [Full Text]  
• Kucher, N., Koo, S., Quiroz, R., Cooper, J. M., Paterno, M. D., Soukonnikov, B., Goldhaber, S. Z. (2005). Electronic Alerts to Prevent Venous Thromboembolism among Hospitalized Patients. NEJM 352: 969-977 [Abstract] [Full Text]  
• Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., Haynes, R. B. (2005). Effects of Computerized Clinical Decision Support Systems on Practitioner Performance and Patient Outcomes: A Systematic Review. JAMA 293: 1223-1238 [Abstract] [Full Text]  
• Goldhaber, S. Z., Turpie, A. G.G. (2005). Prevention of Venous Thromboembolism Among Hospitalized Medical Patients. Circulation 111: e1-e3 [Full Text]  
• Dexter, P. R., Perkins, S. M., Maharry, K. S., Jones, K., McDonald, C. J. (2004). Inpatient Computer-Based Standing Orders vs Physician Reminders to Increase Influenza and Pneumococcal Vaccination Rates: A Randomized Trial. JAMA 292: 2366-2371 [Abstract] [Full Text]  
• Delpierre, C., Cuzin, L., Fillaux, J., Alvarez, M., Massip, P., Lang, T. (2004). A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach?. Int J Qual Health Care 16: 407-416 [Abstract] [Full Text]  
• McMullin, S. T., Lonergan, T. P., Rynearson, C. S., Doerr, T. D., Veregge, P. A., Scanlan, E. S. (2004). Impact of an Evidence-Based Computerized Decision Support System on Primary Care Prescription Costs. Ann Fam Med 2: 494-498 [Abstract] [Full Text]  
• Weingarten, J. P. Jr, Fan, W., Peacher-Ryan, H., Brittman, S., FitzGerald, D., Jowers, S., Przybyszewski, C. (2004). Hospital Quality Improvement Activities and the Effects of Interventions on Pneumonia: A Multistate Study of Medicare Beneficiaries. American Journal of Medical Quality 19: 157-165 [Abstract]  
• Love, R. R., Baumann, L. C., Brown, R. L., Fontana, S. A., Clark, C. C., Sanner, L. A., Davis, J. E. (2004). Cancer Prevention Services and Physician Consensus in Primary Care Group Practices. Cancer Epidemiol. Biomarkers Prev. 13: 958-966 [Abstract] [Full Text]  
• Mosen, D., Elliott, C. G., Egger, M. J., Mundorff, M., Hopkins, J., Patterson, R., Gardner, R. M. (2004). The Effect of a Computerized Reminder System on the Prevention of Postoperative Venous Thromboembolism. Chest 125: 1635-1641 [Abstract] [Full Text]  
• Ash, J. S., Gorman, P. N., Seshadri, V., Hersh, W. R. (2004). Computerized Physician Order Entry in U.S. Hospitals: Results of a 2002 Survey. J. Am. Med. Inform. Assoc. 11: 95-99 [Abstract] [Full Text]  
• Ash, J. S., Berg, M., Coiera, E. (2004). Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors. J. Am. Med. Inform. Assoc. 11: 104-112 [Abstract] [Full Text]  
• McDonald, C. J., Overhage, J. M., Mamlin, B. W., Dexter, P. D., Tierney, W. M. (2004). Physicians, Information Technology, and Health Care Systems: A Journey, Not a Destination. J. Am. Med. Inform. Assoc. 11: 121-124 [Full Text]  
• Atchison, K.A. (2003). Using Information Technology and Community-based Research to Improve the Dental Health-care System. Adv. Dent. Res. 17: 86-88 [Abstract] [Full Text]  
• Geerts, W., Selby, R. (2003). Prevention of Venous Thromboembolism in the ICU. Chest 124: 357S-363S [Abstract] [Full Text]  
• Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., Spurr, C., Khorasani, R., Tanasijevic, M., Middleton, B. (2003). Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality. J. Am. Med. Inform. Assoc. 10: 523-530 [Abstract] [Full Text]  
• Weiner, M., Callahan, C. M., Tierney, W. M., Overhage, J M., Mamlin, B., Dexter, P. R., McDonald, C. J. (2003). Using Information Technology To Improve the Health Care of Older Adults. ANN INTERN MED 139: 430-436 [Abstract] [Full Text]  
• Granai, C. O. (2003). What Matters Matter? P Values, H Values, Leadership, and Us. Obstet Gynecol 102: 393-396 [Abstract] [Full Text]  
• Kuperman, G. J., Gibson, R. F. (2003). Computer Physician Order Entry: Benefits, Costs, and Issues. ANN INTERN MED 139: 31-39 [Abstract] [Full Text]  
• Reinertsen, J. L. (2003). Zen and the Art of Physician Autonomy Maintenance. ANN INTERN MED 138: 992-995 [Abstract] [Full Text]  
• Hsia, D. (2003). Can Administrative Data Assess Physicians' Quality of Care?--Reply. JAMA 289: 2648-2648 [Full Text]  
• Filippi, A., Bignamini, A. A., Sessa, E., Samani, F., Mazzaglia, G. (2003). Secondary Prevention of Stroke in Italy: A Cross-Sectional Survey in Family Practice. Stroke 34: 1010-1014 [Abstract] [Full Text]  
• Maviglia, S. M., Zielstorff, R. D., Paterno, M., Teich, J. M., Bates, D. W., Kuperman, G. J. (2003). Automating Complex Guidelines for Chronic Disease: Lessons Learned. J. Am. Med. Inform. Assoc. 10: 154-165 [Abstract] [Full Text]  
• Sernyak, M. J., Dausey, D., Desai, R., Rosenheck, R. (2003). Prescribers' Nonadherence to Treatment Guidelines for Schizophrenia When Prescribing Neuroleptics. Psychiatr. Serv. 54: 246-248 [Abstract] [Full Text]  
• Doolan, D. F., Bates, D. W., James, B. C. (2003). The Use of Computers for Clinical Care: A Case Series of Advanced U.S. Sites. J. Am. Med. Inform. Assoc. 10: 94-107 [Abstract] [Full Text]  
• Ferraris, V. A., Ferraris, S. P. (2003). Risk Stratification and Comorbidity. Card Surg Adult 2: 187-224 [Full Text]  
• Flansbaum, B. E., Huddleston, J. M. (2002). Update in Hospital Medicine. ANN INTERN MED 137: 814-822 [Abstract] [Full Text]  
• Bratzler, D. W., Houck, P. M., Jiang, H., Nsa, W., Shook, C., Moore, L., Red, L. (2002). Failure to Vaccinate Medicare Inpatients: A Missed Opportunity. Arch Intern Med 162: 2349-2356 [Abstract] [Full Text]  
• Gluckman, R. (2002). Review: organisational change and patient involvement may increase the use of prevention cancer screening services. Evid. Based Med. 7: 192-192 [Full Text]  
• Wyatt, J C, Liu, J L Y (2002). Basic concepts in medical informatics. J. Epidemiol. Community Health 56: 808-812 [Abstract] [Full Text]  
• Gluckman, R. (2002). A comprehensive annotated reminder tool increased appropriate screening in primary care. Evid. Based Med. 7: 127-127 [Full Text]  
• Morris, A H (2002). Decision support and safety of clinical environments. Qual Saf Health Care 11: 69-75 [Abstract] [Full Text]  
• (2002). ADDITIONAL ARTICLES ABSTRACTED IN ACP JOURNAL CLUB. Evid. Based Med. 7: 35-35 [Full Text]  
• Mandl, K. D., Lee, T. H. (2002). Integrating Medical Informatics and Health Services Research: The Need for Dual Training at the Clinical Health Systems and Policy Levels. J. Am. Med. Inform. Assoc. 9: 127-132 [Abstract] [Full Text]  
• Shaneyfelt, T. M. (2001). Building Bridges to Quality. JAMA 286: 2600-2601 [Full Text]  
• James, B. C. (2001). Making It Easy to Do It Right. NEJM 345: 991-993 [Full Text]