|
|
| |||||||||||||||||||||||||||||||||||||||||||||||
Background "America's Best Hospitals," an influential list published annually by U.S. News & World Report, assesses the quality of hospitals. It is not known whether patients admitted to hospitals ranked at the top in cardiology have lower short-term mortality from acute myocardial infarction than those admitted to other hospitals or whether differences in mortality are explained by differential use of recommended therapies.
Methods Using data from the Cooperative Cardiovascular Project on 149,177 elderly Medicare beneficiaries with acute myocardial infarction in 1994 or 1995, we examined the care and outcomes of patients admitted to three types of hospitals: those ranked high in cardiology (top-ranked hospitals); hospitals not in the top rank that had on-site facilities for cardiac catheterization, coronary angioplasty, and bypass surgery (similarly equipped hospitals); and the remaining hospitals (non–similarly equipped hospitals). We compared 30-day mortality; the rates of use of aspirin, beta-blockers, and reperfusion; and the relation of differences in rates of therapy to short-term mortality.
Results Admission to a top-ranked hospital was associated with lower adjusted 30-day mortality (odds ratio, 0.87; 95 percent confidence interval, 0.76 to 1.00; P=0.05 for top-ranked hospitals vs. the others). Among patients without contraindications to therapy, top-ranked hospitals had significantly higher rates of use of aspirin (96.2 percent, as compared with 88.6 percent for similarly equipped hospitals and 83.4 percent for non–similarly equipped hospitals; P<0.01) and beta-blockers (75.0 percent vs. 61.8 percent and 58.7 percent, P<0.01), but lower rates of reperfusion therapy (61.0 percent vs. 70.7 percent and 65.6 percent, P=0.03). The survival advantage associated with admission to top-ranked hospitals was less strong after we adjusted for factors including the use of aspirin and beta-blockers (odds ratio, 0.94; 95 percent confidence interval, 0.82 to 1.08; P=0.38).
Conclusions Admission to a hospital ranked high on the list of "America's Best Hospitals" was associated with lower 30-day mortality among elderly patients with acute myocardial infarction. A substantial portion of the survival advantage may be associated with these hospitals' higher rates of use of aspirin and beta-blocker therapy.
The method used to select the top-ranked hospitals is based on Donabedian's three-element model of structure, process, and outcome for assessing the quality of health care.5,6 National data sources were sought that could be used to evaluate each of these aspects of quality.7 Data on variables representing hospital structure, including factors such as staff-to-bed ratios, teaching or nonteaching status, presence and type of high-technology facilities, and volume of patients, were obtained from the American Hospital Association (AHA). Outcomes were assessed on the basis of in-hospital mortality rates adjusted for case mix that were derived from Medicare discharge claims. Data describing process, defined by Donabedian as "activities in making a diagnosis and recommending or implementing treatment,"5 were not available nationwide. Instead, board-certified physicians were asked to nominate the five "best" hospitals in their specialties by means of questionnaires. The percentages of these physicians who nominated particular hospitals generated "reputation" scores used as a proxy measure of high quality in the process of care. The three elements of quality were weighted equally to generate an institutional index of hospital quality, with hospitals ranked according to their overall quality score. The survey lists approximately 40 hospitals in about 12 specialties each year. Further details of the ranking methods are available elsewhere.7,8
Recent analyses have identified a number of methodologic weaknesses in the selection of top-ranked hospitals relating to all three elements of quality.9,10 The use of hospitals' reputations as a measure of the quality of care was particularly questioned. Because the reputation-based score has a highly skewed distribution, a small group of well-known hospitals were ensured a high ranking. For example, in 1997, the eight hospitals ranked at the top in cardiology had reputation scores ranging from 14.0 percent to 53.1 percent, but seven of the eight hospitals ranked at the bottom of the 1997 list scored below 1.0 percent.8 Since the majority of hospitals would have reputation scores near zero (meaning that they were not nominated by any of the surveyed physicians), hospitals with high name recognition would dominate the rankings.
Whether patients admitted to top-ranked hospitals had lower short-term mortality than those admitted to other hospitals, and whether this difference was explained by better performance on clinically derived (rather than reputation-based) measures of the quality of the processes of care, are unknown. To evaluate this issue, we analyzed data from the Cooperative Cardiovascular Project, an initiative of the Health Care Financing Administration (HCFA) designed to examine patterns of care and improve the outcomes of Medicare beneficiaries with acute myocardial infarction. We addressed the following questions: Do patients with acute myocardial infarction who are admitted to hospitals that are ranked at the top in cardiology have lower 30-day mortality than patients admitted to other hospitals? Do top-ranked hospitals perform better than other hospitals on measures of the quality of the processes of care, based on clinical guidelines for the treatment of acute myocardial infarction (specifically the in-hospital use of aspirin, beta-blockers, and reperfusion)? Are differences in 30-day mortality between top-ranked and other hospitals explained by the differences in the rates of use of these therapies?
Methods
Rankings and Characteristics of Hospitals
We identified hospitals listed at least once in "America's Best Hospitals" as being among the top 40 institutions in cardiology in 1995, 1996, and 1997.11,12,13 We considered hospital rankings up to three years after patients' admissions, because annual hospital rankings incorporate data from up to three years before the list is published.8 Because many hospitals entered or left the top-40 list during the three-year period, and because some of the top-ranked hospitals represented several individual hospitals, 60 top-ranked hospitals were included in the final data set. This group comprised all but two of the institutions identified as among "America's Best Hospitals" during this period. Data for these two hospitals were not available during the initial data-collection phase of the Cooperative Cardiovascular Project.
We compared top-ranked hospitals with all other hospitals represented in the Cooperative Cardiovascular Project. To compare the performance of top-ranked hospitals with that of other tertiary care cardiology centers, we defined a group of similarly equipped hospitals, made up of hospitals with on-site facilities for cardiac catheterization, bypass surgery, and coronary angioplasty. The remaining hospitals were classified as non–similarly equipped hospitals. Hospitals' facilities were identified from data in the AHA's 1994 survey of hospitals.14
The AHA survey of hospitals was also used to classify hospitals as teaching hospitals (those offering postgraduate residency training), major teaching hospitals (those with membership in the Council of Teaching Hospitals), or rural hospitals (those located outside a metropolitan statistical area as defined by the Census Bureau).15
The Cooperative Cardiovascular Project
The study sample of the Cooperative Cardiovascular Project was identified from hospital bills in the Medicare National Claims History File of claims submitted under fee-for-service plans.16 Patients discharged from acute care hospitals with a principal discharge-diagnosis code for acute myocardial infarction (code 410 in the International Classification of Diseases, 9th Revision, Clinical Modification [ICD-9-CM]17) were selected for the initial cohort, except those with codes indicating follow-up care of a patient with acute myocardial infarction. Sampling for the Cooperative Cardiovascular Project was conducted during an eight-month period between February 1994 and July 1995, except for the states in the pilot study (Alabama, Connecticut, Iowa, and Wisconsin), in which sampling took place during a four-month period from August through November 1995. We estimated the yearly volume of Medicare admissions for acute myocardial infarction on the basis of the number of cases and months of sampling for each hospital. The specialty of the admitting physician was determined by linking unique physician identification numbers to a data base of physicians' reported specialties obtained from HCFA, as in a previous study.18
Predefined variables were abstracted from copies of hospital records. The reliability of the data was monitored by means of monthly reabstractions of randomly selected records; the accuracy of abstraction with respect to treatment variables was 95 percent.19 Dates of death were obtained from the Medicare Enrollment Database and the Social Security Administration's Master Beneficiary Record File. We excluded 357 cases in which the date of death was uncertain.20
The initial Cooperative Cardiovascular Project sample consisted of 234,754 patients admitted to the hospital. The study sample was then limited to patients 65 years of age or older who had a confirmed acute myocardial infarction, indicated by a discharge diagnosis of acute myocardial infarction and documentation in the patient's chart of either a serum creatine kinase MB fraction above 5 percent, a serum lactate dehydrogenase level more than 1.5 times the upper limit of the normal value and a lactate dehydrogenase isoenzyme 1 level greater than that of lactate dehydrogenase isoenzyme 2, or two of the following three criteria: chest pain, a serum creatine kinase level twice that hospital's normal value, and evidence of a new acute myocardial infarction as documented on the official report on the electrocardiogram. We included only the first admission for each patient during the study period. We excluded patients who were transferred from another acute care hospital, because we were unable to ascertain their clinical characteristics at the time of the initial admission, and patients considered to have a terminal illness (life expectancy of six months or less as documented in the chart) or metastatic cancer, since their treatment may not have been intended to maximize survival. The majority of exclusions resulted from transfers of patients (42,277 patients), lack of confirmation of acute myocardial infarction (31,179 patients), second or subsequent admissions (25,185 patients) and age of less than 65 years (17,591 patients). Some patients met more than one of the exclusion criteria.
Indicators of Quality
HCFA developed a set of measures of performance based on clinical guidelines to use in evaluating the processes of care related to beneficial outcomes for Medicare beneficiaries with acute myocardial infarction.16 Details of the construction of these indicators of quality have been described elsewhere,21 along with the results of validation studies.22 For each indicator of quality, "ideal" candidates for therapy — that is, patients without contraindications documented on their charts — were identified. The Cooperative Cardiovascular Project uses two indicators of quality for the treatment of acute myocardial infarction: use of aspirin during the hospitalization, and reperfusion therapy (thrombolysis or primary angioplasty).21 Ideal candidates for therapy with beta-blockers were patients who had none of the relative contraindications cited in the clinical guidelines of the American College of Cardiology and the American Heart Association.23
Statistical Analysis
We compared base-line characteristics, mortality, and performance as measured by the indicators of quality among top-ranked, similarly equipped, and non–similarly equipped hospitals with use of the chi-square test for categorical variables and analysis of variance for continuous variables. We used multivariate logistic regression to model differences in short-term mortality between top-ranked hospitals and other hospitals. We examined mortality at 30 days because the effects of differences in therapy would be likely to be evident within this period.
We also measured the performance of individual hospitals in order to evaluate whether institutions varied within categories of hospitals. The analysis of mortality was restricted to hospitals with at least 30 patients during the sampling period and the analysis of indicators of quality to those with at least 30 candidates for therapy so as to reduce random variation due to small samples.
To adjust for differences in the severity of illness, we used a modified version of the Medicare Mortality Predictor System (MMPS), a disease-specific model for predicting 30-day mortality (expressed as a percentage) among elderly patients with acute myocardial infarction.24 Variables used in the MMPS to predict the risk of death among patients with acute myocardial infarction include the APACHE II (Acute Physiology and Chronic Health Evaluation) score,25 age, ability to walk, presence or absence of a do-not-resuscitate order on admission, blood urea nitrogen level, mean arterial pressure, presence or absence of a subendocardial infarction, presence or absence of evidence of congestive heart failure on the roentgenogram, and heart rate. Our version of the MMPS did not include values for the serum potassium level in the APACHE II score (these values were not abstracted from the medical records for the Cooperative Cardiovascular Project) and the presence or absence of metastatic cancer (since its presence was a criterion for exclusion). The average mortality predicted at admission by the MMPS was 14.9 percent among patients who survived to 30 days and 34.3 percent among those who died within the first 30 days; the rates were similar to those previously reported.24
We used the modified version of the MMPS to estimate an expected mortality rate by calculating the mean predicted probability of death at 30 days within each group of hospitals. We calculated the risk-adjusted mortality by dividing the hospital's actual mortality rate by its expected mortality and multiplying this ratio by the overall 30-day mortality in the Cooperative Cardiovascular Project (18.5 percent). The resulting risk-adjusted death rate estimated 30-day mortality for a particular group of hospitals if its patients had the same characteristics as the national sample.
The relation between 30-day mortality and the category of hospital was evaluated with use of sequential logistic-regression models. We first examined the association between mortality and status with respect to ranking alone. Subsequent models added variables used in the MMPS to adjust for differences in base-line clinical characteristics and the rates of use of aspirin, beta-blockers, and reperfusion so as to clarify the extent to which differences in mortality between groups of hospitals were associated with variations in the quality of the processes of care. We also examined models that adjusted for characteristics of hospitals and physicians, both separately and at the same time as variables indicating quality. The analyses were repeated with only top-ranked hospitals and similarly equipped hospitals included. Hubert–White methods were used to provide robust estimates of variance, and we controlled for clustering of patients within individual hospitals.26
All statistical calculations were performed using Stata 5.0 software (StataCorp, College Station, Tex.).26
Results
Characteristics of Patients and Hospitals
The study sample consisted of 149,177 patients treated in 4672 hospitals. Top-ranked hospitals were likely to be large, nonprofit, major teaching centers in urban areas that admit large numbers of patients with acute myocardial infarction (Table 1). All but one top-ranked hospital had on-site facilities for cardiac catheterization, bypass surgery, and coronary angioplasty, and that hospital had access to all three facilities at a nearby affiliated hospital.
|
|
Mortality at 30 days was significantly lower among patients admitted to top-ranked hospitals (14.7 percent) than among those admitted to similarly equipped hospitals (17.6 percent) or non–similarly equipped hospitals (19.0 percent) (Table 3). Mortality remained significantly lower, on average, in top-ranked hospitals after adjustment for the severity of illness. The mean risk-adjusted 30-day mortality was 15.6 percent for top-ranked hospitals, 18.3 percent for similarly equipped hospitals, and 18.6 percent for non–similarly equipped hospitals.
|
Indicators of the Quality of Treatment
Top-ranked hospitals had about the same proportion of patients who were candidates for beta-blocker therapy as other hospitals and slightly lower proportions of ideal candidates for aspirin therapy and reperfusion (Table 3). A significantly higher percentage of ideal candidates received aspirin and beta-blocker therapy in top-ranked hospitals than in the other hospitals. Overall, fewer ideal candidates for reperfusion who were admitted to top-ranked hospitals underwent reperfusion (61.0 percent) than was the case with similarly equipped hospitals (70.7 percent) and non–similarly equipped hospitals (65.6 percent). This difference was primarily due to the lower rate of use of thrombolytic agents.
The rates of use of aspirin and beta-blockers varied considerably among individual hospitals within the three categories. Among hospitals with at least 30 ideal candidates for the therapy, the rate of aspirin use ranged from 87.2 percent to 100 percent (median, 96.9 percent) among top-ranked hospitals, from 54.5 percent to 100 percent (median, 89.2 percent) among similarly equipped hospitals, and from 37.2 percent to 100 percent (median, 85.5 percent) among non–similarly equipped hospitals. The percentage of ideal candidates receiving beta-blockers ranged from 51.2 percent to 93.6 percent (median, 76.2 percent) among top-ranked hospitals, from 22.0 percent to 97.3 percent (median, 63.3 percent) among similarly equipped hospitals, and from 15.6 percent to 97.2 percent (median, 65.5 percent) among non–similarly equipped hospitals. Many similarly equipped hospitals and non–similarly equipped hospitals performed as well as or better than top-ranked hospitals with respect to the use of aspirin and beta-blockers. Since few hospitals had 30 or more ideal candidates for thrombolytic therapy or primary angioplasty, reperfusion therapy was not evaluated.
Multivariate Analysis
In an unadjusted model, admission to a top-ranked hospital was associated with lower 30-day mortality (odds ratio, 0.76; 95 percent confidence interval, 0.69 to 0.84; P<0.001) (Table 4). This survival benefit persisted after we adjusted for patients' demographic characteristics and severity of illness. After adding indicators of the quality of treatment to the model, the survival advantage associated with admission to a top-ranked hospital became weaker and not statistically significant (odds ratio, 0.92; 95 percent confidence interval, 0.82 to 1.04; P=0.20), suggesting that a portion of the lower mortality associated with top-ranked hospitals was due to their greater use of aspirin and beta-blockers. Adjusting for the use of therapies appeared to explain more of the relation between admission to a top-ranked hospital and mortality than did the characteristics of the hospital and the physician. A model that included characteristics of the hospital and of the physician in addition to quality indicators gave results that were nearly identical to those without hospital and physician characteristics. Adjusting individually for indicators of quality, models that added aspirin or beta-blockers separately contributed more toward explaining the difference between top-ranked hospitals and other hospitals than did the model that added reperfusion therapy alone. Models that adjusted for aspirin and beta-blocker use together gave results that were identical to that with adjustment for all three indicators of quality.
|
Discussion
In this study of the "best" hospitals in cardiology, we found that patients admitted to top-ranked hospitals were more likely to survive for 30 days after acute myocardial infarction and to receive aspirin and beta-blockers in the absence of contraindications. Admission to a top-ranked hospital was associated with lower 30-day mortality after adjustment for differences in patients' and hospitals' characteristics, but not after adjustment for the rates of use of aspirin and beta-blockers. These findings suggest that a substantial portion of the difference in mortality among patients with acute myocardial infarction that has been observed between top-ranked hospitals and others was associated with the higher rates of use of aspirin and beta-blockers at the top-ranked hospitals.
One potential explanation for these findings is that therapy with aspirin and beta-blockers contributes directly to the survival of patients treated in top-ranked hospitals, since the efficacy of these therapies in reducing short-term mortality has been clearly demonstrated in randomized clinical trials.27,28 If this is true, it is conceivable that other hospitals could achieve outcomes similar to those of top-ranked hospitals by increasing their rates of use of aspirin and beta-blockers in patients with acute myocardial infarction. The survival advantage associated with admission to top-ranked hospitals appears to be more strongly related to the rates of use of aspirin and beta-blockers rather than to rates of thrombolytic therapy or primary angioplasty. This may be so because relatively few patients were ideal candidates for reperfusion therapy.
It is also possible that the lower mortality at top-ranked hospitals resulted from additional factors that ensure the delivery of high-quality care and that were associated with aspirin and beta-blocker therapy, such as the development of clinical algorithms or improved training of medical staff. If these factors play an important part in reducing mortality, our findings suggest that they are more closely associated with the use of aspirin and beta-blockers than with the particular characteristics of hospitals and physicians considered in this analysis.
There was considerable variation in 30-day mortality and in the use of aspirin and beta-blockers even among the top-ranked hospitals. Several similarly equipped and non–similarly equipped hospitals performed as well as or better than some of the top-ranked institutions. Of ideal candidates for therapy who were admitted to top-ranked hospitals, 3.8 percent did not receive aspirin, 25.0 percent did not receive beta-blockers, and 39.0 percent did not receive reperfusion therapy, indicating areas for improvement even among the "best" hospitals.
The lower 30-day mortality at top-ranked hospitals is not entirely unexpected, since in-hospital mortality was one factor used to determine the hospitals' rankings. There are several reasons, however, why top-ranked hospitals were not necessarily guaranteed to have the lowest mortality among patients with acute myocardial infarction. The data used to determine the top-ranked hospitals in cardiology included deaths from all cardiac diagnoses, not only acute myocardial infarction. In addition, the mortality used in the survey was adjusted for risk on the basis of administrative data rather than clinical measures of the severity of acute myocardial infarction.8 Finally, outcome was only one of three components used to determine the overall score for each hospital.
Our study has several limitations. First, our ability to identify contraindications to the therapies used as indicators of quality and to adjust for differences in clinical severity was limited by the information documented in the medical charts. Differences in patients' severity of illness between hospital categories may have accounted for the better outcomes observed in top-ranked hospitals. However, because the MMPS has been shown to be a stronger predictive model than APACHE II — which itself is a powerful predictor of short-term mortality among patients in intensive care units24 — it is unlikely that the lower mortality in top-ranked hospitals can be explained entirely by differences in the clinical severity of patients' conditions.
Second, it is possible that the weakening of the survival advantage associated with admission to top-ranked hospitals after aspirin and beta-blocker use was added to the regression model may have been due to the fact that patients receiving these therapies had less severe illnesses. However, by including variables to indicate whether patients were candidates for aspirin, beta-blocker, and reperfusion therapy in the risk-stratification model, we limited this effect to differences in the severity of illness within the categories of candidates for therapy and noncandidates.
Finally, we focused on mortality as a measure of outcome. We did not assess other important outcomes in the various groups of hospitals, such as functional status or patients' satisfaction.
Although our study shows "America's Best Hospitals" in a favorable light, it does not necessarily confirm the ranking method used to generate this list. Our finding of an association between top-ranked hospitals and greater use of guideline-based therapies for acute myocardial infarction does not imply that the numerical rankings assigned to hospitals represent meaningful differences in quality. Our results may simply show that prominent hospitals perform better than others with respect to selected indicators of quality. Furthermore, our study pertains to one diagnosis within a single specialty, and our results may not be applicable to other cardiac diseases or to other fields of medicine ranked in the list. As a result, this study should not be interpreted as endorsing the method used to generate the list of "America's Best Hospitals" as the best way to rank hospitals according to the quality of care.
Supported by a contract (500-96-P549) with the Health Care Financing Administration. This article does not necessarily reflect the views or policies of the Department of Health and Human Services. The authors assume full responsibility for the accuracy and completeness of the ideas presented.
Mr. Chen is a Merck–American Federation for Aging Research Student Scholar in Geriatric Pharmacology and an American Heart Association Student Scholar in Cardiovascular Disease and Stroke. Dr. Krumholz is a Paul Beeson Faculty Scholar.
We are indebted to all the health care professionals, hospitals, and organizations who contributed to the development and implementation of the Cooperative Cardiovascular Project; to Drs. Gail Vitagliano and Jaime Murillo for their helpful comments; and to Ms. Maria Johnson for her outstanding editorial assistance.
Source Information
From the Section of Cardiovascular Medicine, Department of Medicine (J.C., H.M.K.), and the Section of Chronic Disease Epidemiology, Department of Epidemiology and Public Health (H.M.K.), Yale University School of Medicine, New Haven, Conn.; the Yale–New Haven Hospital Center for Outcomes Research and Evaluation, New Haven, Conn. (H.M.K., M.J.R.); Qualidigm, Middletown, Conn. (H.M.K., M.J.R., Y.W.); and the Health Care Financing Administration, Baltimore (T.A.M.).
Address reprint requests to Dr. Krumholz at Yale University School of Medicine, 333 Cedar St., P.O. Box 208025, New Haven, CT 06520-8025.
References
| |||||||||||||||||||||||||||||||||||||||||||||||
Related Letters:
Comparing Hospitals
Hartz A., Graber M., Doebbeling B., Lane W. L., Otten J., Hollander R. M., Chen J., Krumholz H. M., Radford M. J., Kassirer J. P.
Extract |
Full Text
N Engl J Med 1999;
340:2006-2007, Jun 24, 1999.
Correspondence
This article has been cited by other articles:
HOME | SUBSCRIBE | SEARCH | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | PRIVACY | TERMS OF USE | HELP | beta.nejm.org Comments and questions? Please contact us. The New England Journal of Medicine is owned, published, and copyrighted © 2009 Massachusetts Medical Society. All rights reserved. |
Previous
