|
|
| |||||||||||||||||||||||||||||||||||||||||||||||
Background Evaluations of the appropriateness of medical care are important to monitor the quality of care and to contain costs and enhance safety by reducing inappropriate care. Experts' views are usually incorporated into evaluations of appropriateness. However, practicing physicians may not concur with these views, and physicians' clinical backgrounds may influence their beliefs.
Methods We asked 1058 internists, family practitioners, and cardiologists in California, Florida, New York, Pennsylvania, and Texas to rate the appropriateness of coronary angiography after acute myocardial infarction for 20 common indications. Nine clinical experts also rated these indications using an established consensus method.
Results For 17 of the 20 indications, median ratings of surveyed physicians and the expert panel agreed within 1 unit on a 9-unit scale. Patients' older age had a negative effect on ratings by the expert panel but not on ratings by surveyed physicians. In multivariable analyses of surveyed physicians, cardiologists rated angiography as significantly more appropriate than did primary care physicians for complicated indications, and for uncomplicated indications cardiologists who performed invasive procedures gave higher appropriateness ratings for angiography than did cardiologists who did not perform such procedures and primary care physicians. For uncomplicated indications, physicians from hospitals providing coronary angioplasty and bypass surgery rated angiography as more appropriate than physicians from other hospitals. Physicians from New York and those employed by health maintenance organizations rated angiography as less appropriate than other physicians.
Conclusions Surveyed physicians agreed with clinical experts about the appropriateness of coronary angiography after myocardial infarction for most indications, indicating that well-designed expert panels can closely reflect the views of practicing physicians. Variations in beliefs among practicing physicians suggest that evaluations of medical practice should incorporate the views of a range of relevant types of physicians.
Although substantial effort has been devoted to developing and applying these methods, little attention has been directed to the views of practicing physicians whose care is evaluated by ratings of appropriateness. Clinical experts' ratings may be limited by the quality of scientific data and the challenges of applying these data to clinical practice.11,12,13 Members of expert panels are often drawn from academic medical centers, so their ratings may not reflect the experiences of physicians in other settings.14 Moreover, understanding differences among practicing physicians in their views about appropriate medical care can help ensure that clinical guidelines and evaluations encompass the views of a range of relevant types of physicians.15
We therefore surveyed internists, family practitioners, and cardiologists in five states to assess their ratings of the appropriateness of coronary angiography after acute myocardial infarction for 20 common clinical indications. These indications were also rated by a panel of clinical experts using an established consensus method. Our study was designed to address two primary questions: Do practicing physicians agree with an expert panel about the appropriateness of coronary angiography? And do practicing physicians vary according to specialty, type of hospital, state, or other characteristics in their beliefs about the appropriateness of coronary angiography?
Methods
Study Population
The study population consisted of cardiologists, internists, and family practitioners in five states: California, Florida, New York, Pennsylvania, and Texas. We focused on these types of physicians because they care for almost all patients with acute myocardial infarction16,17 and they are primarily responsible for recommending coronary angiography. These five states were selected because they have the largest numbers of patients with acute myocardial infarction in the population covered by Medicare, and their rates of coronary angiography after myocardial infarction vary widely.18
Physicians were identified from a list of names, addresses, and self-reported specialties obtained from the American Medical Association's master file (which includes members and nonmembers). We selected a random sample of 4345 physicians stratified according to state and specialty, including 329 cardiologists, 265 internists, and 275 family practitioners from each of the five study states. Physicians were ineligible if they reported at the beginning of the survey that they were in training or that they had not served as attending or consulting physicians for at least one patient hospitalized with myocardial infarction during the prior 12 months.
Ratings of the Expert Panel
During October 1995 we convened a national panel of nine physicians with expertise in the treatment of acute myocardial infarction, including five cardiologists, two cardiac surgeons, one general internist, and one family practitioner. These physicians were nominated by their professional organizations, including the American Academy of Family Physicians, the American Association for Thoracic Surgery, the American College of Cardiology, the American College of Physicians, the American College of Surgeons, the American Heart Association, and the Society of Thoracic Surgeons. Eight of these nine physicians had participated in a prior panel organized by RAND, and we used a similar method to obtain their ratings of the appropriateness of coronary angiography after acute myocardial infarction.3
Panelists were provided with an updated review of the literature that synthesized evidence on the appropriateness of coronary angiography.19 Eight hundred ninety clinical indications were specified for the use of this procedure within 12 weeks after acute myocardial infarction. For the subgroup of 92 indications relating to angiography during the initial hospitalization, key variables included the patient's age (
75 or <75 years), time since the onset of symptoms (<6, 6 to 12, or >12 hours), eligibility for and receipt of thrombolytic therapy, and presence of key complications, such as persistent or recurrent chest pain, stress-induced ischemia, and pulmonary edema. The panelists initially rated the appropriateness of each clinical indication on a 9-unit numerical scale ranging from 1, extremely inappropriate, to 5, uncertain, and 9, extremely appropriate. Using a modified Delphi method to foster consensus, the group discussed their initial ratings and each panelist then assigned final ratings to the individual indications.
Survey Instrument
From the 92 indications for coronary angiography during the initial hospitalization, we selected 20 indications to be rated by practicing physicians (see the Appendix). Our objective was to include a varied set of indications (differing in patient's age, presence or absence of complications, and expert panelists' ratings) that would apply to a substantial proportion of patients with myocardial infarction. In preliminary analyses of data from the Cooperative Cardiovascular Project of the Health Care Financing Administration at the time we designed the survey, approximately four fifths of Medicare patients with myocardial infarction in the five study states were represented by these 20 indications. The survey also collected professional information about the physicians and asked them to rate the importance of consultations, patient or family requests, guidelines, insurance-plan approvals, and the availability of angiography in their decisions about whether to recommend this procedure for patients who had had acute myocardial infarction during the previous 12 months, using questions adapted from a prior survey.20
Data Collection
During November 1996 the survey was mailed to physicians with a letter of support from the American College of Cardiology (for cardiologists), the American College of Physicians (for internists), or the American Academy of Family Physicians (for family practitioners). Physicians who did not respond to the initial mailing received at least two additional mailings. Eight or more attempts were also made to call nonresponding physicians to request their participation.
Statistical Analysis
Characteristics of survey respondents and the importance they attributed to factors influencing the use of coronary angiography were compared according to specialty with use of the Pearson chi-square test. In preliminary analyses, ratings of appropriateness by internists and family practitioners were similar, but the groups' ratings differed according to the number of patients physicians had treated for myocardial infarction in the prior year (
12 vs. >12), so we grouped these specialties together and stratified them in subsequent analyses according to this measure of volume. We also divided cardiologists according to whether they had performed coronary angiography during the previous year, because one earlier study had shown that the ratings of appropriateness differed according to this variable,15 and our preliminary analyses suggested that this variable influenced cardiologists' ratings more than the number of patients they treated for myocardial infarction.
Observed median ratings of appropriateness (and interquartile ranges) were calculated for each indication for the expert panel and the surveyed physicians. Using the Pearson correlation coefficient, we correlated the surveyed physicians' average ratings of appropriateness for patients under the age of 75 and 75 or older with their reports of the proportion of their patients with acute myocardial infarction in each of these age groups who had received coronary angiography during the prior year.
To compare more reliable estimates of the appropriateness of coronary angiography as rated by the surveyed physicians and the expert panelists and to determine the precision associated with these estimates, hierarchical regression models21 were fitted separately to the observed ratings from each group. This method accounted for three important sources of variability: the underlying appropriateness of each indication, each physician's propensity to rate angiography as more or less appropriate in general, and measurement error. To understand differences in ratings between the surveyed physicians and the expert panelists, we then incorporated patients' characteristics, including age, time since the onset of symptoms, and the presence or absence of complications, into the hierarchical regression models. Finally, to assess differences in ratings among the surveyed physicians in multivariable analyses, we used hierarchical regression models to compare the distribution of appropriateness ratings according to numerous physician's characteristics, including state, specialty, age, sex, number of patients treated for myocardial infarction in the prior year (for internists and family practitioners), performance of coronary angiography (for cardiologists), availability of coronary procedures at each physician's principal hospital, and employment by a health maintenance organization (HMO).
Hierarchical models were estimated with "Bayesian inference using Gibbs sampling" (BUGS) software, which employs simulation techniques to obtain parameter estimates in complex models.22 Estimates of appropriateness and the effects of the patient and physician characteristics listed above were obtained, and corresponding 95 percent confidence intervals were calculated. We also plotted smoothed density estimates of the empirical distributions for the effects of physician characteristics from the multivariable analysis of survey respondents. Effects were considered significant if the 95 percent confidence intervals for the groups being compared did not overlap.
Results
Response Rates
Completed surveys were received from 1058 physicians, including 602 cardiologists, 239 internists, and 217 family practitioners. An additional 842 physicians were ineligible because they were in training or had not cared for a patient with acute myocardial infarction in the prior year, and 647 physicians were deleted from the sample because they had moved with no forwarding address. Among physicians known to be eligible, 79 refused to participate. No response was received from 1719 physicians, and we estimate on the basis of the physicians who did respond that 57 percent would have been eligible.23 Thus, assuming this estimated eligibility rate among nonrespondents, the overall response rate was 50 percent, with rates of 47 percent for internists, 53 percent for family practitioners, and 49 percent for cardiologists. Response rates by state were 45 percent for California, 46 percent for Florida, 54 percent for New York, 56 percent for Pennsylvania, and 46 percent for Texas.
Surveyed Physicians' Characteristics and Factors Influencing Decisions about Angiography
The characteristics of respondents are presented according to specialty in Table 1. Among the cardiologists, 62 percent had performed coronary angiography during the previous year. Internists and family practitioners were more likely than cardiologists to cite consultations and hospital guidelines as very or moderately important factors in their recommendations about coronary angiography after myocardial infarction (Table 2). About half the physicians in each group rated patient or family requests and the availability of angiography at their own hospital as very or moderately important factors in their recommendations. Approximately one fifth of the physicians perceived insurance-plan guidelines or requirements for approval as important factors.
|
|
When we compared the ratings of the surveyed physicians with those of the expert panel (Table 3), the median ratings were identical for 6 of the 10 indications in patients under the age of 75 and within 1 unit on the 9-unit scale for the remaining 4 indications (the survey respondents' ratings were higher in 2 cases and the expert panelists' ratings higher in 2). Agreement was less consistent for patients who were 75 or older, with identical median ratings for four indications and higher median ratings by the survey respondents than by the expert panelists for the other six indications. Two of these six indications (indications K and M) differed by 3 units on the 9-unit scale, and one indication (indication T) differed by 2 units. Surveyed physicians' observed ratings of appropriateness were moderately correlated with their self-reported use of coronary angiography after myocardial infarction for patients under the age of 75 (r=0.31, P<0.001) and 75 or older (r=0.37, P<0.001).
|
75 and <75 years, –1.4 units; 95 percent confidence interval, –2.1 to –0.6), but not on those of the surveyed physicians (mean difference, –0.4 unit; 95 percent confidence interval, –1.2 to +0.5).
|
In multivariable analyses, differences in beliefs were evident between cardiologists and primary care physicians and within each of these groups. For patients with cardiac complications (Figure 2A), cardiologists who performed invasive procedures and those who did not gave similar ratings of appropriateness, and ratings by all cardiologists combined were significantly higher than those of primary care physicians with high numbers of patients with myocardial infarctions and those with low numbers (mean differences, 0.6 and 1.0 unit, respectively). For patients with uncomplicated myocardial infarctions (Figure 2B), however, cardiologists who did not perform invasive procedures and primary care physicians reported similar ratings, whereas the ratings of cardiologists who performed invasive procedures were significantly higher (mean differences, 0.5 to 0.6 unit).
Access to coronary procedures at the physicians' hospitals was not associated with their ratings of the appropriateness of angiography for patients with complications (Figure 3A), but for patients with uncomplicated infarctions (Figure 3B), physicians with access to coronary angioplasty and bypass surgery gave significantly higher ratings than physicians from hospitals offering coronary angiography alone or no coronary procedures (mean difference, 0.6 unit). Smaller significant differences (0.3 unit) were noted between ratings by physicians in New York and those by physicians in Florida and Pennsylvania (Figure 4A) and between physicians employed by HMOs and other physicians (Figure 4B); these patterns were similar for both complicated and uncomplicated indications (data not shown). Physicians' age and sex were not associated with their beliefs about the appropriateness of angiography.
|
In this survey of more than 1000 physicians in five states, ratings of the appropriateness of coronary angiography after acute myocardial infarction were very similar to those of a panel of clinical experts. This convergence suggests that evaluations of medical practice based on the judgments of expert panels can closely reflect the beliefs and experiences of practicing physicians for well-studied procedures. However, variations in ratings among practicing physicians suggest that the judgments of physicians participating in panels will be influenced by their professional characteristics, so the composition of panels should be carefully considered when one is interpreting their results.
Agreement between the expert panel and the surveyed physicians was greatest for patients under the age of 75. For patients who were 75 or older, the surveyed physicians rated angiography as more appropriate than did the expert panel, particularly for elderly patients with uncomplicated myocardial infarctions who presented within six hours after the onset of symptoms. Differences of opinion between clinical experts and surveyed physicians highlight the need for further clinical research to assess the risks and benefits of coronary angiography in older patients who may be candidates for angioplasty or thrombolytic therapy.24,25,26,27
Beliefs about the appropriateness of coronary angiography differed substantially among primary care physicians, cardiologists who did not perform invasive procedures, and cardiologists who did, as seen in expert panels.15 Other studies have demonstrated that cardiologists' patients are much more likely than the patients of primary care physicians to undergo coronary angiography.16,17,28 Among primary care physicians in our survey, those who treated larger numbers of patients with myocardial infarctions rated angiography as more appropriate for complicated indications than did physicians with fewer such patients, possibly because of their greater experience in referring such patients for invasive coronary procedures. Previous surveys have found that about half of primary care physicians underestimate the benefits of coronary-artery bypass surgery,29 and some primary care physicians may also overestimate the risks of coronary angiography.30 The difference we found in ratings of angiography for uncomplicated indications between cardiologists who performed invasive procedures and those who did not suggests persistent uncertainty or disagreement about the role of coronary angioplasty in patients with acute myocardial infarction.26,27,31 Future studies should explore the consequences of differing beliefs and practices within and between specialties.
Physicians who practiced in hospitals that offered coronary angioplasty and bypass surgery were more likely to view angiography as appropriate for uncomplicated infarctions than physicians in hospitals without these procedures — a finding consistent with the results of other studies in which the availability of coronary procedures strongly predicted their use.18,32,33,34 Physicians employed by HMOs were somewhat less likely than other physicians to view angiography as appropriate, but differences in practice within HMOs may be more consequential.35 Coronary angiography was also perceived to be somewhat less appropriate by physicians in New York than by physicians in other states, which may contribute to the lower rate of coronary angiography after myocardial infarction in New York.18,36,37 Larger differences in appropriateness ratings have been noted between expert panels from the United States and the United Kingdom.5
Our study has four potential limitations. First, our response rate was similar to that of other published surveys of physicians,38 but the physicians who responded may have been more familiar than those who did not with studies of the appropriate use of coronary procedures, thereby fostering greater agreement with the expert panel. However, median ratings of appropriateness would have had to differ substantially between respondents and nonrespondents to alter the high level of agreement between the surveyed physicians and the expert panel for most indications. Second, although the elements of the clinical indications were determined by expert panelists from clinical experience and an extensive review of the literature,19 practicing physicians may incorporate other clinical factors, such as the anatomical site or size of a myocardial infarction, in decisions about angiography for individual patients. Third, coronary angiography is a relatively well studied procedure for which guidelines have been disseminated by respected professional organizations,39 so the generalizability of our findings to other important procedures needs to be assessed. Fourth, we did not assess physicians' actual use of coronary angiography.40 Nonetheless, physicians' ratings of appropriateness in this survey were correlated with self-reported rates of angiography, and in another recent study physicians' responses to clinical scenarios regarding coronary angiography were significantly associated with actual angiography rates in the areas they served.41
In conclusion, surveyed physicians in five large states agreed closely with an expert panel about the appropriateness of coronary angiography after acute myocardial infarction, particularly for patients under the age of 75. Thus, studies and guidelines that use the judgments of expert panels to evaluate and improve medical care can be highly concordant with the views of practicing physicians. Areas of disagreement between clinical experts and practicing physicians or within either group — over the appropriateness of angiography for older patients or those with uncomplicated infarctions, for example — point to important topics for further clinical research to guide practice. Local and national organizations striving to promote more appropriate medical care should identify and address the systematic ways in which clinical beliefs may differ among practicing physicians.
|
|
We are indebted to Christine Kreider and the staff of Northeast Research for assistance with data collection; to Christine K. Cassel, M.D., of the American College of Physicians, Richard P. Lewis, M.D., of the American College of Cardiology, and Daniel J. Ostergaard, M.D., of the American Academy of Family Physicians, for providing letters of support to participating physicians; and to Arnold M. Epstein, M.D., and Matthew H. Liang, M.D., M.P.H., for reviewing an earlier draft of the manuscript.
From the Department of Medicine, Division of General Medicine and Primary Care, Section on Health Services and Policy Research (J.Z.A.), and the Department of Radiology (B.J.M.), Brigham and Women's Hospital and Harvard Medical School; the Department of Health Care Policy, Harvard Medical School (J.Z.A., M.B.L., S.-L.T.N., E.G., B.J.M.); and the Department of Biostatistics, Harvard School of Public Health (S.-L.T.N.) — all in Boston.
Address reprint requests to Dr. Ayanian at the Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave., Boston, MA 02115.
References
Physicians were asked to rate the appropriateness of coronary angiography according to the following definition and clinical indications on a 9-point scale ranging from extremely inappropriate (1) to uncertain (5) and extremely appropriate (9):
Appropriateness means that the expected health benefits (i.e., increased life expectancy or functional capacity and relief of pain or anxiety) exceed the expected negative consequences (i.e., mortality, morbidity, pain, or anxiety associated with the procedure) by a sufficiently wide margin, exclusive of cost, that angiography is worth doing. Since angiography does not reduce morbidity or mortality, its benefit or harm is generally related to its effect on the subsequent use of medical therapy, percutaneous transluminal coronary angioplasty, or coronary-artery bypass graft surgery. Use your own best clinical judgment for patients presenting to you in each of the following situations during an initial hospitalization for acute myocardial infarction.
In a patient under the age of 75 with acute myocardial infarction, how would you rate the appropriateness of coronary angiography?
A. Within six hours of the onset of symptoms, the patient has not received thrombolytic therapy because of strong contraindications, and the myocardial infarction is uncomplicated.
B. Within six hours of the onset of symptoms, the patient has not received thrombolytic therapy because of strong contraindications and has persistent chest pain.
C. Within six hours of the onset of symptoms, the patient has not received thrombolytic therapy but has no strong contraindications, and the myocardial infarction is uncomplicated.
D. Within six hours of the onset of symptoms, the patient has not received thrombolytic therapy but has no strong contraindications and has persistent chest pain.
E. Within six hours of the onset of symptoms, the patient has received thrombolytic therapy, and the myocardial infarction is uncomplicated.
F. Within six hours of the onset of symptoms, the patient has received thrombolytic therapy and has persistent chest pain.
G. Between 12 hours after the onset of symptoms and discharge, the patient has not received thrombolytic therapy, and the myocardial infarction is uncomplicated.
H. Between 12 hours after the onset of symptoms and discharge, the patient has not received thrombolytic therapy and has persistent chest pain.
I. Between 12 hours after the onset of symptoms and discharge, the patient has not received thrombolytic therapy and has persistent pulmonary edema.
J. Between 12 hours after the onset of symptoms and discharge, the patient has not received thrombolytic therapy and has stress-induced ischemia.
Indications K through T were phrased identically for patients who were 75 or older.
| |||||||||||||||||||||||||||||||||||||||||||||||
Related Letters:
Assessing the Appropriateness of Medical Care
Wassertheil-Smoller S., Tobin J., Steingart R., Hsu J., Black N., Ayanian J. Z., Shekelle P. G., Park R.E., Naylor C. D.
Extract |
Full Text
N Engl J Med 1998;
339:1478-1481, Nov 12, 1998.
Correspondence
This article has been cited by other articles:
HOME | SUBSCRIBE | SEARCH | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | PRIVACY | TERMS OF USE | HELP | beta.nejm.org Comments and questions? Please contact us. The New England Journal of Medicine is owned, published, and copyrighted © 2009 Massachusetts Medical Society. All rights reserved. |
Previous
