|
|
Background In 2004, after a series of national initiatives associated with marked improvements in the quality of care, the National Health Service of the United Kingdom introduced a pay-for-performance contract for family practitioners. This contract increases existing income according to performance with respect to 146 quality indicators covering clinical care for 10 chronic diseases, organization of care, and patient experience.
Methods We analyzed data extracted automatically from clinical computing systems for 8105 family practices in England in the first year of the pay-for-performance program (April 2004 through March 2005), data from the U.K. Census, and data on characteristics of individual family practices. We examined the proportion of patients deemed eligible for a clinical quality indicator for whom the indicator was met (reported achievement) and the proportion of the total number of patients with a medical condition for whom a quality indicator was met (population achievement), and we used multiple regression analysis to determine the extent to which practices achieved high scores by classifying patients as ineligible for quality indicators (exception reporting).
Results The median reported achievement in the first year of the new contract was 83.4 percent (interquartile range, 78.2 to 87.0 percent). Sociodemographic characteristics of the patients (age and socioeconomic features) and practices (size of practice, number of patients per practitioner, age of practitioner, and whether the practitioner was medically educated in the United Kingdom) had moderate but significant effects on performance. Exception reporting by practices was not extensive (median rate, 6 percent), but it was the strongest predictor of achievement: a 1 percent increase in the rate of exception reporting was associated with a 0.31 percent increase in reported achievement. Exception reporting was high in a small number of practices: 1 percent of practices excluded more than 15 percent of patients.
Conclusions English family practices attained high levels of achievement in the first year of the new pay-for-performance contract. A small number of practices appear to have achieved high scores by excluding large numbers of patients by exception reporting. More research is needed to determine whether these practices are excluding patients for sound clinical reasons or in order to increase income.
In 2004, the National Health Service committed £1.8 billion ($3.2 billion) in additional funding over a period of three years for a new pay-for-performance program for family practitioners (the sole type of primary care physician in the United Kingdom).8 This program was intended to increase family practitioners' income by up to 25 percent, depending on their performance with respect to 146 quality indicators relating to clinical care for 10 chronic diseases, organization of care, and patient experience.9 For the clinical indicators, practices claim points that generate payments according to the proportion of patients for whom they achieve each target (for examples, see Table 1; for the complete list, see Supplementary Appendix 1, available with the full text of this article at www.nejm.org). Points are awarded on a sliding scale within the payment range. For example, for asthma indicator number 6, practices gain points for clinically reviewing at least 25 percent of patients with asthma in the previous 15 months. The maximum of 20 points is gained if at least 70 percent of patients with asthma are reviewed. For 2004–2005, payment was limited to £76 ($133) per point, adjusted for the relative prevalence of the disease (payment is multiplied by the square root of the prevalence of the disease among the patients served by the practice and divided by the square root of the mean national prevalence of the disease). A maximum of 1050 points was available, which was equivalent to an average of £79,800 ($139,400). From 2005–2006 onward, practices have been earning £125 ($218) per point. The payments are in addition to the practices' core funding, which is based on the number of patients, adjusted for characteristics of the patients and the area.
|
Evidence-based quality indicators should not be applied unthinkingly, since patients have coexisting conditions that affect their optimal care.15 It is inappropriate, for example, to strive to control the cholesterol level of someone terminally ill with cancer. Consequently, the new U.K. pay-for-performance contract allows family practitioners to exclude patients from eligibility for specific indicators in the performance calculations.16 The reasons for these exclusions (termed "exception reports") are summarized in Table 2. However, exception reporting also provides an opportunity for family practitioners to increase their income by inappropriately excluding patients for whom they have missed the targets (a practice known as gaming).
|
Methods
Analyses in this article are restricted to the 76 clinical quality indicators, which account for 550 of the 1050 potential points. The remaining 500 points were based on quality indicators assessing organizational quality and patient experience (not analyzed here). Details of data and methods are given in Supplementary Appendix 2, available with the full text of this article at www.nejm.org. The practices recorded the number of patients with each condition who were listed in their practice registers on February 14, 2005 (R0). For each clinical indicator (i), the practices flagged the electronic medical records of patients deemed appropriate for that indicator (the number of these patients is the denominator, Di) and patients for whom the indicator was met (the number of these patients is the numerator, Ni). These data were extracted by an automated national computer system and collated in a central database called the Quality Management and Analysis System (QMAS). Register data for each condition (R0) and summary data for each indicator (Di, Ni) were published for all practices in England in August 2005 (available at www.ic.nhs.UK./services/qof/).
Data for exception reports were not available on the QMAS database and were imputed. For example, QMAS might document that there were 100 patients (R0) in a practice's hypertension register and that the practice met the target of recording the blood pressure of 75 (Ni) of 80 (Di) patients with hypertension. The practice's reported achievement on this indicator is 94 percent (Ni/Di). We can calculate that 20 patients (100–80, i.e., R0–Di) were excluded (exception-reported) for this indicator by the practice, and that the actual level of achievement of blood-pressure recording in the whole population of patients with hypertension was 75 percent (75 of 100). The practices were paid on the basis of their reported achievement (94 percent in this example).
Imputation of exception reports was possible only for indicators that were based on all patients with the condition (30 of the 76 indicators) (see Supplementary Appendix 1). For example, imputation was possible for asthma indicator number 6 (the percentage of patients with asthma reviewed in the previous 15 months) but not for epilepsy indicator number 4 (the percentage of patients 16 years of age or older receiving drug treatment for epilepsy who were convulsion-free for the previous 12 months), because the latter indicator is based on a subgroup of patients. We therefore calculated a score that measured the overall propensity of a practice to exclude patients. A complication of the QMAS data-collection method was that although the practices reported overall disease prevalence on February 14 (R0), the numerators and denominators for individual indicators were reported six weeks later, on March 31, and the practices could continue to add patients to their disease registers during this period. Consequently, the denominator for an indicator (Di) could be greater than the reported number of patients registered for the disease (R0), and as a result the true number of patients registered on March 31 (R1) would be unknown. In such cases, we estimated R1 by using the largest available denominator, Di, for the indicators for that disease.
We report practice outcomes with respect to clinical indicators in three ways: the reported achievement is the proportion of patients declared appropriate for an indicator for whom the practice met the indicator (Ni÷Di); the population achievement is the number of patients for whom the indicator was met as a proportion of the estimated number of patients with the condition (Ni÷R1); and the rate of exception reporting is the estimated number of patients who were exception-reported (excluded) for the indicator as a proportion of the estimated number of patients with the condition [(R1–Di)÷R1].
Summary outcome scores for each condition were constructed as weighted mean scores for the score on each indicator, where the weights were the number of points available for each indicator. A global score was similarly constructed as a weighted mean across all conditions.
Information on practice characteristics was taken from the 2004 General Medical Statistics database maintained by the Department of Health. Socioeconomic characteristics were attributed to each practice with data from the 2001 Census and the Indices of Deprivation17 for the population of the electoral district where the practice was located. Although all practices took part in the new quality incentive framework, 42 percent were operating under an alternative Primary Medical Services contract with minor differences in the pay-for-performance scheme that diluted their financial incentives. However, the same clinical data were collected for all practices. We included an indicator for Primary Medical Services practices.
The QMAS database contains data from 8576 practices with 1 or more family practitioners (with a mean of 3.2 full-time equivalents) providing services for a defined population. Practices were excluded from the study if they had fewer than 1000 patients (61 practices) or if the reported register was missing (92 practices), contained no patients (272 practices), or included less than half the patients subsequently reported for individual indicators (109 practices). Our results are drawn from 8105 practices (94.5 percent; some practices had more than one of the exclusion criteria).
Linear multiple least-squares regressions with robust estimates of error variance were estimated for each outcome with the use of Stata software, version 8. The practice scores had heterogeneous variance, depending on patient numbers, and therefore analytical weights that were approximately inversely proportional to the variance at each practice were used. Each analysis used the same set of practice and population characteristics as explanatory variables. The regression for reported achievement also included the rate of exception reporting. Indicator variables (fixed effects) were included to allow for unobserved effects of policies of the Primary Care Trust where the practice was located. Outcome variables were expressed as percentages ranging from 0 to 100. All variables were divided by their standard deviations, and therefore the reported coefficients show the increase in the standard deviations of outcome variables for each increase of one standard deviation in predictor variables. Sensitivity analyses with respect to weighting scores and calculating rates of exception reporting are reported in Supplementary Appendix 2.
Ethics-committee approval was not required for this study, since it was based on publicly available data.
Results
Practice Performance under the New Financial Incentive Program
The practices scored a median of 1003 points (interquartile range, 948 to 1033), or 95.5 percent of those available, earning an average of £76,200 ($133,200) each. Of the 8105 practices, 230 (2.8 percent) scored the maximum of 1050 points. On the clinical indicators, the practices scored a median of 532 points (interquartile range, 504 to 545), or 96.7 percent of those available, with 591 practices (7.3 percent) scoring the maximum of 550.
The median reported achievement — that is, the proportion of eligible patients for whom the targets were actually achieved — was 83.4 percent overall; the reported achievement for individual diseases ranged from 80.1 percent for diabetes to 96.0 percent for hypothyroidism (Table 3).
|
The median population achievement (the proportion of all patients with the condition for whom the target was achieved, including those who were exception-reported) was estimated at 82.9 percent overall, ranging from 72.8 percent for asthma to 95.3 percent for hypothyroidism (Table 3).
Factors Associated with Overall Performance
Characteristics of practices, patients, and local areas had significant effects on population achievement (Table 4). Achievement was lower in practices with a high proportion of patients who were living in single-parent or low-income households or were 65 years of age or older. Achievement was also lower in larger practices and in practices with a high proportion of family practitioners who received their medical education outside the United Kingdom or were 50 years of age or older. Achievement was higher in practices with a high ratio of family practitioners to patients and lower in practices that were on the Primary Medical Services contract. However, the multiple regression model explained only 20 percent of the variation between practices, and all of these effects were small. The number of patients, for example, had the effect of reducing achievement by 0.13 percent for every additional 1000 patients on the practice list.
|
The rates of exception reporting were higher in Primary Medical Services practices and lower in practices with larger populations of elderly patients, patients with good self-rated health, and patients without any formal educational qualifications (Table 4). However, the model explained only 20 percent of the variation between practices, and the effects of individual explanatory variables were small.
Discussion
In the first year of the pay-for-performance program, English family practitioners performed extremely well with respect to the quality targets, attaining a median of 96.7 percent of the available points for clinical indicators. This greatly exceeded the 75 percent predicted when the scheme was negotiated, and consequently the cost to the taxpayers was considerably more than expected. Before the new contract was implemented, family practitioners typically earned from £70,000 to £75,000 ($122,000 to $131,000). The pay-for-performance program increased the gross income of the average family practitioner by £23,000 ($40,200), although the practitioners bore any additional nursing and administrative costs of meeting the targets. In 2005–2006, family-practitioner income will rise even more, since quality payments have been increased to £125 ($218) per point.
Exact comparisons with U.S. data on quality of care are difficult because of differences in indicators, dates of data collection, and samples.18 However, some limited comparisons are possible. For example, 91 percent of patients with diabetes had their glycated hemoglobin levels measured in 2004–2005 under the new pay-for-performance program in England. In comparison, glycated hemoglobin levels were measured in 94 percent of patients with diabetes treated by the U.S. Department of Veterans Affairs in 1999–200019 and 93 percent of such patients in 2000–2002,20 in 83 percent of patients with diabetes treated by commercial managed care groups in the United States in 2000–2002,20 in 82 percent of patients with diabetes treated by Pacific Northwest physician groups with pay-for-performance programs in 2001–2002, in 64 percent of patients with diabetes treated by California physician groups with pay-for-performance programs in 2002–2003,21 in 89 percent of Medicare patients with diabetes in 2004,22 and in 76 percent of Medicaid patients with diabetes in 2004.22 For other aspects of diabetes care, including blood lipid testing, eye examinations, microalbuminuria screening, and influenza vaccination, performance in the first year of the new contract also placed family practitioners in England on a par with their better-performing U.S. peers.
There is no baseline with which to compare performance in the first year of the U.K. program, although the quality of care was already improving before its introduction.6 The high levels of achievement might suggest that the targets were too easy to achieve.23 The scheme has been revised for 2006–2007: all minimum and some maximum payment thresholds have been raised, 30 indicators have been dropped or modified, and 18 new indicators have been introduced.24 The high achievement levels might also have resulted from misreporting by practices. To counter misreporting, Primary Care Trusts, statutory bodies responsible for the delivery of health care in local areas, inspect all local practices and undertake detailed audits of randomly selected practices and those suspected of incorrect or fraudulent returns. The results of these audits are not, however, publicly available.
Because achievement was universally high, there was little variation between practices. It was not surprising, therefore, that socioeconomic and demographic factors, which profoundly affect population health and the use of health care facilities,25,26 had relatively little influence on achievement. Although practices that served lower-income populations had worse overall population achievement, the effect was small, and they were no more likely to use exception reporting to exclude patients than were practices with more affluent populations. Deprivation-related health inequalities therefore appear unlikely to have been greatly increased by the introduction of the financial incentive program. Smaller practices performed marginally better overall than large ones, although there was much greater variation in the performance of small practices, and many smaller practices are believed to have merged in the face of the administrative pressures from the new contract.
Imputation of rates of exception reporting was possible for only 30 of the clinical indicators (39 percent), and we cannot determine how representative these indicators were. There was a significant positive relationship between rates of exception reporting and reported achievement for these indicators, but the effect was small. It is possible that practices that were better at identifying and treating patients with chronic conditions also tended to identify more patients for whom the targets were inappropriate. Alternatively, practices may have "gamed" the new system. The generally low levels of exception reporting suggest that large-scale gaming was uncommon. However, a small minority of practices exception-reported a much larger proportion of their patients: 91 (1.1 percent) excluded over 15 percent of their patients. These practices warrant closer examination to determine whether their use of exception reporting was appropriate.
The rate of exception reporting varied considerably according to disease group. There were very low levels of exception reporting for hypothyroidism and relatively high levels for mental health problems, coronary heart disease, and chronic obstructive pulmonary disease. This variation may reflect the nature of the indicators for each disease. For example, to meet the main hypothyroidism target, practices were required to record that a patient's thyroid functions had been checked in the previous 15 months. This was a relatively easy target to meet; hence the achievement level was high and there was little reason to exclude these patients. Since the indicator carried only a modest financial reward of 6 points (£456, or $800), there was also little incentive to game. In comparison, the main mental health indicator required a review of medication, physical health, and coordination arrangements with secondary care for patients with severe long-term mental health problems. Not only would one expect legitimate exception reporting to be higher for this indicator, but the incentive to game would also be greater, since the indicator was worth 23 points (£1,748 or $3,050).
Several lessons can be drawn from the U.K. experience. First, the U.K. program was costly and was funded with substantial additional monies rather than by restructuring existing payment systems. In addition to the payments for achieving quality targets, there were further costs, to both the practitioners and the government, of developing and implementing the information-technology systems required to monitor the program. Budget-neutral programs would face greater resistance from family practitioners. Second, a clear baseline is needed to avoid paying for improvements that have already occurred. Third, geographically staggered introduction would enable policymakers to better estimate the quality effects of the program. Fourth, introducing pay-for-performance incrementally reduces risks for providers and payers. Fifth, payers should allow for the possibility of higher-than-expected achievement. Sixth, the risk of inappropriate treatment can be decreased with the use of mechanisms such as exception reporting, but monitoring is required to prevent abuse.
The U.K. experience suggests that greater changes in professional practice can be achieved through pay-for-performance programs than previous research indicates.21 We do not know whether the size of the financial incentives made the difference in the United Kingdom, and if so, how big incentives need to be. Whatever the case, financial incentives should be aligned to physicians' professional values to avoid serious distortions of care.
The National Primary Care Research and Development Centre receives core funding from the U.K. Department of Health. In 2002 and 2003, Dr. Roland was a member of a small team of academics who advised the government and professional negotiating teams that developed the pay-for-performance scheme. No other potential conflict of interest relevant to this article was reported.
The views expressed are those of the authors and not necessarily those of the U.K. Department of Health.
Source Information
From the National Primary Care Research and Development Centre, University of Manchester, Manchester, United Kingdom.
Address reprint requests to Dr. Doran at the National Primary Care Research and Development Centre, Williamson Bldg., University of Manchester, Manchester M13 9PL, United Kingdom, or at tim.doran{at}manchester.ac.uk.
References
| |||||||||||||||||||||||||||||||||||||||||||||||||||||
Related Letters:
Pay-for-Performance Programs in the United Kingdom
Sanderson A. A.F., Tahrani A. A., Varughese G. I., Macleod A. F., Epstein A. M.
Extract |
Full Text |
PDF
N Engl J Med 2006;
355:1832-1833, Oct 26, 2006.
Correspondence
This article has been cited by other articles:
HOME | SUBSCRIBE | SEARCH | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | PRIVACY | HELP | beta.nejm.org Comments and questions? Please contact us. The New England Journal of Medicine is owned, published, and copyrighted © 2008 Massachusetts Medical Society. All rights reserved. |
Previous
