The New England Journal of Medicine
e-mail icon  FREE NEJM E-TOC    HOME   |   SUBSCRIBE   |   CURRENT ISSUE   |   PAST ISSUES   |   COLLECTIONS   |    Advanced Search
Sign in | Get NEJM's E-Mail Table of Contents — Free | Subscribe
 
Special Article
PreviousPrevious
Volume 344:817-823 March 15, 2001 Number 11
NextNext

Expenditures for the Care of HIV-Infected Patients in the Era of Highly Active Antiretroviral Therapy
Samuel A. Bozzette, M.D., Ph.D., Geoffrey Joyce, Ph.D., Daniel F. McCaffrey, Ph.D., Arleen A. Leibowitz, Ph.D., Sally C. Morton, Ph.D., Sandra H. Berry, M.A., Afshin Rastegar, M.S., David Timberlake, M.P.H., Martin F. Shapiro, M.D., Ph.D., Dana P. Goldman, Ph.D., for The HIV Cost and Services Utilization Study Consortium

 

This Article
-Abstract
- PDF

Commentary
-Editorial
 by Steinbrook, R.

Tools and Services
-Add to Personal Archive
-Add to Citation Manager
-Notify a Friend
-E-mail When Cited

More Information
-PubMed Citation
ABSTRACT

Background The introduction of expensive but very effective antiviral medications has led to questions about the effects on the total use of resources for the care of patients with human immunodeficiency virus (HIV) infection. We examined expenditures for the care of HIV-infected patients since the introduction of highly active antiretroviral therapy.

Methods We interviewed a random sample of 2864 patients who were representative of all American adults receiving care for HIV infection in early 1996, and followed them for up to 36 months. We estimated the average expenditure per patient per month on the basis of self-reported information about care received.

Results The mean expenditure was $1,792 per patient per month at base line, but it declined to $1,359 for survivors in 1997, since the increases in pharmaceutical expenditures were smaller than the reductions in hospital costs. Use of highly active antiretroviral therapy was independently associated with a reduction in expenditures. After adjustments for the interview date, clinical status, and deaths, the estimated annual expenditure declined from $20,300 per patient in 1996 to $18,300 in 1998. Expenditures among subgroups of patients varied by a factor of as much as three. Pharmaceutical costs were lowest and hospital costs highest among underserved groups, including blacks, women, and patients without private insurance.

Conclusions The total cost of care for adults with HIV infection has declined since the introduction of highly active antiretroviral therapy. Expenditures have increased for medications but have declined for other services. However, there are large variations in expenditures across subgroups of patients.


The introduction of highly active combination antiretroviral therapy has been accompanied by increased concern about the costs of care for patients with human immunodeficiency virus (HIV) infection.1,2,3,4,5,6,7 Some investigators have reported an association between use of highly active antiretroviral therapy and reductions in overall health care costs.8,9 However, the available data are from short-term studies that involved nonrepresentative samples, that were based at major academic centers, or that did not examine differences in expenditures among subgroups of patients. To provide a better basis for clinical and policy debates, we analyzed expenditures for a nationally representative sample of HIV-infected patients over a period of three years.10,11

Methods

Sample

Detailed information on the study sample has been reported elsewhere.12 Briefly, we assembled a national probability sample representing all adults with known HIV infection who made at least one visit for regular care at a health care facility other than a military, prison, or emergency department facility in the contiguous United States in January or February 1996. We selected the sample from comprehensive lists in three stages. First, we randomly selected 28 cities plus 25 clusters of rural counties.13 From these areas, we randomly selected 90 urban and 22 rural physicians from among those identified as caring for HIV-infected patients in a screening survey of approximately 4000 physicians, plus an additional 58 urban and 29 rural providers identified by local informants. Finally, we randomly selected patients from anonymous lists of patients seen by participating providers in January or February 1996 and oversampled women and members of private staff-model health maintenance organizations.

Centrally trained interviewers from the National Opinion Research Center used computer-assisted personal interviewing instruments to conduct 92 percent of the interviews in person and the remainder by telephone.14 The coverage rate for interviews (the ratio of the number of patients represented to the number who would have been represented if there had been no refusals to participate) was 68 percent. Partial information was available on the majority of nonparticipants. The RAND institutional review board and local boards, where available, reviewed informed-consent materials and approved the study. Wherever a local board was unavailable, providers signed Single Project Assurances or Independent Investigator Agreements. All patients gave written informed consent to participate in the study.

Data on Costs

We conducted four sets of interviews over a period of 36 months, beginning in January 1996. Study participants reported their use of resources according to category (e.g., clinic visits and hospital days) for the six-month period before the base-line interview and for the period between interviews 2 and 3. In the fourth interview, participants reported pharmaceutical use for the prior six months and service use for the period between interviews.

We converted resource use into estimated expenditures by assigning costs in 1996 U.S. dollars to services and drugs. For services, we estimated costs on the basis of either payments or charges plus ratios of payments to charges obtained from the financial records for about a third of hospital stays and for about a quarter of outpatient visits (including visits to the emergency department). The resulting estimates were similar to prior estimates: $1,657 per hospital day, $347 for a visit to the emergency department, and $178 for an outpatient visit other than a visit to the emergency department.11

We estimated pharmaceutical use on the basis of participants' responses to questions about use and expert judgment on dosages. We obtained prices from a national retail pharmacy, a federal distributor of drugs, and a hospital buying consortium. In the fourth interview, participants were asked only about the use of antiretroviral drugs. Since the cost of other drugs was nearly constant, at an average of $235 per month, in the second and third interviews, for each respondent, we imputed the value for the fourth interview to be equal to the value for the third interview.

Statistical Analysis

We converted expenditures to rates per patient per month and used "hot deck" imputation to fill in missing values, which accounted for less than 5 percent of data on CD4 counts and insurance coverage, and for less than 0.5 percent of other data.15 We applied unique analytic weights to the responses of each patient at each interview so that the sample would represent the target population at base line and at follow-up. The weights were used to adjust for differential selection probabilities, nonresponse, multiple observations, and loss to follow-up but not death. All statistical tests were two-sided, and all tests and confidence intervals were adjusted to account for the complex sample design and the use of multiple observations.16

We used weighted responses to estimate simple average expenditures per month for the represented population and for subgroups of interest. The interview dates and, after base line, the period of recall between interviews varied among the participants, so reported utilization reflected different but overlapping parts of the follow-up period. We took advantage of this to identify the separate effects of the interview date and characteristics of the patient as follows. We created a data set with one record per patient per month for the entire study period and performed a regression analysis to estimate the fixed independent effects of the interview month and of the characteristics of the patient on expenditures. We used the results to predict expenditures for each respondent after replacing the actual month of the interview with the median month for each of the four interviews, thereby adjusting for time dispersion within each set of interviews. We also used the same results in the same way to estimate expenditures for each month while holding the characteristics of the patients constant and equal to those of the base-line sample, thereby eliminating the effects of deaths and changes in clinical status on estimates of the trends in cost. We performed sensitivity analyses on all estimations. We also examined the predictors of a decline in expenditures between the base-line and third interviews for individual patients.

Results

We interviewed a randomly selected sample of 2864 patients at base line (median date of the first interview, June 1996) and conducted second, third, and fourth interviews (median dates, March 1997, November 1997, and October 1998) with 2466, 2265, and 1915 of these patients, respectively. This sample directly represents the approximately 231,000 adults who received care for HIV infection in the contiguous United States during the first two months of 1996, as well as the 220,000, 214,000, and 181,000 persons in that population who survived to the dates of the three respective follow-up interviews. We have presented details of the represented population elsewhere, and all data reported here refer to that population rather than to the sample.11

The estimated mean health care expenditure for the represented population was $1,792 per patient per month at base line. The expenditure declined to $1,359, $1,419, and $1,410 per patient per month for those surviving to the second, third, and fourth interviews, respectively (Table 1). Two confirmatory analyses showed a similar pattern of decline. The first included only patients who survived for the entire study period. The second included only patients who were not asked about care received during the sampling period because they were interviewed at least six months later. The consistency of these results is evidence that the observed decline in costs was not an artifact of attrition, deaths, or the resolution of illnesses requiring medical attention during the sampling period.

View this table:
[in this window]
[in a new window]
 
Table 1. Unadjusted and Adjusted Monthly Expenditures for the Care of HIV-Infected Patients, According to the Median Interview Date.

 
Estimated expenditures given in the remainder of this report have been adjusted to compensate for the dispersion of interview dates, changes in clinical status, and deaths. This approach simulates a series of monthly cross-sectional assessments of expenditures in a single, stable population. At the median months of the four interviews — June 1996, March 1997, November 1997, and October 1998 — the adjusted expenditures were $1,749, $1,512, $1,534, and $1,503 (Table 1). Changes in adjusted expenditures between the base-line and second, third, and fourth interviews were –$237 (95 percent confidence interval, –$388 to –$88), –$215 (95 percent confidence interval, –$350 to –$81), and –$246 (95 percent confidence interval, –$451 to –$41) per patient per month, respectively, but the differences between the second or third and later interviews were smaller and not significant.

The adjusted total expenditure was $20,300 per patient in 1996 but dropped to $18,300 per patient in 1998. Changes in total expenditures per month are shown in Figure 1. The estimated total expenditure in the first month of the study, January 1996, was $1,811 per patient per month. By mid-1997, when the use of highly active antiretroviral therapy had become common, the expenditure for drugs had increased from $618 to $821 per patient per month, and hospital costs had declined from $878 to $500 per patient per month, resulting in an overall decrease in expenditures to $1,521 per patient per month.17 From mid-1997 through the end of 1998, adjusted total expenditures changed little, but by the end of the study period, hospital costs had increased to the mid-1996 levels of $594 per patient per month.


View larger version (12K):
[in this window]
[in a new window]
 
Figure 1. Average Monthly Expenditures for the Care of HIV-Infected American Adults (Outpatient Services, Inpatient Services, and Drugs) between January 1996 and January 1999, Adjusted for Loss to Follow-up and Death.

 
Independent predictors of a decline in expenditures for individual patients between the first and third interviews were public insurance coverage (odds ratio for the comparison with private coverage, 2.5; 95 percent confidence interval, 1.7 to 3.6), receipt of care from a practice serving more than 500 HIV-infected patients during the two-month sampling period (odds ratio for the comparison with a practice serving 10 or fewer HIV-infected patients, 2.9; 95 percent confidence interval, 1.4 to 5.9), and use of protease inhibitors or non-nucleoside reverse-transcriptase inhibitors (odds ratio for the comparison with nonuse, 2.2; 95 percent confidence interval, 1.7 to 2.7).

Figure 2 shows adjusted monthly expenditures for subgroups of patients in 1998. Unadjusted and adjusted data are in the Appendix (available only with the electronic version of the article). Total expenditures were lower in 1998 than in 1996 for all subgroups, but they varied greatly among the subgroups in both years. In late 1998, expenditures were about three times as high for patients with CD4 counts of less than 50 per cubic millimeter as for those with CD4 counts of more than 500 per cubic millimeter (P<0.001). Expenditures were also higher for patients with any insurance coverage than for uninsured patients (P<0.001), for patients with public insurance coverage than for those with private insurance coverage (P=0.004), for patients living in the Northeast than for those in other regions (P=0.009), and for patients seen in practices serving 10 or fewer HIV-infected patients during the two-month sampling period than for those in practices with larger numbers of HIV-infected patients (P<0.001).


View larger version (35K):
[in this window]
[in a new window]
 
Figure 2. Monthly Expenditures among Subgroups of HIV-Infected American Adults in 1998, Adjusted for Loss to Follow-up.

IV denotes intravenous, and MSM men who had sex with men. The size of the HIV practice refers to the number of HIV-infected patients seen during the sampling period.

 
The proportion of expenditures attributable to drugs was markedly higher for men than for women (P<0.001), for men who had sex with men than for other risk groups (P=0.003), and for whites than for blacks (P<0.001) or Hispanics (P=0.004) (Figure 3). Expenditures for drugs were also higher for patients with private insurance or Medicare coverage than for those with Medicaid or no insurance coverage (P<0.001 for all comparisons), for patients living in the West than for those living in the South (P= 0.05) or the Northeast (P=0.003), and for patients with more education than for those with less (P< 0.001 for high-school graduates vs. nongraduates, and P=0.004 for patients with some college education vs. patients with none) (Figure 3). The groups with higher expenditures for drugs had relatively lower hospital costs and, with few exceptions, lower overall expenditures (Figure 2 and Figure 3).


View larger version (34K):
[in this window]
[in a new window]
 
Figure 3. Components of Care Shown as Proportions of Total Expenditures among Subgroups of Patients in 1998, Adjusted for Loss to Follow-up and Death.

Dark gray bars represent drugs, white bars inpatient services, and light gray bars outpatient services. MSM denotes men who had sex with men, and IV intravenous.

 
Discussion

Our study began in January 1996, just as the licensure of the first protease inhibitor began to transform chemotherapy for HIV infection. Our estimate of the average direct expenditure for an American adult receiving regular care for HIV infection that month was consistent with estimates in other studies, at about $1,800.1,2,3,4,5,6,8,11 Eighteen months later, highly active antiretroviral therapy was in wide use, the clinical status of many patients had improved, and expenditures for clinical care had declined. Overall expenditures declined by 16 percent, and expenditures for hospital care, then the largest category of expense, declined by 43 percent. Over the same period, pharmaceutical expenditures increased by 33 percent. However, use of the expensive, newer therapies at base line was one of the few independent predictors of a decrease in total expenditures for individual patients. Eighteen months later (36 months after the licensure of protease inhibitors), total expenditures had not increased appreciably, but hospital expenditures had risen to the mid-1996 level. This trend warrants ongoing observation. If it is sustained, the increased use of hospital services probably reflects the aggregate effect of treatment failures due to viral resistance or drug intolerance. This possibility argues for further investments in treatment research and stands as a warning that the service capacity lost with the closing of HIV wards might be needed again.

On the basis of our estimate that the average annual cost of HIV-related care was $20,300 per American adult in 1996 and our previous estimate that 335,000 adults were seen for HIV-related care at least every six months during that year, the total direct expenditure for this population was about $6.8 billion in 1996. This figure is consistent with but, as might be expected, lower than recent payment-based estimates of the costs of all HIV care received by all patients in 1996.18

Resource use declined as the use of highly active antiretroviral therapy increased, so that the adjusted overall expenditure in 1998 was 90 percent of the expenditure in 1996. This value reflects changes in the content of HIV-related care and may not exactly reflect changes in the amount of funds that actually changed hands. However, we are able to estimate the upper and lower limits of the latter value. Retaining the adjustment for death and allowing for a decline in CD4 counts over the course of the study, we calculated that a reasonable upper limit of the expenditure in 1998 is 95 percent of the figure for 1996. Similarly, examination of the values that have not been adjusted for death places the lower limit at 83 percent of the expenditure in 1996.

The decline in annual cost demonstrated here is unlikely to result in a decline in the total lifetime cost, because the decline in cost per unit of time is proportionately less than the expected increase in life expectancy from the use of highly active antiretroviral therapy.19,20 However, the savings do ensure that the cost-effectiveness ratio, or the ratio of the increase in lifetime costs to the increase in life expectancy, is reasonable. Simple calculations based on our data indicate that improvements in life expectancy over a wide range — for example, 50 to 1000 percent, or from 3 to 4.5 or 30 years — would cost approximately $11,000 to $18,000 per year of life saved, a finding that is consistent with the results of other studies.21 Of course, this will hold true only while highly active antiretroviral therapy remains virologically effective, with acceptable toxic effects in most patients, which may not hold true over a long period.

Not all groups benefited equally from the effects of highly active antiretroviral therapy. By the end of 1998, hospital care was still the largest component of expenditures for disadvantaged groups such as women, blacks, patients with public health insurance, and those who had not completed high school. This pattern of less outpatient care and more in-hospital care among disadvantaged groups is consistent with their documented inferior access to high-quality care. Our analyses indicate that unequal access has economic as well as clinical consequences.17,22

The experience of providers also affects expenditures, which were 50 percent higher for patients who received all their care from providers whose practices included few HIV-infected patients. One cannot conclude that concentrating care in larger practices through the transfer of patients or the comanagement of their care reduces costs, because good decision making must take practical factors, system effects, and other indirect costs into account. However, the possibility of a savings and the desirability of reduced hospitalizations do add to this complex debate, and to calls for a better understanding of the effects of provider and practice characteristics on expenditures.23

The most important limitations of our study were our reliance on a closed panel and on self-reports of care received and our use of fixed prices, which do not reflect variations in the intensity of services or prices among providers or over time. These limitations are not severe. We used fixed prices representing actual average reimbursements for a subgroup of study participants as a relative-value scale, which is the only way to make comparisons among systems. The literature — including some HIV-specific data — indicates that self-reports of use are accurate for making overall estimates.24

Our sample was representative of a defined national population, included diverse providers and patients, provided for long-term follow-up information, and allowed for the simulation of repeated cross-sectional data as a means of adjusting the data for changes in factors other than the content of HIV-related care. These adjustments provide unbiased estimates, but our central findings do not depend on the adjustments. The trends over time and across subgroups were also reflected in the unadjusted values and in subgroups such as patients who completed all four interviews.

An important implication of this study is that investments in the development and provision of new treatments for HIV-infected patients have had net economic as well as clinical returns, even without a consideration of their indirect effects such as a return to work. However, as we discuss in more detail elsewhere, increasing insurance benefits to accommodate increased expenditures for drugs can have complex effects, especially on public programs.25 Newer treatments can be very expensive for programs that must increase their drug expenditures but that cannot benefit from the decreased hospital expenditures. Examples of such programs are the AIDS Drug Assistance Program and Medicaid programs that cover drugs but not hospital care for the 90 percent of HIV-infected Medicare beneficiaries who also have Medicaid coverage.

In summary, our long-term data on expenditures for the care of a nationally representative cohort of HIV-infected patients demonstrate worrisome differences in the amount and pattern of expenditures among subgroups of patients, but also indicate that improvements in antiviral treatment have been accompanied by marked declines in total use of hospital services for HIV care. Although our results do not provide justification for the high costs of highly active antiretroviral therapy, they indicate that the improved treatment of HIV infection has had substantial economic returns. Our findings support the view that decisions about the use of new treatments should not be based solely on the costs of acquiring these treatments.26

View this table:
[in this window]
[in a new window]
 
Monthly Expenditures for the Care of HIV-Infected Patients.

 

Supported by a cooperative agreement (HS08578) between RAND and the Agency for Healthcare Research and Quality, with additional funding from the Health Services Resources Administration, the National Institute for Mental Health, the National Institute on Drug Abuse, and the National Institutes of Health Office of Research on Minority Health through the National Institute for Dental Research, and by a grant from the National Institute for Child Health and Human Development (R01-HD35040). Dr. Bozzette was a Health Services Research and Development Senior Research Associate with the Department of Veterans Affairs during the conduct of this study.


Source Information

From RAND Health, Santa Monica (S.A.B., G.J., D.F.M., A.A.L., S.C.M., S.H.B., A.R., M.F.S., D.P.G.); the University of California, San Diego (S.A.B., D.T.); the University of California, Los Angeles (A.A.L., M.F.S.); and the Veterans Affairs San Diego Healthcare System (S.A.B., D.T.) — all in California.

Address reprint requests to Dr. Bozzette at RAND, 1700 Main St., Santa Monica, CA 90407-2318, or at sbozzette{at}ucsd.edu.

References

  1. Sisk JE. The costs of AIDS: a review of the estimates. Health Aff (Millwood) 1987;6:5-21. [Medline]
  2. Lord LJ, Carey J, Work CP, Goode EE. The staggering price of AIDS. US News & World Report. June 15, 1987:16-8.
  3. Scitovsky AA, Rice DP. Estimates of the direct and indirect costs of acquired immunodeficiency syndrome in the United States, 1985, 1986, and 1991. Public Health Rep 1987;102:5-17. [Web of Science][Medline]
  4. Hay JW, Osmond DH, Jacobson MA. Projecting the medical costs of AIDS and ARC in the United States. J Acquir Immune Defic Syndr 1988;1:466-485.
  5. Scitovsky AA, Cline MW, Abrams DI. Effects of the use of AZT on the medical care costs of persons with AIDS in the first 12 months. J Acquir Immune Defic Syndr 1990;3:904-912.
  6. Hellinger FJ. The lifetime cost of treating a person with HIV. JAMA 1993;270:474-478. [Free Full Text]
  7. Newhouse JP. An iconoclastic view of health cost containment. Health Aff (Millwood) 1993;12:Suppl:152-171. [Abstract]
  8. Gebo KA, Chaisson RE, Folkemer JG, Bartlett JG, Moore RD. Costs of HIV medical care in the era of highly active antiretroviral therapy. AIDS 1999;13:963-969. [CrossRef][Web of Science][Medline]
  9. Keiser P, Kvanli MB, Turner D, et al. Protease inhibitor-based therapy is associated with decreased HIV-related health care costs in men treated at a Veterans Administration hospital. J Acquir Immune Defic Syndr Hum Retrovirol 1999;20:28-33. [Medline]
  10. Rabeneck L, Menke T, Wray NP. How good are US studies of HIV costs of care? Med Care 1999;37:748-759. [Medline]
  11. Bozzette SA, Berry SH, Duan N, et al. The care of HIV-infected adults in the United States. N Engl J Med 1998;339:1897-1904. [Free Full Text]
  12. Frankel MR, Shapiro MF, Duan N, et al. National probability samples in studies of low-prevalence diseases. II. Designing and implementing the HIV Cost and Services Utilization Study sample. Health Serv Res 1999;34:969-992.
  13. Lam NSN, Liu KB. Use of space-filling curves in generating a national rural sampling frame for HIV/AIDS research. Prof Geogr 1996;48:321-32.
  14. Berry SH, Brown JA, Athey L, et al. HCSUC Baseline Patient Questionnaire documentation. Santa Monica, Calif.: RAND (in press).
  15. Kalton G. Compensating for missing survey data. Ann Arbor: Survey Research Center, Institute for Social Research, University of Michigan, 1983.
  16. Kish L, Frankel MR. Inference from complex samples. J R Stat Soc [B] 1974;36:1-37.
  17. Shapiro MF, Morton SC, McCaffrey DF, et al. Variations in the care of HIV-infected adults in the United States: results from the HIV Cost and Services Utilization Study. JAMA 1999;281:2305-2315. [Free Full Text]
  18. Hellinger FJ, Fleishman JA. Estimating the national cost of treating people with HIV disease: patient, payer, and provider data. J Acquir Immune Defic Syndr 2000;24:182-188.
  19. Palella FJ Jr, Delaney KM, Moorman AC, et al. Declining morbidity and mortality among patients with advanced human immunodeficiency virus infection. N Engl J Med 1998;338:853-860. [Free Full Text]
  20. Schwarcz SK, Hsu LC, Vittinghoff E, Katz MH. Impact of protease inhibitors and other antiretroviral treatments on acquired immunodeficiency syndrome survival in San Francisco, California, 1987-1996. Am J Epidemiol 2000;152:178-185. [Free Full Text]
  21. Moore RD. Cost effectiveness of combination HIV therapy: 3 years later. Pharmacoeconomics 2000;17:325-330. [CrossRef][Web of Science][Medline]
  22. Cunningham WE, Markson LE, Andersen RM, et al. Prevalence and predictors of highly active antiretroviral therapy use in patients with HIV infection in the United States. J Acquir Immune Defic Syndr 2000;25:115-123.
  23. Landon BE, Wilson IB, Shapiro M, Bozzette S, Cleary P. Providers of HIV care. Presented at the 16th Annual Meeting of the Association for Health Services Research, Chicago, June 27–29, 1999:135. abstract.
  24. Weissman JS, Levin K, Chasan-Taber S, Massagli MP, Seage GR III, Scampini L. The validity of self-reported health-care utilization by AIDS patients. AIDS 1996;10:775-783. [Web of Science][Medline]
  25. Goldman DP, Bhattacharya J, Leibowitz AA, Joyce GF, Shapiro MF, Bozzette SA. The impact of state policy on the costs of HIV infection. Med Care Res Rev 2001;58:31-54.
  26. Murphy S. Does new technology increase or decrease health care costs? The treatment of peptic ulceration. J Health Serv Res Policy 1998;3:215-218. [Medline]
Appendix

Appendix.


 

This Article
-Abstract
- PDF

Commentary
-Editorial
 by Steinbrook, R.

Tools and Services
-Add to Personal Archive
-Add to Citation Manager
-Notify a Friend
-E-mail When Cited

More Information
-PubMed Citation

This article has been cited by other articles:



HOME  |  SUBSCRIBE  |  SEARCH  |  CURRENT ISSUE  |  PAST ISSUES  |  COLLECTIONS  |  PRIVACY  |  TERMS OF USE  |  HELP  |  beta.nejm.org

Comments and questions? Please contact us.

The New England Journal of Medicine is owned, published, and copyrighted © 2009 Massachusetts Medical Society. All rights reserved.