|
|
| |||||||||||||||||||||||||||||||||||||||||||||
Background The news media are an important source of information about new medical treatments, but there is concern that some coverage may be inaccurate and overly enthusiastic.
Methods We studied coverage by U.S. news media of the benefits and risks of three medications that are used to prevent major diseases. The medications were pravastatin, a cholesterol-lowering drug for the prevention of cardiovascular disease; alendronate, a bisphosphonate for the treatment and prevention of osteoporosis; and aspirin, which is used for the prevention of cardiovascular disease. We analyzed a systematic probability sample of 180 newspaper articles (60 for each drug) and 27 television reports that appeared between 1994 and 1998.
Results Of the 207 stories, 83 (40 percent) did not report benefits quantitatively. Of the 124 that did, 103 (83 percent) reported relative benefits only, 3 (2 percent) absolute benefits only, and 18 (15 percent) both absolute and relative benefits. Of the 207 stories, 98 (47 percent) mentioned potential harm to patients, and only 63 (30 percent) mentioned costs. Of the 170 stories citing an expert or a scientific study, 85 (50 percent) cited at least one expert or study with a financial tie to a manufacturer of the drug that had been disclosed in the scientific literature. These ties were disclosed in only 33 (39 percent) of the 85 stories.
Conclusions News-media stories about medications may include inadequate or incomplete information about the benefits, risks, and costs of the drugs as well as the financial ties between study groups or experts and pharmaceutical manufacturers.
Physicians, consumers, and third-party payers may be more enthusiastic about long-term preventive treatments when benefits are stated as relative, rather than absolute, reductions in the risk of adverse events.18,19,20,21,22 Medical-journal editors have said that reporting only relative reductions in risk is usually inadequate in scientific articles and have urged the news media to consider the importance of discussing both absolute and relative risks.3,23 For example, a story reporting that in patients with myocardial infarction, a new drug reduces the mortality rate at two years from 10 percent to 7 percent may help patients weigh both the 3 percent absolute and the 30 percent relative reduction in risk against the costs of the drug and its side effects.
Other issues are whether news stories cover potential adverse effects as well as benefits24 and whether stories report on the ties of cited experts or study groups to industry.25 Because evidence suggests that commercial funding may sometimes be associated with study outcomes that are more favorable to sponsors' products, disclosure of such funding in media stories, as is done in a number of major peer-reviewed journals, may be desirable.26,27,28
We studied news stories about three medications that are used for the prevention of major diseases. We examined whether benefits were stated in relative or absolute terms, whether potential harm and costs were discussed, and whether ties of cited experts or study groups to industry were included.
Methods
Study Drugs
We studied coverage of three medications with important preventive benefits and potentially large markets that have attracted considerable media attention: two relatively new patented drugs (pravastatin [Pravachol, Bristol-Myers Squibb], a cholesterol-lowering drug for the prevention of cardiovascular disease that was released in 1991, and alendronate [Fosamax, Merck], a bisphosphonate for the treatment and prevention of osteoporosis that was released in 1995) and an older, off-patent drug (aspirin for the prevention of cardiovascular disease).
We focused on preventive medications, since there is evidence (particularly for cholesterol-lowering drugs) that when event rates are low for untreated persons and treatment is long-term, the way treatment benefits are presented can affect the enthusiasm of patients and clinicians for therapy.18 Gastrointestinal side effects can occur after the use of aspirin and alendronate29,30; pravastatin is considered to have a relatively low rate of adverse effects, although it may share the tendency of other statins to induce liver and muscle abnormalities in rare cases.31
Selection of News Stories
We searched the Lexis–Nexis data base to obtain stories from 36 U.S. newspapers, including both large-circulation national papers and regional papers, between 1994 and 1998. The key words used in the search strategy were "osteoporosis" and additional terms "alendronate or Fosamax" for alendronate; "cholesterol" and additional terms "pravastatin or Pravachol" for pravastatin; and "heart" and "aspirin" for aspirin. We identified 196 candidate stories on alendronate, 119 on pravastatin, and 275 on aspirin.
A systematic probability sample of 60 newspaper stories for each drug32 was obtained from the complete list of stories on each drug, ordered according to date and divided into 60 equal-sized blocks. Using a random starting point in each block, we reviewed the stories for content, excluding each story that met the exclusion criteria listed below and substituting the next story from the block. In this way, 401 newspaper stories were reviewed and 180 of them were included in the study.
The Vanderbilt Television News Archive Evening News data base was used to obtain videotaped stories on ABC, CBS, and NBC nightly network news and CNN about the three drugs between 1994 and 1998. Search strategies similar to those used for newspaper articles were employed. Ninety television stories were identified and reviewed, of which 27 were included (10 on alendronate, 10 on pravastatin, and 7 on aspirin).
Newspaper and television stories were excluded if they emphasized another topic, with only a brief mention of the study drug (146 stories); were in a "question and answer" or "Dear Doctor" format (37 stories); dealt solely with business issues (17 stories); concerned indications other than those listed (12 stories); were letters to the editor or corrections, or lacked sufficient information (17 stories); or did not cover the study drug at all (55 stories; e.g., stories on "super aspirin" referring to other antiplatelet drugs). Stories in different newspapers that were based on the same wire-service report accounted for 14 percent of the sample.
Measures
We used an abstraction form to collect information from the stories. The form consisted largely of simple dichotomous (yes or no) items so as to limit subjective judgments by coders. The key items extracted were whether benefits were reported quantitatively; whether benefits were stated in relative terms (e.g., a "halving" of the risk of fracture) or absolute terms (e.g., a reduction in the rate of fracture by 1 percent or the need to treat 100 people to prevent 1 fracture); whether the presence or absence of adverse effects (potential harm) was mentioned; and whether costs were mentioned. Stories that gave the event rates in treatment and control groups, permitting both absolute and relative changes to be derived, were categorized as reporting both absolute and relative benefits (e.g., a reduction in the risk of nonfatal heart attack from 6 percent to 4 percent). The coders also listed the experts and studies cited in each story.
We then searched Medline (for the years 1991 through 1999) for the published scientific literature referred to in the media stories and for other studies by the experts cited (identified by a search for names). By reading the retrieved articles, including the acknowledgments, we determined whether the experts had financial ties to manufacturers and whether the studies had been supported primarily by the manufacturer of the drug that was the focus of the story. We then determined whether the news stories included any information about the financial ties disclosed in the scientific literature.
Data Collection
Newspapers' names and bylines were removed from printed copies of the Lexis–Nexis stories. Masked versions were coded independently by two coders trained in the use of the form, working according to an instruction manual, and blinded to the results until all data collection was complete. (Full-text versions of articles in the Wall Street Journal and the Miami Herald were not available through Lexis–Nexis. They were obtained from a library and could not be masked.) Television stories were transcribed by an independent transcription service; the coders reviewed both the videotapes and the written transcripts, using the same form. All disagreements were resolved by two of the authors independently of the coders. The rates of agreement between coders on dichotomous variables were generally high: whether benefits were quantified in the story (95 percent observed agreement, kappa=0.89), whether relative benefits were given (95 percent observed agreement, kappa=0.90), and mention of costs (94 percent observed agreement, kappa=0.87) and potential harm (96 percent observed agreement, kappa=0.90). Decisions were more difficult in the few instances in which absolute benefit was reported; interrater agreement was 95 percent, but the kappa statistic was lower (0.59).
Statistical Analysis
We derived proportions of reports and their exact binomial 95 percent confidence intervals for each of the outcomes.32 These estimates are based on the assumption that each report was independent (i.e., there was no correlation between stories from the same media outlet). We attempted to adjust for possible effects of clustering by using a variance-inflation factor based on the average cluster size and intracluster correlation.33 We found that the intracluster correlation was negligible for most subgroups and for most outcomes. Hence, we present only confidence intervals derived from this approach when the variance-inflation factor altered the 95 percent confidence interval. To test whether there were significant differences in reporting styles among different media, chi-square statistics and odds ratios and their confidence intervals were computed by a generalized estimating approach.34 All P values are two-tailed.
Results
The characteristics of the stories are described in Table 1. Two hundred seven stories released by 40 media outlets (36 newspapers and 4 television networks) were included. Of all stories, 27 (13 percent) were reported by the television networks, 53 (26 percent) by leading national newspapers, and 127 (61 percent) by other newspapers. The stories were well distributed with respect to drugs, years, and regions.
|
Eighty-three of the 207 stories (40 percent) did not report benefits quantitatively (Table 2). Of the 124 stories that quantified benefits, 103 (83 percent) reported only relative benefits, 18 (15 percent) reported both absolute and relative benefits, and 3 (2 percent) reported only absolute benefits. All three stories reporting only absolute benefits were about aspirin.
|
Of the 207 stories, 98 (47 percent) mentioned potential harm, and only 63 (30 percent) mentioned the costs of therapy (Table 2). The likelihood that a story would mention possible adverse effects or costs was not affected by whether it appeared on television, in a leading national newspaper, or in another newspaper (P=0.93 for adverse effects and P=0.51 for costs) (Table 3).
|
Of the 170 stories citing an expert or a scientific study, 85 (50 percent) cited at least one with an industrial tie disclosed in the scientific literature (Table 2); in 33 of these 85 stories (39 percent), the tie with industry was mentioned. The majority of these stories cited both experts and study groups with ties.
An Example
An example may help to illustrate our findings. On the evening of May 22, 1996, ABC, NBC, and CBS television news broadcast stories about alendronate,35,36,37 sparked by a conference at which the results of an important randomized, controlled trial were reported. All three stories gave only the relative reduction in risk, stating that the new osteoporosis drug could reduce the incidence of hip fractures by 50 percent, or one half. The CBS reporter described these results as "almost miraculous."37 None of the stories cited actual event rates in treated patients (1 percent) and untreated patients (2 percent); only one mentioned gastrointestinal distress as a potential adverse effect37; and no story disclosed that the study investigator being interviewed had received funding for the study from the drug manufacturer.
Discussion
Our evaluation of 207 newspaper and television stories on three drugs used for disease prevention showed substantial shortcomings in journalistic practices. Of stories quantifying the benefits of medications, only 15 percent presented both relative and absolute benefits. Eighty-three percent presented information on benefits in relative terms only — an approach that has been shown to increase the enthusiasm of doctors and patients for long-term preventive treatments and that could be viewed as potentially misleading.18,20,38
In general, giving only the absolute or only the relative benefits does not tell the full story; it is more informative if both researchers and the media make data available in both absolute and relative terms. For individual decisions about long-term preventive therapies, consumers need information to weigh the probability of benefit and harm; in such cases it seems desirable for media stories to include actual event rates with and without treatment.
In the case of public health interventions, such as vaccination, the use of seat belts, or prevention of Reye's syndrome by avoidance of aspirin use, it is difficult to impart effective messages by reporting only on absolute reductions in risk, which would tend to minimize important population-wide benefits. In such cases, media reports might emphasize the relative benefits.
Fifty-three percent of the stories in our study did not include information about potential harms, which is a matter for concern, given the study drugs' associations with a range of adverse effects.29,30,31 The finding that 70 percent of stories made no mention of cost is also important, since cost effectiveness is increasingly considered an important factor in medical advances. For the two prescription drugs, a majority of the stories citing a study group or an expert with a link to the drug manufacturer failed to mention that link, despite the current emphasis on disclosure of such links in the scientific literature.39
There are several important limitations to the generalizability of our findings. First, these study drugs are not a representative sample. All three can be used as preventive therapies, for which benefits are readily framed in relative or absolute terms. For other therapies and conditions (e.g., Alzheimer's disease), more complex approaches would be required for reliable measurement of media coverage of benefits.40
Second, the coverage of the three study drugs was overwhelmingly positive and focused mainly on benefits, thus limiting the relevance of our findings to negative coverage, in which the risk of harm becomes the focus. For example, in 1995 several media stories about calcium-channel blockers emphasized the increased relative risk of heart attack among patients treated with these drugs, rather than the much smaller change in the absolute risk.41,42
Third, our results probably underestimate the extent of ties with industry, because we relied on disclosure in the scientific literature, where such ties have been found to be underreported.39 Finally, we analyzed only the textual content of news stories, omitting the important features of placement, illustration, and length.
The appropriate role of the news media in reporting on medical advances requires more focused attention from both researchers and the media. Some may see the role of media stories as primarily to alert the public to medical advances, which can be further investigated by consumers together with their physicians. Others would favor more complete media reporting of salient aspects of scientific studies, including disclosure of ties with industry. An effective educational program or resource kit for journalists and editors, focusing on the reporting and interpretation of clinical findings, might be timely.3,11,43
Rather than prescribing or proscribing specific behavior on the part of the media, we believe it may be valuable to articulate basic principles of high-quality medical reporting, in line with an evidence-based approach to medicine. When reporting on new forms of technology or new treatments, journalists and editors might consider the evidence available in relation to the following questions: What is the magnitude of the benefit (e.g., both absolute and relative), and what groups of patients can be helped? What are the associated risks and costs? What are the possible links between the sources of information (studies or experts) and those (such as the manufacturers) who promote the therapy? Although not exhaustive, these questions could help inform attempts to improve the quality of medical reporting.
Supported by the Commonwealth Fund and the Harvard Pilgrim Health Care Foundation. The views expressed in this article are those of the authors and not necessarily those of the Commonwealth Fund or its director, officers, or staff. The work was conducted while Mr. Moynihan was a 1998–1999 Harkness Fellow in Health Care Policy.
We are indebted to Sumit Majumdar, Irene Weilawski, and Mary Seddon for reviewing the manuscript; to Maureen Connelly, Huw Davies, Robert Fletcher, Sue Hill, and Ann Payson for assistance in developing the data-collection instrument; and to Stevan de Prosse, Garnet Harris, Leighton Farquharson, Vinh Do, Sue John, Robert Deveau, Dianne O'Connell, Jayne Fryer, Robert LeCates, and Suzy Conway for technical assistance.
Source Information
From the Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston (R.M., D.R.-D., C.M., S.B.S.); the Australian Broadcasting Corporation, Sydney, N.S.W., Australia (R.M.); the Department of Clinical Pharmacy, School of Pharmacy (L.B., K.L.), and Institute for Health Policy Studies, School of Medicine (L.B.), University of California, San Francisco; and the School of Population Health Sciences, Faculty of Medicine and Health Sciences, University of Newcastle, Newcastle, N.S.W., Australia (D.H.).
Address reprint requests to Dr. Soumerai at the Department of Ambulatory Care and Prevention, Harvard Medical School, 126 Brookline Ave., Suite 200, Boston, MA 02215, or at ssoumerai{at}hms.harvard.edu.
References
| |||||||||||||||||||||||||||||||||||||||||||||
This article has been cited by other articles:
HOME | SUBSCRIBE | SEARCH | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | PRIVACY | HELP | beta.nejm.org Comments and questions? Please contact us. The New England Journal of Medicine is owned, published, and copyrighted © 2008 Massachusetts Medical Society. All rights reserved. |
Previous
