|
|
| |||||||||||||||||||||||||||||||||||||||||||||
Background Institutional review boards (IRBs) are charged with safeguarding potential research subjects with limited literacy but may have an inadvertent role in promulgating unreadable consent forms. We hypothesized that text provided by IRBs in informed-consent forms falls short of the IRBs' own readability standards and that readability is influenced by the level of research activity, local literacy rates, and federal oversight.
Methods To test these hypotheses, we conducted a cross-sectional study linking data from several public-use sources. A total of 114 Web sites of U.S. medical schools were surveyed for IRB readability standards and informed-consent-form templates. Actual readability was measured with the Flesch–Kincaid scale, which assigns a score on the basis of the minimal grade level required to read and understand English text (range, 0 to 12). Data on the level of research activity, local literacy rates, and federal oversight were obtained from organizational Web sites.
Results The average readability score for text provided by IRBs was 10.6 (95 percent confidence interval, 10.3 to 10.8) on the Flesch–Kincaid scale. Specific readability standards, found on 61 Web sites (54 percent), ranged from a 5th-grade reading level to a 10th-grade reading level. The mean Flesch–Kincaid scores for the readability of sample text provided by IRBs exceeded the stated standard by 2.8 grade levels (95 percent confidence interval, 2.4 to 3.2; P<0.001). Readability was not associated with either the level of research funding (P=0.89) or local rates of literacy (P=0.92). However, the 52 schools that had been made subject to oversight by the Office for Human Research Protections (46 percent) had lower Flesch–Kincaid scores than the other schools (10.2 vs. 10.9, P=0.005).
Conclusions IRBs commonly provide text for informed-consent forms that falls short of their own readability standards. Federal oversight is associated with better readability.
Obtaining informed consent for participation in medical research is particularly challenging because it requires a level of comprehension beyond that required for consent to usual care.13 A large literature already supports the notion that the language used in informed-consent forms is not comprehensible to most Americans.14 However, the origin of this problem is unclear.
Institutional review boards (IRBs) are charged with safeguarding potential subjects with limited literacy,15,16 but they may have an inadvertent role in promulgating unreadable informed-consent forms. IRBs commonly present investigators with readability standards and informed-consent text, in the form of templates and sample forms. We surveyed U.S. medical schools to test the hypothesis that IRBs fail to meet their own standards for readability in the informed-consent text offered to investigators.
Methods
Data Sources
We obtained all data from publicly available Web sites, which were accessed between December 2001 and February 2002. A total of 123 U.S. medical-school Web sites were surveyed for specified readability standards and suggested text for informed-consent documents. Data were obtained from 114 of the Web sites (93 percent). We were unable to collect data from the other nine Web sites because information about the IRB was restricted to intranet access (six sites) or because the Web site contained no information about an IRB (three).
Data on the rank order of medical schools that received funding in 2001, obtained from the Web site of the National Institutes of Health,17 were used as an indicator of research activity. Local literacy rates were obtained from the Web site of the National Institute for Literacy, which provides literacy rates by congressional district.18 Data on federal oversight were obtained from the Web site of the Office for Human Research Protections, which lists medical schools that were made subject to oversight between July 2000 and February 2002.19 Federal oversight is initiated to evaluate indications of noncompliance with federal policy.20
Readability Standards
Each medical-school Web site was examined for information about readability standards. Descriptive standards (e.g., "in lay language") and specific grade-level standards (e.g., "at or below an 8th-grade reading level") were recorded.
Sample Text
Web sites were examined for sample informed-consent text to be used for adults who have full decision-making capacity. Text intended for special populations (e.g., children) or special circumstances (e.g., genetic testing) was excluded from the analysis. Applicable templates and sample language were downloaded into Microsoft Word 2000 (Microsoft).
Readability Score
The readability of sample text was measured with the use of the Flesch–Kincaid readability scale (grade-level range, 0 to 12), which is automated in Microsoft Word and has been demonstrated to be reliable and valid.21 The Flesch–Kincaid scale assesses readability on the basis of the average number of syllables per word and the average number of words per sentence.22 The Flesch–Kincaid grade level that we assigned to a document was the average of three separate measurements conducted on the three largest uninterrupted blocks of text (Table 1).
|
Statistical Analysis
We used the Wilcoxon signed-rank test to compare readability scores with grade-level standards for readability. Analysis of variance was used to test the association of readability with quartile means for the level of research funding and local literacy rates. Readability scores for schools that had been subjected to federal oversight and those that had not were compared with use of the Wilcoxon rank-sum test. All significance tests were two-tailed. Analyses were conducted with Stata software, version 8 (Stata).
Results
Readability Standards
Grade-level readability standards were specified at 61 of the 114 Web sites (54 percent). The standards ranged from a 5th-grade reading level to a 10th-grade level (mode, 8th grade). Forty-seven other Web sites (41 percent) contained descriptive guidelines such as "simple lay language," and six (5 percent) contained no guidelines for readability.
Readability of Sample Text
The mean Flesch–Kincaid grade level for sample text supplied by IRBs was 10.6 (95 percent confidence interval, 10.3 to 10.8). Among the 61 schools with specific grade-level standards, only 8 percent (95 percent confidence interval, 3 to 18 percent) met their own standards; the mean score for readability exceeded the stated standard by 2.8 grade levels (95 percent confidence interval, 2.4 to 3.2; P<0.001) (Figure 1).
|
Several schools presented sample text for informed-consent forms that did meet their own standards. For example, the text supplied by the State University of New York Downstate Medical School, which specifies a 6th-grade reading level as the standard for readability, had a Flesch–Kincaid score of 5.8 and a Fry score of 6. The University of Washington and the schools of the University of Minnesota met their goal of an 8th-grade reading level.
Factors Thought to Influence Readability
Neither the level of research funding received from the National Institutes of Health nor the local rate of literacy was associated with the readability of informed-consent text (P=0.89 and P=0.92, respectively). However, there was a small but significant association between federal oversight and readability. The grade-level score was lower for text provided by the 52 medical schools that had been subjected to federal oversight than for text presented by the other schools (10.2 vs. 10.9, P=0.005). This association was even stronger when the oversight predated the posting of the informed-consent text (readability score, 9.8 vs. 10.9; P<0.001).
Discussion
Our findings suggest that the sample texts provided to investigators by IRBs of U.S. medical schools generally fail to meet the IRBs' own standards for readability. The readability of sample text does not appear to be influenced by the level of research activity or the local rate of literacy but does appear to be influenced by prior oversight by the Office for Human Research Protections. Previous reports have demonstrated the complexity of informed-consent forms23,24,25,26,27,28,29,30,31,32 but have not examined the language promulgated by IRBs.
The strengths of this study that lend weight to our conclusions are its nearly complete coverage of IRBs at U.S. medical schools and the use of standardized instruments for data abstraction. Nonetheless, several limitations should be kept in mind. First, data were obtained exclusively from Web sites. Although it is likely that the materials presented on IRB Web sites accurately reflect local practices, additional materials were not examined. Second, our analysis was limited to templates and samples rather than actual consent forms. Investigators sometimes modify the language in templates to make the forms more readable. Third, we did not attempt to evaluate content. It is possible that variations in conceptual complexity influence readability as well. Fourth, the grade level of text is only one aspect of the suitability of informed-consent forms. Additional factors, such as the type font, layout, and length, also affect readability.27,33,34,35,36,37
We chose the Flesch–Kincaid scale for our main analysis primarily because of its convenience: of the dozens of readability scales, the Flesch–Kincaid system is the most widely available for computerized use, since it is embedded in Microsoft Word. Other advantages include its wide use in studies of readability, excellent repeatability, and high correlation with other established readability scales (r=0.87 to 0.90).21 However, since the Flesch–Kincaid scoring method in Microsoft Word artificially truncates readability at the 12th-grade level, it underestimates the actual reading level required for complex text. Had we applied the more laborious Fry method in our main analyses, the disparities between readability standards and sample forms would have been even more striking. Furthermore, neither method accounts for the complexity introduced by short but unfamiliar medical terms. Both these limitations tend to result in underestimation of the actual reading level required for medical documents.
Almost half of American adults read at or below the 8th-grade level.3 Illiteracy in the United States is mainly the result of insufficient education. The text of informed-consent documents can be written at a 4th-grade level, as Table 1 shows. Even though consent forms are never used in isolation, text written at a 4th-grade level would promote the autonomy of most candidates for participation in medical research. People who have poor literacy skills should not be excluded from such participation. Alternative methods for obtaining informed consent, such as multimedia systems, are also likely to be of substantial benefit.38
To achieve systemic change, IRBs need to improve the templates and sample text they offer to investigators. Although federal review appears to improve the readability of these materials, models are already available for interested medical schools. Institutions as diverse as the National Cancer Institute and the State University of New York Downstate Medical School have presented informed-consent templates and sample forms that are below an 8th-grade reading level.39 Our study suggests that a 4th-grade to 6th-grade reading level may be a suitable target, since text at this level appears to convey key concepts simply and directly.
Important documents for patients must be written in clear, direct language. Plain language requires honesty and a good understanding of what to convey.40 We believe that IRBs should lead by example.
Dr. Paasche-Orlow is the recipient of a grant (T32HL007180) from the National Heart, Lung, and Blood Institute. Dr. Brancati is the recipient of a Mid-Career Investigator Award in Patient-Oriented Research (K24DK62222) from the National Institute of Diabetes and Digestive and Kidney Diseases.
Source Information
From the Departments of Medicine (M.K.P.-O., F.L.B.), Health Policy and Management (H.A.T.), and Epidemiology (F.L.B.) and the Phoebe R. Berman Bioethics Institute (M.K.P.-O., H.A.T.), Johns Hopkins University, Baltimore.
Address reprint requests to Dr. Paasche-Orlow at the Welch Center for Prevention, Epidemiology, and Clinical Research, Johns Hopkins Medical Institutions, 2024 E. Monument St., Suite 2-600, Baltimore, MD 21205, or at mpaasche{at}jhmi.edu.
References
46.109 and 50.20 (1993).
46 (1993).
| |||||||||||||||||||||||||||||||||||||||||||||
Related Letters:
Readability of Informed-Consent Forms
Beyer D. R., Lauer M. S., Davis S., Mayo D. J., Paasche-Orlow M. K., Taylor H. A., Brancati F. L.
Extract |
Full Text |
PDF
N Engl J Med 2003;
348:2262-2263, May 29, 2003.
Correspondence
This article has been cited by other articles:
HOME | SUBSCRIBE | SEARCH | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | PRIVACY | HELP | beta.nejm.org Comments and questions? Please contact us. The New England Journal of Medicine is owned, published, and copyrighted © 2008 Massachusetts Medical Society. All rights reserved. |
Previous
