The section of the Federal Regulations for the Protection of Human Subjects related to informed consent (21 CFR 50.25) stipulates that investigators provide descriptions of “any reasonable foreseeable risks or discomforts” and “any benefits to the subjects or to others which may be reasonably expected from the research.”1 Despite this directive, studies suggest that many adult subjects and parents of child subjects have poor understanding of the information provided.2–4
There may be several reasons for poor understanding, including information written at higher than the recommended eighth-grade reading level, use of unfamiliar medical terms, and incomplete or unclear information.4–6 Another potential impediment is that consent for anesthesia research is typically presented on the day of surgery with only a narrow window of opportunity for disclosure. Tait et al. identified several factors that influence parents' assessments of the risks and benefits of a study, including their perceptions of the clarity and amount of information provided.7 There is also evidence to suggest that the format and framing of the message used to present risks and benefits may affect subject understanding.8,9 For example, descriptive presentation of risk (e.g., the risk of nausea is “low”) is often used but may be difficult to translate into a meaningful numerical value. Similarly, using numerical values alone (e.g., the risk of nausea is 10% or 10 of 100) may pose difficulties for individuals with low numeracy skills.8,9
Recently, there have been some data to suggest that graphical presentation of risks and benefits may improve understanding;10,11 however, there is no consensus regarding the best method of presenting these statistics in the clinical and research setting, particularly to individuals with poor numeracy and literacy skills. This study, therefore, was designed to test the hypothesis that graphical presentation of research risks and benefits to parents of children scheduled for elective surgery would result in greater understanding in comparison with standard text presentations.
This study was approved by the University of Michigan's IRB with a waiver of documentation of informed consent. The study sample included parents (>18 years) of children scheduled for an elective surgical procedure. Parents were approached in the preoperative screening area or preoperative waiting area and, after a standardized verbal disclosure and consent to participate, were randomized to receive consent information for a sham pediatric study in which risk/benefit information was presented in 1 of 3 formats, i.e., text, tables, or pictographs. The choice of pictographs as the primary graphical format was based on previous studies that showed that pictographs were more consistently associated with adequate understanding than were other formats such as pie charts, bar graphs, etc.11,12 The sham study was based on a real protocol conducted previously in our department comparing a standard drug (drug A) with an new drug (drug B) for postoperative pain management in children. Each consent document followed our IRB's standard template for risk/benefit presentation and contained the same basic information but presented in different formats. Parents were informed that we were evaluating the process of giving study information and the quality of our consent documents. Parents were also told that their child would not be participating in an actual study but that they should consider the hypothetical information as if it were real. Although the study represented a sham protocol, it was nevertheless presented to simulate our standard practice for consenting research participants. In this respect, the timing of when the information was given (i.e., on the day of surgery), the time allotted for parents to read the consent information, and the environment in which consent was sought were similar to those in our usual practice.
After an explanation of the study and consent to participate, parents were given time to read the consent form as per our routine practice. Parents were asked whether they had sufficient time to read the document and, if under real circumstances, whether they would have allowed their child to participate. They were then given a questionnaire to determine their gist (essential) and verbatim (actual) understanding of the risks and benefits of the “study.” The gist and verbatim items in the questionnaire were based on a survey previously conducted in our department.12 Additionally, parents were asked their perceptions of the effectiveness of the presentation and their preferences for the way in which risks and benefits should be presented.
Describing and Comparing the Risks and Benefits
Risks (itching, nausea and vomiting, and slowed breathing) and the benefits (pain relief) of the 2 drugs were presented using text, tables, or pictographs. The numbers used to present risks and benefits were based loosely on actual drug data and were derived to emphasize differences between the “study drugs” for the purposes of measuring understanding. In the text version, risks and benefits were described in absolute terms, i.e., the number of children out of 100 (%) experiencing the outcome. In the table format, the same information was presented as is shown in Table 1. The pictograph format included a matrix of 100 figures representing the reference population. The figures were displayed using different colors (blue and gray) to represent the number of individuals with or without the risk or benefit (Fig. 1).
Gist refers to the ability of the subject to understand the essential meaning about the differences between the risks and benefits of the 2 drugs. Five multiple-choice questions were designed to measure gist understanding. For example, parents were asked, “Who is more likely to experience nausea/vomiting (itching, slowed breathing, pain relief): 1) a child who received drug A, 2) a child who received drug B, or 3) they are equally likely.” Gist understanding was measured as the total number of correct responses (0–5).
Verbatim understanding refers to the subjects' ability to correctly report the absolute risk and benefit frequencies of the 2 drugs. Seven items in the questionnaire measured verbatim understanding. Examples include “If 100 children took drug B, approximately how many would experience pain after surgery?” and “Compared to children who took drug A, how many fewer children would experience nausea/vomiting if they took drug B?” Each item required a fill-in-the-blank response, and only correct answers were considered. A total verbatim score was based on the number of correct answers (0–7).
Sociodemographic data were collected, including age (parent's and child's), gender, racial/ethnic background (self-identified), education, surgical procedure, and prior research participation. In addition, parental numeracy and general literacy were measured using 8 items from the Subjective Numeracy Scale13,14 and the shortened version of the Rapid Estimate of Adult Learning in Medicine reading test, respectively.15 Need for cognition, which measures the tendency of an individual to engage in and enjoy effortful thinking, was measured using the shortened version of the Need for Cognition form developed by Cacciopo et al.16,17
Statistical analyses were performed using SPSS® statistical software (v. 16.0, SPSS Inc., Chicago, Illinois). Sample size determination was based on preliminary data describing parents' understanding of risks presented in a standard text format. Accepting a 20% difference in understanding between formats as the smallest clinically important difference, we required a minimal sample size of 134/group (α = 0.05, β = 0.10, 2-tailed, n = 402).
Nonparametric data were analyzed using Kruskal– Wallis, Mann–Whitney U, χ2, and Fisher's Exact tests, as appropriate. Comparisons of parametric data between groups (e.g., age) were analyzed using analysis of variance (ANOVA). Factors determined to be significant by univariate analysis were entered into a multiple regression model to identify predictors of understanding. Data are expressed as percentages, median and interquartile range, and mean ± SD. Significance was accepted as P < 0.05 for single comparisons and P < 0.025 for multiple interformat group comparisons (Bonferroni corrected).
Four hundred forty-four parents of children scheduled for elective surgery were approached to participate, 36 of whom declined. Data are thus presented for 408 subjects. The demographics of the nonconsenters were similar to those who participated, although a larger percentage of nonconsenters were African American, in comparison with those who participated (25% vs. 13.4%, P = 0.059). There were no differences in demographics between each of the format groups (Table 2).
Tables 3 and 4 compare the effect of format on parents' gist and verbatim understanding of the risks and benefits posed by the study. As is shown, both gist and verbatim understanding were significantly greater among parents who received information using either tables or pictographs, in comparison with the standard text presentation. Subgroup analysis also showed that pictographs and, to a greater extent, tables were superior to text in improving gist and verbatim understanding among those with poor literacy and numeracy skills. Indeed, when presented as text, none of the parents with low literacy could answer all the gist questions correctly, and only 1 could correctly answer all the verbatim questions.
Table 5 describes the effect of parental demographics and characteristics on gist and verbatim understanding. Results showed that parents with a college degree had better verbatim understanding of the information, and those with greater literacy and numeracy skills had significantly better gist and verbatim understanding than did those with lower abilities. Multiple regression models identified several factors predictive of gist and verbatim understanding. Predictors of gist understanding included message format (tables or pictographs vs. text, P < 0.001) and high numeracy (P = 0.001). Independent predictors of verbatim understanding included message format (tables vs. text, P < 0.001), high numeracy (P < 0.001), and high literacy (P = 0.01).
Overall, there were no differences between groups with respect to the parents' perceptions of the information presentation (Table 6), although parents who received tables did find them easier in determining the risks and benefits. There were, however, no differences among the text, table, and pictograph groups with respect to how likely parents would have been to allow their child to participate in the study had it been real (5.6 ± 3.2 vs. 5.4 ± 3.2 vs. 5.6 ± 3.1, respectively; 0 to 10 scale in which 10 = extremely likely).
Historically, text has been the standard method for presenting risks and benefits for treatment and research, yet several studies suggest that many parents, patients, and research subjects have poor understanding of these statistics in this format.2–4,7 Recently, there have been some data to suggest that graphical presentation of risks and benefits are better understood and may enhance decision making, particularly among individuals with low numeracy.8,11,18 Indeed, in comparison with standard text, visual depictions of risks and benefits may reveal patterns that otherwise go unnoticed,19 improve the processing of mathematical computation,20,21 and better attract and retain an individual's attention. Other studies also show that graphical formats require less cognitive effort or provide different affective responses to numerical information.22 This may explain why parents with lower numeracy and literacy skills in our study were significantly less able to interpret the risks and benefits when presented as text. Indeed, Peters et al. showed that, whereas numerate individuals are more likely to pay attention to numbers in making health care decisions, innumerate individuals rely less on numbers and more on emotion, mood, and trust or distrust of the physician or medical system.23 The improved visual salience of tables and pictographs may thus be helpful in promoting understanding among those with poor language and numerical skills. This is particularly important given that >90 million individuals are either illiterate or semi-illiterate in the United States,24 and a large portion of the population experience difficulties with everyday mathematical skills such as calculating a tip or discounting a store item.25,26 Development of strategies that will enhance understanding among these vulnerable groups is therefore critical.
Results of this study showed that both pictographs and tables were superior to text in promoting gist and verbatim understanding of risk/benefit information. These results confirm the findings and establish the practical relevance of 2 recent Internet surveys comparing different graphical formats for the presentation of risk/benefit statistics.11,12 In one of these, a survey of parents demonstrated that pictographs were superior to both tables and text in presenting understandable risk/benefit information to both numerate and innumerate individuals.12 However, in the present study, tables were shown to be equally effective as pictographs. The reasons for this are unclear but may simply reflect differences in the populations surveyed (i.e., Internet vs. hospital), differences in the design of the tables and pictographs used in the two studies, or differences in statistical power. In another survey, Hawley et al.11 compared 6 different graphical formats and showed that pictographs were consistently associated with adequate gist and verbatim understanding among individuals with different numeracy levels but that tables were superior to pictographs in conveying verbatim understanding.
Although these previous studies provided evidence that use of pictograph and tabular presentations can improve understanding of risks and benefits, they were limited by their reliance on an Internet sample reflecting circumstances far removed from that experienced by real subjects in the clinical setting. Specifically, parents in the Internet survey were not exposed to the stresses of having their child undergo surgery and were not under the same time constraints typical of the preoperative setting. The present study was designed, therefore, to overcome some of these limitations by presenting information to a group of parents whose children were scheduled to undergo an elective surgical procedure. That our results were similar demonstrates that presenting risks and benefits in tabular or graphical formats can enhance parents' understanding even at times when they are anxious and stressed.
The observed equivalence in the parents' perceptions of the effectiveness and quality of the presented material is interesting given the differences in their understanding using the different formats. In a previous study, we showed that pictographs were deemed more effective, helpful, scientific, and trustworthy in presenting risk/benefit information in comparison with text.12 Another study, however, comparing an interactive computer-based consent program with standard written consent, found no differences in patients' preferences for the mode of message delivery.27 These observed differences between studies may reflect differences in statistical power or may simply reflect the observation that most individuals, unless offered an alternative, appear to be satisfied with the information that they receive even though they may not fully understand it.
An important finding of this study was that the manner in which the information was presented did not appear to change parents' risk perceptions or willingness to participate in the proposed research. Thus, while presenting information in tables or pictographs may increase knowledge, it does not necessarily affect risk perception decision making.
The results of this study beg the question of whether consent on the day of surgery is appropriate. While some institutions prohibit this practice as potentially coercive, the IRB at the University of Michigan allows for day-of-surgery consents. Although several studies confirm that many parents (and patients) have poor understanding of consent information,2–4 we previously showed that the timing of when consent was given (e.g., day of surgery vs. several days prior) did not appear to correlate with the level of understanding.4 Furthermore, one study suggested that providing risk information about anesthesia on the day of surgery does not increase patient anxiety.28 This suggests that a lack of understanding may have more to do with the manner in which we communicate the information rather than when it is given.
The results of this study must be interpreted in the context of some limitations. First, we recognize that consent is a process that requires both a written component (for most studies) and a verbal component. For the purposes of this study, we concentrated specifically on the written component as a means to optimize the manner in which risks and benefits are presented in the consent document. This study, therefore, is limited to one component of the consent process, recognizing that the addition of a verbal component (albeit quite variable) would likely further enhance parents' understanding. Second, the consent information provided to parents in this study was presented in a sham format. While this decision could affect the ability to generalize beyond an experimental setting, we note that the study information was presented under the same circumstances (environment and time) as in standard consent practice. There is also considerable precedence for the use of sham methodologies and strong evidence to show that behaviors based on real and sham or hypothetical situations are highly correlated.29,30 Third, the results of this study are based on knowledge of one pediatric anesthesia study from a single institution and thus may not be generalizable to all situations or populations. Furthermore, although tables and pictographs appear to improve understanding of well-defined risks and benefits as are described in this study, we recognize that they may be less effective if accurate incidence data are lacking. Finally, we recognize that because the power analysis was based on expected differences in understanding between parents randomized to the different format groups, there may have been insufficient power for all the subgroup analyses.
Informed consent for anesthesiology research (and procedures) is unique in that it is typically provided on the day of surgery with only a narrow window of opportunity for investigators and physicians to impart the information necessary for a parent or patient to make an informed decision. Given these time constraints and the potential anxiety of the subject, parent, or patient, it is perhaps even more imperative that anesthesia research and clinical personnel optimize the manner in which consent information is communicated. The results of this study suggest that disclosure of two of the most important elements of consent, the risks and the benefits, can be enhanced by presenting them in simple tabular or graphical formats. Although these results were based on parents' understanding of an anesthesia study, it is likely that these findings will also be relevant to a variety of other clinical and research settings. We therefore advocate use of these techniques as a simple, practical, and inexpensive approach to improving parents' understanding of these critical elements of consent.
The authors are indebted to Jennifer Hemberg, Caela Hesano, Elsa Pechlivanidis, and Lauren Perlin for help with subject recruitment and data collection. The authors also wish to thank Mark Dickson for his expertise in creating the pictographs.
1. . Protection of Human Subjects: 21CFR50.25. 2008
2. Barrett R. Quality of informed consent: measuring understanding among participants in oncology clinical trials. Oncol Nurs Forum 2005;32:751–5
3. Stryker J, Wray R, Emmons K, Winer E, Demetri G. Understanding the decisions of cancer clinical trial participants to enter research studies: Factors associated with informed consent, patient satisfaction, and decisional regret. Patient Educ & Counsel 2005;63:104–9
4. Tait AR, Voepel-Lewis T, Malviya S. Do they understand? (Part I): Parental consent for children participating in clinical anesthesia and surgery research. Anesthesiology 2003;98:603–8
5. Murgatroyd R, Cooper R. Readability of informed consent forms. Am J Hosp Pharm 1991;48(12):2651–2
6. Tarnowski K, Allen D, Mayhall C, Kelly P. Readability of pediatric biomedical research informed consent forms. Pediatrics 1990;85(1):58–62
7. Tait AR, Voepel-Lewis T, Malviya S. Factors that influence parents' assessments of the risks and benefits of research involving their children. Pediatrics 2004;113:727–32
8. Burkell J. What are the chances? Evaluating risk and benefit information in consumer health materials. J Med Library Assoc 2004;92:200–8
9. Theil M. The role of translations of verbal into numerical probability expressions in risk management meta-analysis. J Risk Res 2002;5:177–86
10. Brundage M, Feldman-Stewart D, Leis A, Bezjak A, Degner L, Velji K, et al. Communicating quality of life information to cancer patients: A study of six presentation formats. J Clin Oncol 2005;23:6949–56
11. Hawley S, Zikmund-Fisher B, Ubel P, Jancovic M, Lucas T, Fagerlin A. The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Educ Counsel 2008;73:448–55
12. Tait AR, Voepel-Lewis T, Zikmund-Fisher B, Fagerlin A. The effect of format on parents' understanding of the risks and benefits of clinical research: A comparison between text, tables, and graphics. J Health Comm (in press)
13. Fagerlin A, Zikmund-Fisher B, Ubel P, Jankovic A, Derry H, Smith D. Measuring numeracy without a math test: development of the subjective numeracy scale (SNS). Med Decis Making 2007;27:672–80
14. Zikmund-Fisher B, Smith D, Ubel P, Fagerlin A. A validation of the subjective numeracy scale: effects of low numeracy on comprehension of risk communication and utility elicitations. Med Decis Making 2007;27:663–71
15. Davis T, Long S, Jackson R, Mayeaux E, George R, Murphy P, Crouch M. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med 1993;25(6):391–5
16. Cacioppo J, Petty R. The need for cognition. J Pers Soc Psychol 1982;42:116–31
17. Cacioppo J, Petty R, Kao C. The efficient assessment of need for cognition. J Pers Assess 1984;48:306–7
18. Feldman-Stewart D, Brundage M, Zotov V. Further insight into the perception of quantitative information: Judgments of gist in treatment decisions. Med Decis Making 2007;27:34–43
19. Lipkus I, Hollands J. The visual communication of risk. J Natl Cancer Inst Monogr 1999;25:149–63
20. Hollands J, Spence I. Judging proportion with graphs: the summation model. Appl Cogn Psychol 1998;12:173–90
21. Simkin D, Hastie R. An information processing analysis of graph perception. J Am Stat Assoc 1987;82:454–65
22. Slovic P, Monahan J, MacGregor D. Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency benefits. Law Hum Behav 2000;24:271–96
23. Peters E, Hibbard J, Slovic P, Dieckmann N. Numeracy skill and the communication, comprehension, and use of risk– benefit information. Health Affairs 2007;26:741–8
24. Kirsch I, Jungelblut A, Jenkins L, Kalstad A. Adult literacy in America: a first look at the results of the adult national adult literacy survey. Washington, DC: National Center for Education Statistics, 1993
25. Lipkus I, Samsa G, Rimer B. General performance on a numeracy scale among highly educated samples. Med Decis Making 2001;21:37–44
26. Schwartz L, Woloshin S, Black W, Welch H. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med 1997;127:966–72
27. Tait AR, Voepel-Lewis T, Moscucci M, Brennan-Martinez C, Levine R. Patient comprehension of an interactive, computer-based information program for cardiac catheterization. Arch Intern Med 2009;169:1907–14
28. Inglis S, Farnill D. The effects of providing preoperative statistical anaesthetic-risk information. Anaesth Intens Care 1993;21:799–805
29. Jago A, Vroom V. Predicting leader behavior from a measure of behavioral intent. Acad Manage J 1978;21:715–21
30. Robinson M, Clore G. Simulation, scenarios, and emotional appraisal: Testing the convergence of real and imagined reactions to emotional stimuli. Personal Social Psychol Bull 2001;27:1520–32