OBJECTIVE: The aim of this research was to test a multimedia permission/assent (P/A) process. The overall hypothesis was that children and their parents exposed to a multimedia P/A process would have better comprehension compared with those exposed to a text-based process.
METHODS: Traditional and multimedia P/A processes were created by using an innovative learning-objective approach. A total of 194 parent-child dyads (children aged 11–14 years) were enrolled: 24 dyads in a prestudy testing P/A components for preference and effect on comprehension and 170 dyads in a randomized trial of a multimedia or paper P/A process for a hypothetical study. Participants were predominantly white and were from a metropolitan area served by a tertiary care pediatric hospital and outpatient facility. Comprehension of 8 essential elements of the P/A process was assessed.
RESULTS: The majority of prestudy subjects preferred the video version of the dual-energy radiograph absorptiometry description over the animated and paper versions combined (41 of 48 [85%]; P < .0001), and there were similar results for the abdominal ultrasound description (38 of 47 [81%]; P < .0001). Children exposed to the novel process showed significantly better overall comprehension compared with the paper P/A process (P = .0009), and there were highly significant differences in understanding of study procedures (P = .0002) and risks (P < .0001). The parental multimedia group had significantly better overall comprehension (P = .03).
CONCLUSIONS: Multimedia approaches to the research P/A process may improve overall understanding of research participation for children and parents. Improved understanding of study-specific research components (rather than research rights) may improve overall comprehension.
- multimedia consent
- multimedia assent
- consent comprehension
- assent comprehension
- study-specific research information
- regulatory research information
- learning objectives
WHAT'S KNOWN ON THIS SUBJECT:
It is generally acknowledged that the assent and consent process for research does not result in acceptable levels of comprehension by research participants. Various causes have been posited as contributing to poor comprehension, including the legal tone of forms.
WHAT THIS STUDY ADDS:
Efforts to improve participant comprehension have focused on improving the readability of forms. This article reports on a novel approach in which visual and audio media (multimedia), not a paper-based written text document, were used to improve the process.
The inclusion of children and adolescents in clinical research requires permission from the parent and assent from any child capable of giving it (≥7 years old). Institutional review boards (IRBs) and investigators take this requirement seriously, but the level of understanding resulting from the permission/assent (P/A) process remains very low, especially for study-specific information.1,–,7 Typically, P/A materials are created ad hoc by investigators, using IRB-mandated “standard language” regarding research rights and study procedures. Attempts to improve the process have included investigator education and changes in P/A forms.8,9 These efforts rarely included the use of audio-video technology or computer-aided instruction.
The aims of the present study were to develop audiovisual descriptions about research procedures and rights for incorporation into a multimedia P/A process and then to determine if incorporation of these media improved child and/or parent comprehension. A systematic approach to generation of materials and comprehension assessment based on learning objectives for research procedures and rights information was used as a first step toward an evidence-based method for materials creation. We hypothesized that exposure to the multimedia P/A process would result in improved comprehension compared with the standard paper-based process.
This research was approved by the Colorado Multiple Institutional Review Board. Parental permission and child assent were obtained before any research participation.
The study population included parent-child dyads (children 11–14 years of age) enrolling in either the prestudy (survey groups) or randomized trial of 2 P/A processes for a hypothetical research study. The hypothetical research study involved 2 common pediatric research procedures: dual-energy radiograph absorptiometry (DXA) and abdominal ultrasound (Fig 1). Participants were recruited from a large metropolitan area served by an urban medical campus with a tertiary care pediatric facility and several regional chronic care centers. Children who had known cognitive, vision, or hearing deficits or had previously undergone a DXA or ultrasound were excluded.
Part I: Survey Groups
Three survey groups of 8 parent-child dyads each were conducted in separate parent and child rooms where participants viewed 3 versions of DXA and ultrasound procedure descriptions in varied order: (1) a video with voice-over explanation; (2) a standard IRB-approved text; and (3) an animated version with voice-over explanation. The animated version used the same footage as the video version, but an animated boy replaced the live boy. The survey groups were used to determine the suitability and acceptability of the procedure descriptions for the research population.
Parents provided demographic data, including parent age, marital status, employment, education, race, ethnicity, and any child medical diagnosis.
Participants were asked preference and comprehension questions. An audience response system was used to collect data and ensure privacy. All questions were multiple choice and shown individually on a central screen. Preference questions asked about suitability for the study population (children aged 11–14 years), if respondents wanted the version used with other research participants, and if the description was frightening. Participants ranked their first, second, and least favorite among video, paper, or animated versions for the DXA and ultrasound descriptions. A total of 9 preference questions per participant were asked, and group preferences were calculated as the proportion who rated each version as their first favorite.
After the first DXA and first ultrasound viewed by each group, 5 comprehension questions about each procedure (10 total) were asked about procedure risk, whether the child had to wear a hospital gown, the child's position during the procedure, how the procedure made a picture of the child's body, and what part of the body the procedure involved. Comprehension scores were calculated as percent correct for the 10 questions.
Part II: Randomized Controlled Trial
Using data from the survey groups, standard and multimedia P/A processes were created using a learning-objective approach and third grade language. These processes described a hypothetical research study entitled “Pretend Study to Determine the Amount of Muscle, Tissue, and Fat in Bodies.” The standard assent document (IRB approved) was constructed by using institutional templates and language (Fig 2). The explanatory text used in both paper and multimedia was identical and both took 7 to 10 minutes to complete.
The multimedia process was constructed in PowerPoint (Microsoft Corporation, Redmond, WA) using the learning-objective approach, as was done for the standard document (Fig 3). The information about research rights and the DXA and ultrasound procedures was contained in 5 hyperlinks to video content embedded in the PowerPoint document. Each hyperlink was accessed once by each parent-child dyad. Three hyperlinks about research rights showed an adolescent boy reciting the hyperlink-specific text: (1) what research is; (2) what assent means; and (3) the right to refuse participation or withdraw. The text was created by using short declarative sentences and age-appropriate examples. The multimedia process contained 2 hyperlinks about the study-specific procedures (DXA and ultrasound). These videos showed an adolescent male being prepared for and having each procedure with voice-over containing the learning-objective content, such as risks and purpose of each procedure.
During the study visit, each child completed the 2-subset Wechsler Abbreviated Scale of Intelligence as a brief cognitive measure (vocabulary and matrix reasoning subsets). Parents completed a demographic questionnaire.
After exposure to the test P/A process, parents and children (separately) used a visual analog scale (VAS), scored 0 to 10, to indicate how well they thought they understood the hypothetical research study. The VAS was anchored with 0 (“I don't understand anything about the study”) and 10 (“I understand everything about the study”). These data were collected to determine if there were between-group differences in the degree of overestimation. Participants independently answered questions about 8 essential elements of the P/A process (Table 1). Children participated in the Post-assent Comprehension Interview (PCI), a semistructured interview including these questions. Parents completed the Parental Post-consent Comprehension Interview (PPCI), a paper form with the same questions (Table 1). All children were told that we were testing the materials to see how well they helped the child remember the details of the pretend study. The child interviews were audio-recorded, and file names included only a study number. Interviews were subsequently transcribed onto a template with all verbiage uttered by the participant included. Interviews were scored from the written transcript (to avoid recognition bias) by 3 coders.
Assessment of P/A Comprehension
The coding instructions for scoring parent and child comprehension assessments used the same learning objectives as for the P/A process. Possible scores were 0 (no understanding), 5 (correct but incomplete understanding), and 10 (correct and complete understanding [after Tait et al9]). An answer receiving a score of 5 contained the necessary components for a correct answer for any individual question. A score of 10 included these necessary components with greater detail. Only those items that were included in the set of learning objectives for each item were considered scorable responses. The first 58 PCIs scored were used as the coding instructions development set. Examples of responses for each possible score for each question were extracted from this set of 58 and used as guidance in the coding instructions. Table 2 is an example of the scoring instructions for question 2 (study procedures). All other questions used a similar schema with necessary components (score of 5) and additional detail (score of 10). Because of an error in presentation of the test P/A instruments, the learning objectives for question 1 (study purpose) were not delivered as designed for the multimedia P/A process. Thus, question 1 was eliminated from the total score. The question about alternatives to research participation (question 6), required by the local IRB regardless of study design, was confusing to children in the context of a noninterventional study; there were frequent non sequitur responses. Thus, this question was also eliminated from the total score, leaving a total possible score for children of 0 to 60.
Parent-child dyads (170 dyads, 340 people) participated in a 30-minute visit. Parent and child understanding of the difference between the actual study and the hypothetical study were established. All participants understood that they were not being asked to be in the hypothetical study. Each parent-child dyad was randomly assigned to either the paper-based or the multimedia process group. Parent and child were seated next to each other with either the paper form or a laptop computer in front of them. All researcher-participant interactions were scripted to ensure consistency. At the end of the process, the parent was asked if he or she would allow the child to be in the study (if it were a real study) and the child was asked if he or she would agree to be in the study. All participants agreed. The child was interviewed by the principal investigator (blinded to group assignment) using the PCI and approved scripted prompts. No child revealed his or her group assignment to the interviewer.
All analyses assumed a 2-sided test of hypothesis with a significance level of .05. SAS 9.2 or JMP 8.0 (SAS Institute, Inc, Cary, NC) was used for the statistical analysis.
The proportion of participants preferring each procedure description was calculated, and pairwise comparisons were made using a binomial proportions test. For the comprehension score, we calculated the percentage correct of the 10 questions (5 DXA, 5 ultrasound). Because of small sample sizes, a Kruskal-Wallis test was used to compare comprehension across the 3 groups and, if the overall test was significant, pairwise comparisons were made.
Randomized Controlled Trial
Power was based on total comprehension score assuming an SD of 13.6.9 Using a 2-sample Student's t test, 82 subjects per group provided 80% power to detect an overall difference of 6 points. To ensure consistent scoring, the first 58 PCIs were independently coded by 3 coders using the first iteration of the coding instructions developed as part of this study. Scores for each question were compared among the coders. Any scores that did not have complete agreement between 2 of the 3 coders were marked, as were any answers with scores varying by 10 between any 2 coders. These 2 types of discordant scores (13% of questions) were discussed by the coders as a group. Amplifications to the coding instructions were agreed on that eliminated any variance of >5 points for any of the discordant questions. Bland-Altman plots10 and κ statistics showed that 95% of the differences lay within ±2 SDs (4.6–4.8) of the mean difference for the measurements (data not shown). With possible individual question scores of 0, 5, and 10 and total scores of 0 to 60, this degree of interrater reliability is acceptable. In addition, for each of 2 important study-specific questions (study purpose [question 1] and study procedures [question 2]), standard and weighted κ statistics were calculated with 95% confidence intervals (Table 3).
Participant characteristics were compared between groups using Student's t tests, χ2 tests, and Fisher's exact tests as appropriate. Because there were no characteristic differences between groups, the comprehension assessments were compared using Student's t tests and χ2 tests.
Part I: Survey Groups
The video version of the DXA procedure was preferred over the paper and animated versions combined for all subjects (41 of 48 [85%]; P < .0001), and there were similar results for the ultrasound description (38 of 47 [81%]; P < .0001). There was no overall difference in child understanding among the 3 versions (P = .82). An overall difference was observed for parents' comprehension along with a higher median score for the video and animated versions (P = .05 and .02, respectively) compared with the paper format. The animated procedure versions were not well received; “creepy” was the most frequent comment. These versions, therefore, were not included in the randomized trial segment.
Part II: Randomized Controlled Trial
The study population of parents and children were evenly balanced for demographic variables between groups (Table 4); they were generally well educated and were predominantly non-Hispanic white. Children were within the range of normal intelligence for their age.
For children, better comprehension was observed in the multimedia group for total score, study procedures, and risks (P = .0009, P = .0002, and P < .0001, respectively; Table 5). There were no significant differences for questions pertaining to benefit to self and others (questions 4 and 5), and all children, regardless of group, knew that they could refuse participation or withdraw if they changed their minds (questions 7 and 8). Seventy-six percent (129 of 169) did not understand there were risks associated with the study. However, the proportion with some understanding of risk (a score of 5 or 10) was higher for the multimedia group than for the paper group (0.41 vs 0.07; P < .0001). There were no between-group differences observed for the child's self-assessment of study comprehension VAS.
For parents, we observed a significant difference for the PPCI total score (P = .03) (Table 6). There was a between-group difference in the self-assessed comprehension score, but it did not reach statistical significance (VAS; P = .15). Similar to the children, 73% (123 of 169) of parents did not understand the risks associated with the study. The proportion with some understanding of risk (a score of 5 or 10) was higher for the multimedia group (0.43 vs 0.12; P < .0001).
In the survey groups (prestudy), parents and children significantly preferred video procedure descriptions. We observed significantly better comprehension of procedure details in parents exposed to the video or animated descriptions compared with text.
In the randomized trial, we observed significantly better overall comprehension of the hypothetical study and its research procedures for children and parents exposed to the multimedia P/A process compared with text. Most children and parents did not comprehend the presence or nature of risk associated with (hypothetical study) participation. Of those who received a positive score on the risk question, an overwhelming majority had been exposed to the multimedia assent, a highly significant finding. The risk associated with the DXA was described as equivalent to 2 days of ambient radiation for the study location. Perhaps this equivalence approach was not understood, or, more likely, the risk was perceived as negligible to none for this population. All children answered the question about their rights to withdraw correctly and knew the research was voluntary. All children and parents overestimated their comprehension of the P/A process as measured by the VAS compared with the PCI and PPCI scores, respectively, and there were no significant between-group differences.
These results suggest that a multimedia approach to the P/A process improves study comprehension in both parents and children. This novel technique may be a valuable tool for use in pediatric research for parents and children.
The P/A process involved a hypothetical research study involving simple, low-risk procedures and thus may not represent actual research studies. However, this limitation affected both groups equally, making the significant differences important. Testing of this approach with a higher-risk and more complex study is under way. The study population was neither ethnically nor racially diverse. Thus, testing in minority groups is an important next step. The approach can be customized using regional dialects, colloquial speech, and minority participants. Cost differences between the 2 approaches were not calculated. The monetary cost for the novel approach might be higher because of equipment and expertise needs; however, the increase in quality and improvement in participant understanding may justify these costs. In addition, production of such materials centrally may be a cost savings for multisite projects.
Multimedia Approaches: Potential to Improve Participant Understanding of Research
The use of multimedia in both academic and health education has positively influenced the effects of interventions to improve self-care for chronic diseases, improve healthy behaviors, and reduce risky behaviors. Adopting such approaches for the P/A process has the potential to improve comprehension and participant satisfaction with the process.8,11,–,13 Hyperlinks to deliver study-specific research information could be included as part of the P/A process or as supplemental materials.
Researcher training in informed consent has focused on the importance of informing the potential participant of his or her rights while neglecting study-specific information. Studies have shown better participant understanding of research rights compared with the more study-specific information.1,–,7 Efforts to improve participant comprehension should focus on improving understanding of this information. The approach described here may facilitate 3 changes in the practice of the research P/A process. First, by using accepted educational and health literacy strategies,14 use of learning objectives, and an array of media and methods to deliver carefully formulated educational content, the P/A process can be transformed from a legal advisory process to an educational one.
Second, the multimedia approach is accessible to any institution with standard computer resources. Furthermore, research sponsors could produce and distribute multimedia materials for multisite studies, thereby assuring both quality and consistency through the inclusion of “standard video footage” as an adjunct to standard language.
Third, a multimedia approach has important strengths inherent for delivery of the learning objective content, as well as potential for additional improvements. Multimedia is visually and technologically engaging, capturing the viewer's attention and prolonging educational content exposure.15 Content can be presented in a variety of ways, targeting different learning styles; wide, flexible, and continuous accessibility can be provided via CDs and DVDs, memory devices, and the Internet. Finally, production can be centralized and controlled for clinical details and information, thereby improving accuracy and quality. Libraries could be created to include the most common research procedures for specific research disciplines. The procedure segments could then have explanatory voice-over text added in any language, regional dialect, or accent to optimize acceptability.
Additional investigation of these methods in diverse populations, across multiple research disciplines, is warranted and will help establish the contribution such approaches can make to the conduct of clinical research and education of participants. Although this study used readily available technology, more sophisticated technology and more appealing materials could be created with built-in review and assessment functions.
Informing human subjects about their research participation has not appreciably changed in 5 decades. In that time, technologic innovations have caused dramatic improvements in educational methods from early grades through postgraduate degree programs. These same technologies have been leveraged to effect important improvements in health education for patients with a variety of chronic diseases. By 1 estimate, >80% of Internet users have accessed health information online16 It is well beyond the time to bring technology to bear on the important informed consent and assent process.
Dr O'Lonergan received funding support from the Children's Hospital Research Institute, which covered materials, participant compensation, and the salary of research assistants, but she received no salary support for this project.
This study also received partial support from NIH/NCRR Colorado CTSI Grant UL1RR025780. Its contents are the authors' sole responsibility and do not necessarily represent official NIH views.
We acknowledge our research assistants, Luy Nguyen and Bridget Nilsson, for their help in conducting this study. We also thank Phil Zeitler, Steve Daniels, and Marianne Wamboldt for their helpful comments on the manuscript.
- Accepted January 31, 2011.
- Address correspondence to Theresa A. O'Lonergan, MA, PhD, Department of Pediatrics, University of Colorado Denver, 13123 E 16th Ave, B218, Aurora, CO 80045. E-mail:
Dr O'Lonergan wrote the first draft of the manuscript, designed and conducted the study, and contributed to data analyses. Dr Harwood contributed significantly to the statistical design of the study, authoring those sections of the research protocol; she was primarily responsible for statistical analyses of the study results, contributed significantly to writing the manuscript, and created all results tables.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Funded in part by the National Institutes of Health (NIH).
- IRB =
- institutional review board •
- P/A =
- permission/assent •
- DXA =
- dual-energy radiograph absorptiometry •
- VAS =
- visual analog scale •
- PPCI =
- Parental Post-consent Comprehension Interview •
- PCI =
- Post-assent Comprehension Interview
- Chappuy H,
- Doz F,
- Blanche S,
- Gentet JC,
- Pons G,
- Tréluyer M
- Jeste DV,
- Palmer BW,
- Golshan S,
- et al
- Jimison HB,
- Sher PP,
- Appleyard R,
- LeVernois Y
- 16.↵Pew Research Center Publications. E-patients: chronically ill seek health information online, 2008. Available from: www.pewresearch.org/pubs/938/e-patientschronically-ill-seek-health-information-online. Accessed January 18, 2011
- Copyright © 2011 by the American Academy of Pediatrics