ARTICLE |
a Departments of Pediatrics
d Medicine
e Academic Affairs, National Jewish Medical and Research Center, Denver, Colorado
b Department of Medicine, Johns Hopkins University, Baltimore, Maryland
c Department of Sociology, City University of New York, Queens College and the Graduate Center, Flushing, New York
| ABSTRACT |
|---|
|
|
|---|
PATIENTS AND METHODS. Participants included 104 children who were being treated with an inhaled corticosteroid delivered by a metered-dose inhaler for asthma diagnosed by their health care provider. Each parent and child dyad was randomly assigned to 1 of 3 self-report adherence-assessment modes: (1) audio computer-assisted self-interviewing; (2) face-to-face interview with study staff; or (3) self-administered paper-and-pencil questionnaire. At the 4 monthly visits, the parent and child were interviewed separately and asked questions about adherence on the previous day and in the past week. Electronic devices were attached to the each participant's metered-dose inhaler to provide an objective record of actual daily medication activations.
RESULTS. Both children and parents greatly overreported their inhaled corticosteroid adherence when queried about either time frame (1 day or 1 week) in any of the 3 interview modes. One of 3 responses reported full adherence when no medication had been taken. Inconsistent with the study hypothesis, discrepancy between self-report and objectively measured adherence was greatest in the computer-interview condition. In the optimal circumstance where children were interviewed by study staff about their adherence within the previous 24 hours, reported adherence was within the ±25% accuracy range for only half of the participants. Larger discrepancy scores were observed for both parents and children when reporting by computer or questionnaire.
CONCLUSIONS. Under the best of conditions in this study, accuracy of self-report was insufficient to provide a stand-alone measure of adherence. Verification of treatment adherence by objective measures remains necessary.
Key Words: asthma corticosteroids patient nonadherence interview mode
Abbreviations: ICS—inhaled corticosteroid ACASI—audio computer-assisted self-interviewing MDI—metered-dose inhaler
Self-report is the most commonly used method for assessing parent and child health-related behaviors in clinical and research settings. Researchers and health care providers routinely rely on parent and patient reports of health-related behaviors, including health-promoting (eg, exercise, diet, and medication adherence) and health-jeopardizing (eg, smoking and illicit drug use) behaviors. Pediatric clinical trials rely on self-report, often including parent- and patient-provided information about symptoms, behaviors, quality of life, and treatment adherence. The accuracy of conclusions based on these data is dependent on the validity of the self-reported information.
The accuracy of self-reports of health-related behavior, in particular, medication treatment adherence, has been examined across a wide range of diseases and medical regimens and has been found to be highly variable and, in many instances, markedly inaccurate.1 Children with asthma frequently take no more than half of their prescribed controller medication, whereas they report much higher levels of adherence. For example, electronic monitoring documented that children with asthma used only 50% of prescribed inhaled corticosteroid (ICS) over 6 months, whereas the patients and their mothers reported >80% adherence.2
Accuracy of parent and child self-reported adherence may be influenced by a number of variables, including content, memory ability, social desirability, characteristics of the respondent, and assessment mode. Content that is potentially embarrassing to parents, such as failure to vaccinate their child,3 may lead to underreporting. Younger children have less developed cognitive skills and, therefore, may do less well than older children on tasks that demand memory ability.4 Parents who are strongly influenced by the social desirability effect, reflecting the desire to present a positive impression to the health care provider or researcher, will tend to underreport undesirable behaviors, such as excessive eating,5 or overreport desirable behaviors, such as exercise.6 Information accuracy also depends on the respondent. Parents and children may produce differing accounts of the child's medication adherence,2 and children may underreport behaviors such as alcohol use if a parent is present.7 Finally, variation in assessment mode, including questionnaire, direct interview, or computer responding, can influence reporting bias.8
Pediatric researchers seeking to maximize self-report accuracy to obtain valid information should consider the variables that influence accuracy. For example, patient recall accuracy improves when possibility of embarrassment is reduced and recall is limited to events that occurred in the last 24 hours.9,10 Multiple 24-hour interviews can be combined to produce a comprehensive estimate of treatment adherence.10–12 Mode of reporting is another determinant of accuracy but seems to interact with other variables, most particularly content and risk of embarrassment. Face-to-face inquiry about stigmatized behaviors may lead to less forthcoming responses. For example, compared with face-to-face inquiry, interviews using audio computer-assisted self-interviewing (ACASI) in assessing parent reports of infant feeding practices found that ACASI seemed to decrease socially desirable responses and improve information quality.13 However, in these studies, whereas relative rates of reported events were compared between inquiry conditions, no objective behavior measure was included to determine actual accuracy of each self-report condition.
The admission of treatment nonadherence is a source of potential embarrassment and, therefore, may account for the tendency of parents and children to largely overreport their use of medications. However, it is not known whether nonadherence is stigmatized to a degree so that increased anonymity improves reporting accuracy. The purpose of this investigation was to test the effect of reporting mode on accuracy of ICS adherence reporting in children with asthma under conditions similar to an asthma clinical trial. We hypothesized that adherence report accuracy would be highest where parent and child interviews were conducted by a computer with no other person present and lowest when conducted in a face-to-face interview with study personnel. Study design included random assignment of participants to 3 assessment modes reflecting differing levels of potential exposure to interpersonal embarrassment. Multiple respondents (children and parents) and questions were included to contrast additional variables that might impact recall accuracy. Objective measurement of adherence was included to allow for the determination of relative discrepancy among the 3 self-report conditions.
| METHODS |
|---|
|
|
|---|
One parent was required to participate with their child. Children <18 years of age provided informed assent; parents of these children and participants who were 18 years of age provided informed consent. Each parent and child dyad was randomly assigned to 1 of 3 self-report adherence-assessment modes: (1) ACASI; (2) face-to-face interview with study staff; or (3) self-administered paper-and-pencil questionnaire. Research assistants who conducted each visit were unblinded, but the investigators remained blind to condition until completion of data collection. Visits occurred at baseline, 1 month, 2 months, 3 months, and 4 months. The 4-month study period with monthly visits was included to provide a sufficient length of time to document the comparative validity of the adherence measures. At each 45-minute clinic visit, the participant's MDI was equipped with a newly initialized MDILog (WestMed, Englewood, CO) or Doser (Meditrack, Hudson, MA), which are electronic devices that attach to the side of a plastic MDI or the top of the canister and record daily activations of the MDI on a microchip. To track device failures and prevent data contamination, all of the devices were tested immediately before they were given to participants and again after they were returned at monthly visits.
Parents were physically separated from the child for the period when they were reporting adherence. The ACASI technology was developed by researchers at Research Triangle Institute (Research Triangle Park, NC) and City University of New York to administer complex questionnaires in personal interview surveys.14 Interview procedures were piloted on the first 20 children meeting study inclusion criteria to determine that the children understood the questions and were able to respond using the ACASI system. Adherence questions were adapted from the Adherence in Clinical Trials Questionnaire15 and focused on puffs of ICS taken on the previous day and in the past week. Adherence questions were identical across the 3 interview modes. At the baseline visit, questions were reviewed to assure that they were clearly understood by both parent and child. Procedures were explained for those in the ACASI condition, in which the participant wore headphones connected to the computer, heard each recorded question, responded by pushing a numbered button on the computer keyboard, and could elect to hear the question repeated.
Consistent with most clinical guidelines for the management of asthma, participants received asthma education and educational materials at study visits and were encouraged to adhere to their prescribed treatment protocol. All of the children received the same educational content in the 5 asthma education modules, which were culturally sensitive, required low literacy, and included (1) establishing a partnership to manage asthma, (2) asthma basics, (3) asthma medicines, inhaler/spacer skills, and using a peak flow meter, (4) action plan to manage asthma, and (5) triggers of asthma. This study was approved by the institutional review boards at National Jewish Medical and Research Center, Research Triangle Institute, and the City University of New York.
Data Analysis
Objective and self-reported adherence were determined separately for each of the 2 respondents (child and parent) and for each of the time periods (yesterday and last 7 days). Percentage of adherence was determined by dividing the number of puffs taken (reported or recorded) by the number prescribed. Self-report discrepancy was the primary outcome variable and was obtained by subtracting each participant's self-report percentage of adherence from their objective measure percentage of adherence. Thus, positive discrepancy scores represent underreporting, whereas negative discrepancy scores represent overreporting of adherence. A 0 discrepancy indicates exact agreement between objective and self-report measures of adherence. A ±25% accuracy range was chosen for this analysis because, first, a precedent has been established of using a 25% drop in adherence as the cutoff between adequate and inadequate adherence16–18 and, second, because that range is below the average discrepancy scores reported previously for parents and children self-reporting adherence.2
All of the statistical analyses were conducted with SAS 9.1 (SAS Institute Inc, Cary, NC). Percentage of adherence of both the self-report and objective measures was truncated to 100% if it was >100% to prevent artificially elevating mean adherence. The nonparametric Kruskal-Wallis test was used to compare discrepancy scores. A generalized linear regression model (SAS Proc Genmod, SAS Institute Inc) was used to compare ordinal discrepancy scores among reporting modes with and without covariates (age, race, and gender). The ordinal discrepancy scores were assumed to have multinomial distributions with a cumulative logit link. The footnotes in Table 2 provide the categorization of ordinal discrepancy scores. For all of the analyses, 2-tailed tests were used, and P values of
.05 were determined to be statistically significant.
|
| RESULTS |
|---|
|
|
|---|
Adherence Discrepancy Scores According to Mode and Respondent
Sixty-four subjects had a Doser and 40 subjects had an MDIlog. Adherence-monitoring–device failures were detected at 25 visits, or 6% of total visits. These measurement intervals were omitted from all of the data analyses. Because discrepancy scores were not normally distributed, a nonparametric Kruskal-Wallis test was used to compare the 3 group median discrepancy scores and interquartile ranges (range between the 25th and 75th quartiles; Table 1). For child self-reports, the discrepancy scores were significantly different among the 3 interview modes on both the yesterday and last-7-days questions, with greatest discrepancy in the ACASI mode. For parent self-reports, the discrepancy scores were significantly different among the 3 interview modes on the last-7-days question, where ACASI mode produced the lowest median and the largest discrepancies. Mode differences were not found for the yesterday question, although the data trend followed a pattern similar to the last-7-days question where ACASI mode had the least number of accurate scores and the most –100% scores, suggesting that parents were least accurate in the ACASI mode (Table 1).
|
25% and, in the ACASI mode, more than one third reported 100% adherence when no ICS had been used. Underreporting of adherence occurred very infrequently. Child self-reported adherence was in the ±25% accuracy range for 50% and 48.7% of those children in the clinical interview condition for the yesterday and last-7-days questions, respectively. In contrast, child reports had 39.2% and 32.8% of responses in the ±25% accuracy range for the ACASI condition for the yesterday and last-7-days questions, respectively. Results were similar for parents, where 47.8% and 41.4% were in the ±25% accuracy range when interviewed with the 2 kinds of questions in contrast to 29.4% when responding to both questions via computer. Increasing the accuracy range to ±35% or decreasing it to ± 15% predictably resulted in greater or fewer numbers of accurate reports, respectively. Nonetheless, where group differences emerged, the ACASI group produced the least proportion of accurate responses (data not shown).
|
|
4 measurements was accounted for in the model. As in the nonparametric comparison of medians, this model revealed self-report discrepancies that were greatest in the ACASI condition for both questions (yesterday and past 7 days) when the child was queried and a trend (P = .068) indicating a similar pattern when parents were queried about adherence over the past 7 days. These results were not affected when age, gender, and race were added as covariates into the model (Table 2). Device (Doser versus MDILog) was also entered as a covariate but, because it was nonsignificant, was not included in the final model. | DISCUSSION |
|---|
|
|
|---|
Inadequate accounting of adherence may compromise clinical trials and the treatment guidelines that they inform. Data from clinical trials provide the evidence base for pediatric treatment guidelines. When information about dose-response sensitivity does not take into account variability in adherence of trial participants, the possibility of recommending unnecessarily high drug dose increases.21 Diary-card reporting of medication in clinical trials, long the standard for establishing adherence sufficient to support a trial's validity, has been shown repeatedly to overestimate adherence. For example, diaries resulted in overreporting of adherence by 67% in a trial of treatment for chronic obstructive pulmonary disease22 and by 60% and 68% in studies of children2 and adults23 with asthma. Evidence from this study, in combination with previous research, indicates that no self-report methodology has yet been demonstrated to match objective adherence measures, yet self-report measures of adherence continue to be used in pediatric clinical trials more frequently than electronic adherence measures.
Patient or parent misrepresentation of medication adherence in the pediatrician's office may also result in unnecessary escalation of treatment while failing to remediate an important factor underlying poor illness control, medication nonadherence. Although this study was not designed to evaluate adherence reporting in pediatric offices, it nonetheless suggests that accuracy of self-report in clinical settings is of concern and warrants further investigation, particularly in light of findings that patients frequently provide inaccurate information on medication adherence to their physicians.24–25
Limitations to this study should be considered. The relative accuracy of 24-hour and 7-day adherence queries may not be best established through simultaneous questioning. The ±25% accuracy range chosen for this analysis may not represent an acceptable margin of accuracy for all physicians and researchers. The 5-module patient education program may have increased adherence of participants, although it is likely that such effects would have been distributed equally across conditions. Measurement of patient or parent embarrassment, not included in this study, should be incorporated in future investigations of self-report accuracy. The interview methodology used in this study did not include the use of telephone-based adherence inquiry, which may provide a greater degree of anonymity and, therefore, greater encouragement of disclosure than seen in the 3 conditions included here. Pediatricians who invest time and effort communicating effectively with families may obtain more accurate adherence information than seen in this investigation.26 Objective adherence measures, although more accurate than self-report, may still introduce measurement error, which must be managed by judicious data inspection and management, including truncation of daily adherence scores.27–29
| CONCLUSIONS |
|---|
|
|
|---|
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
Address correspondence to Bruce G. Bender, PhD, Pediatric Behavioral Health, National Jewish Medical and Research Center, 1400 Jackson St, Denver, CO 80206. E-mail: benderb{at}njc.org
The authors have indicated they have no financial relationships relevant to this article to disclose.
| REFERENCES |
|---|
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||