An Evaluation of an Innovative Multimedia Educational Software Program for Asthma Management: Report of a Randomized, Controlled Trial
Background. Asthma continues to be a substantial cause of morbidity in pediatric populations. New strategies are needed to provide cost-effective educational interventions for children with asthma, particularly those in the inner city.
Objective. To assess the effectiveness of a multimedia educational software program about asthma.
Setting. A hospital-based primary care clinic and an affiliated neighborhood health center.
Design. Randomized, controlled trial.
Population. Children 3 to 12 years old with physician-diagnosed asthma.
Intervention. An interactive educational computer program, Asthma Control, designed to teach children about asthma and its management. Using a graphic display of a child going through simulated daily events, the game emphasizes: 1) monitoring; 2) allergen identification; 3) use of medications; 4) use of health services; and 5) maintenance of normal activity. Control group participants reviewed printed educational materials with a research assistant.
Outcomes. Acute health care use (emergency department and outpatient) was the primary outcome. Secondary outcome measures included maternal report of asthma symptom severity, child functional status and school absences, satisfaction with care, and parental and child knowledge of asthma.
Results. A total of 137 families were enrolled in the study (76 intervention, 61 control). Both intervention and control groups showed substantial improvement in all outcomes during the 12-month follow-up period. Aside from improvement in knowledge after use of the computer program, no differences were demonstrated between the 2 groups in primary or secondary outcome measures. Children reported enjoyment of program use.
Conclusions. This trial of an educational software program found that it did not produce greater improvement than occurred with review of traditional written materials. Because both groups showed substantial improvement over baseline, computer-based education may be more cost-effective. Alternatively, improvements in illness severity over time may overshadow the effects of such interventions.
Rigorous comprehensive evaluations such as this are necessary to assess new interventions intended to improve management and outcomes of asthma.
Asthma continues to cause substantial morbidity among children and young adults.1 Morbidity and mortality persist despite the availability of new and effective therapies for this condition.
Effective management of asthma requires the active participation of children and their families in a complex set of contingent management activities. Good asthma care requires that the child and family adjust the child's activities and home environment contingent on the child's particular sensitivities, and vary the child's medical regimen depending on the child's physiologic state.2 In addition, children and families need to communicate effectively with the child's clinician so that the treatment regimen can be tailored to the particular situation.3
Educators and health professionals have developed numerous patient self-management programs that seek to enhance both the child's and the family's efficacy in asthma care.4 These programs have been found to have a modest effect on patients with the most severe asthma, but have not been widely implemented in most care settings.5 Such programs typically require an ongoing commitment of time by a health educator or nurse and present substantial logistic obstacles to implementation in the clinical environment.
We developed an interactive multimedia asthma education computer program, both to provide patients with the substantive knowledge required for good asthma care and to provide a simulated environment in which children and their families could safely gain experience with asthma-related contingent decision-making. Educational theory supports linking the teaching of a desired set of actions, such as asthma management, with a desirable pastime activity for children, such as playing video/computer games. In addition, social learning theory suggests that knowledge and skills gained in such simulated environments do transfer to real situations.6
Rubin et al7 previously demonstrated that a computer-based approach to improve asthma knowledge and behaviors was effective in decreasing acute care visits. We sought to extend this work by updating the recommendations in the program so that they were consistent with current expert guidelines, particularly in emphasizing the use of preventive medications. In addition, we sought to expand the range of simulated environments, educational content, and graphic capabilities of the program. We focused on a high-risk, urban population, as this population is at the greatest risk of experiencing adverse asthma-related outcomes. We hypothesized that this program would be an effective and efficient approach to improve asthma-related knowledge and skills, and thereby reduce asthma-related morbidity.
A randomized, controlled trial of a computer-based multimedia asthma education program, Asthma Control, was conducted among children with asthma who received care at 1 of 2 programs serving urban youth in Boston—a hospital-based primary care clinic at Children's Hospital, Boston, and an affiliated neighborhood health center.
Children were eligible for inclusion in the study if they were between ages 3 and 12 and had any outpatient visits, emergency department visits, or inpatient admissions for asthma during the year before enrollment. Patients were excluded for the presence of a second major chronic illness with a pulmonary component (eg, cystic fibrosis), residence outside of Boston and surrounding communities, or involvement in other clinical trials or protocols related to asthma. Children were recruited to participate at the time of visits to the care site, either for scheduled health care maintenance visits or for illness-related encounters, including visits for asthma.
Asthma Control is an interactive educational computer program designed to teach children about asthma and its management. Using a graphic display of a child going through simulated daily events, the game emphasizes 5 educational objectives: 1) symptom recognition and monitoring; 2) allergen identification; 3) appropriate use of medications; 4) appropriate use of health services; and 5) maintenance of normal activity, such as school attendance.
The object of the game is to help the main character, Spacer, a superhero who has asthma, complete all 6 levels (3 home and 3 outdoor levels) of the game while keeping his/her asthma under control. The player must use his/her knowledge of asthma management to help Spacer eliminate common indoor allergens (such as cats, dusts, and cigarette smoke) in the home and recognize and avoid allergens found outside. To maximize their game scores, players must also make decisions for Spacer using the same asthma management options available to them in their lives. Decisions include taking preventive medications on a regular basis; taking rescue medication during an asthma exacerbation; premedicating before exercise; resting; consulting a doctor for advice; and going to the doctor's office for treatment.
The program uses several different approaches to providing education about asthma, and incorporates multiple incentives for undertaking appropriate asthma-related practices. Encounters with specific objects—such as a cat, rug, particular medicines, or cigarette smoke—initiate brief (10–40 second) video clips linked to that object. More general asthma education is also provided at each of the home levels through encountering an asthma book. Once a peak flow meter icon is selected at the first home level, a graphic representation of a peak flow meter with 3 colored zones appears on the side of the screen at all times. The indicator on this meter serves as an asthma condition scale for the player, moving up or down depending on the specific actions of Spacer. As Spacer's condition worsens, the program produces first coughing and then wheezing sounds. When the indicator enters the yellow zone, Spacer cannot jump or run. If Spacer does not eliminate home allergens and triggers, Spacer's mother blocks his/her exit from the home. If Spacer omits the use of preventive medicines, then subsequent exposures lower Spacer's peak flow meter reading at twice the original pace.
An instructional designer, a software programmer, a generalist and a specialist asthma expert, a research assistant, and the principal investigator constituted the team that developed the software. Before implementation, the game underwent individual and group pilot testing among children of comparable age, but somewhat more socioeconomically advantaged than study children. After implementation, the game underwent minor modifications in response to comments and suggestions by early users.
At the time of recruitment, research assistants obtained informed consent and then administered the baseline assessment measures. After these were completed, children were randomized to either the intervention or the control group. Study assignment was indicated in a sealed, opaque envelope. Separate randomization lists were generated by computer for each site, and, within site, for children younger than 7 years and 7 years and older. Randomization did not match or stratify on any other characteristics.
Children randomized to the intervention arm were asked to return for 3 visits to use the game. Control group parents were also requested to return 3 times. At each of these visits, control group children reviewed an age-appropriate asthma education book and play a noneducational computer game (Lemmings). Both children and parents were surveyed before and after each use of the computer game to learn their impressions of the computer game and to assess their knowledge and understanding of asthma. Children and parents were observed while using the game by a research assistant who made qualitative observations and filled out a structured encounter form.
Parents were surveyed by telephone at monthly intervals for 8 months. They were asked how many emergency visits, hospitalizations, acute office visits, school days, and parental work days were missed because of their child's asthma, and about their child's medication and peak flow meter use since the previous contact.
Parents were mailed an exit questionnaire 9 months after enrollment. A $5 restaurant voucher was offered to all those who would respond to the questionnaire. Nonrespondents were telephoned by a research assistant 2 weeks after and periodically, and they either were administered the survey by telephone or, if they preferred, were mailed another copy.
The primary outcome measure was the total number of emergency department visits and acute office visits for asthma during the study period, obtained both by parental report and review of administrative encounter data. The administrative data are a component of the hospital billing system, and reliably capture visits to the institution and program. Visits outside the host institution are not captured by this system.
Secondary outcome measures were the child's average asthma specific symptom severity during the study period and functional status at the conclusion of the study. Asthma symptom severity was assessed using a 3-point scale (mild, moderate, and severe) based on the symptom and functional indicators included in the severity categorization of the first National Asthma Education and Prevention Program guidelines.9 As acknowledged in the more recent National Asthma Education and Prevention Program asthma guidelines, these indicators are based on expert consensus and their psychometric properties have not been formally assessed.2 Functional status was assessed using the Child Health Questionnaire (CHQ-PF50) summary (physical and psychological) and dimension specific scores.8
Additional outcome measures included satisfaction with care, use of peak flow monitoring, number of common triggers and allergens in the home environment, and knowledge of asthma. Patient satisfaction was assessed by ratings of physician communication, physician involvement, and asthma content-specific care on a 5-point scale from excellent to poor; this measure, although similar in format to the well-established MISS, was developed specifically for this study without extensive pilot testing or validation. Parental knowledge of asthma was assessed at exit with 11 true or false questions taken from the National Heart, Lung, and Blood Institute (NHLBI) Asthma IQ.9
Short-term assessment of game efficacy was assessed by children completing a 13-item asthma knowledge questionnaire both before and after sessions with the game. The questionnaire consisted of a checklist of 10 items from which to select likely environmental triggers, and open- and closed-ended questions about the use of a peak flow meter, exercise, and medicines.
The numbers of emergency and acute office visits and means of all outcome measures were calculated before and after treatment for each group. Functional status dimension scores were computed using an algorithm provided with the CHQ. Raw score means were transformed to a 0 to 100 score, and the scale scores were standardized against a sample of the general US population using a z score transformation and then compared between intervention and control families.
Baseline characteristics of groups were compared with parametric (t test) and nonparametric (Kruskal-Wallis) tests for continuous measures, and χ2 and Fisher's exact tests for categorical measures. Changes over time and differences in changes over time between intervention and control groups were assessed through Poisson regression and 2-way analysis of variance using models of the form outcomei = constant + β1 time (0,1) + β2group (0,1) + β3 time × group + … (covariates)… Analyses of covariance were also used to test for relative difference in outcomes between groups adjusting for baseline measures and produced identical results. All data analyses were performed using STATA statistical software; all tests of statistical significance were 2-sided.
During a 10-month enrollment period, parents of 471 children were approached for inclusion in the study, and 137 (25%) consented to participate and completed the baseline assessment instrument. Of these, 118 were from the hospital-based clinic, and 19 were from the health center; 90 (66%) were 7 years and older, and 47 (34%) were younger than 7 years old. No differences were found in age or proportion covered by private insurance (P = .199) between those who enrolled in the study and those who did not.
Time commitment was the reason given for refusal to participate by about four-fifths of parents. Other reasons included acute illness of patient, need to go the emergency department, and lack of interest in more asthma management information. Control and treatment group populations were similar in demographic (age, gender, race, insurance coverage), illness (parents reported and functional severity, age of first asthma diagnosis, and missed school or day care days), home environment, and acute utilization (emergency visits in past year) characteristics (Table 1).
One hundred six parents (49/61 [80%] control and 57/76 [75%] treatment; P = .540) completed the exit questionnaire a mean of 10 months after enrollment. Reasons for dropout from the study included being unable to locate the family or other nonresponse to the final survey (eg, refusal to complete the instrument). Baseline characteristics (gender, age, race, insurance, parent's report of severity, age of first asthma diagnosis, home environment, and health services use) of children who did not complete the exit questionnaire did not differ from those who did.
Although the protocol required parents and children to return for 3 visits, 61% of enrolled families returned for >1 visit (35/61, 57%, control and 48/76, 63%, treatment), and just under one-third (13/61, 21%, control and 24/76, 32%, treatment) came for the second return visit. There was no statistically significant difference in the number of sessions between the 2 groups.
Children who participated in either arm of the study were improved during the year after enrollment compared with the period before entering the study, even when controlling for age (Table 2). Substantial reductions occurred in emergency department and unscheduled office visits for asthma, reported asthma severity, and the impact of asthma on parental personal time. Parents in both groups also reported improvements in child behavior and use of peak flow meters (Table 2). However, only in the area of asthma knowledge did those randomized to receive the computer-based educational program have significantly better results than those receiving the noncomputerized education program. Those who used the computer education improved from 60% to 77% correct with each game use, while those reviewing written materials with the research assistant changed from 57% to 63% correct (P < .001). Participants in both groups also reported moreenvironmental triggers in their homes at exit than at baseline. Study participants did not differ in health care use, health status, ratings of care, or other outcome measure from controls.
Qualitative Finding—Approval and Enjoyment
All of the children in the treatment group reported that they enjoyed using the game (N = 75). The average time to complete the game for a first-time player was 45 to 60 minutes. The time decreased to approximately 30 to 45 minutes for second time players aged 6 years and older.
Children age 8 and older generally were able to master the skills required to operate the game after several minutes of coaching. Younger children required more coaching, and 3- and 4-year-olds were guided in the use of the program by a research assistant. These younger patients focused mostly on the mechanics of moving Spacer to his destination.
A majority of parents felt uncomfortable playing the game along with their children, regarding it as “kids' stuff.” Some watched and occasionally offered their comments, while others sat in a separate room until the session was over. Parents generally enjoyed the educational videos that were incorporated into the program, whereas children found that these interfered with the game aspects of the program.
We developed an innovative educational software program to provide children with knowledge, skills, and decision-making practice about optimal asthma management. We tested this program through a randomized, controlled trial based at 2 inner-city primary care sites, comparing the effects of using the program with the supervised review of standard educational materials. Although practically all outcomes improved substantially for all study participants, these improvements in outcomes did not differ between the intervention and control groups except for a greater improvement in child knowledge among intervention recipients.
What could explain the substantial improvements observed in both treatment groups? First, although severity of illness was not a criteria for entry into the study, parents of children who were having difficulty with their asthma probably were more likely to enroll. Although we did not specifically track the type of visit at which recruitment occurred, we believe most children were recruited either at the time of an acute exacerbation or, more commonly, in the first follow-up visit after such an exacerbation. Despite widespread publicity about the study at the clinical setting, and despite personalized letters to parents of children coming in for scheduled visits if the child met study criteria, few children were identified at well visits or scheduled asthma maintenance encounters. Statistical regression to the mean, ie, the phenomenon whereby observations identified due to their extreme values at one point in time tend to have a more normal value on a subsequent measurement, could account for some improvement. In addition, children enrolled aged during the study period; for many children, particularly those who are younger, symptoms may diminish with increasing age. Improvements in symptoms were not found to be associated with age, though, in our study sample. Because the exit questionnaire was completed less than one full year after enrollment, seasonal characteristics may have contributed to the child's baseline condition and not be present at the time of the exit survey. Administrative data for one whole year after enrollment, however, also showed a similar pattern of improvement.
Our findings are also consistent with the hypothesis that both treatment and control groups received effective education, and the education improved outcomes. Control group families received written educational materials, which were reviewed with the family by the research assistants. If individualized education and the software program are equally effective, then the computer-based approach may be more cost-effective in that ongoing individual teaching is not needed (except with younger children).
Another possible explanation might be that because this intervention took place shortly after dissemination of the NHLBI asthma guidelines, the findings may reflect improvements in care secondary to guideline dissemination. Indeed, a concurrent initiative was underway at the hospital-based program to enhance adherence to the NHLBI guidelines. A separate assessment of that program demonstrated only minor improvements in medical management over time, with enhanced peak flow use alone attributable to the specific hospital-based program intervention. The substantial increase in peak flow meter use in both study groups is consistent with this explanation.
The limited sample size of this trial may have prevented us from detecting potentially meaningful differences in the primary outcome. In particular, given the baseline rate of acute health care use and the large reduction in use observed among the control group (nearly 70%), we could have only detected reductions of 92% to 100% (ie, almost the complete elimination of such visits) with a power of .80. However, the consistent lack of effect across several measures suggests the sample size alone did not account for our not finding an effect.
The potential effectiveness of the software intervention may have been limited by the manner in which it was used in the study. Typically children use software games repeatedly, learning the subtleties by trial and error or by discussions with peers. Because of this program's requirements for what at the time of the study was relatively advanced computer technology, and our focus on the inner-city population, children were only able to use the program at the clinic. Physical constraints at the clinic, typical of those in inner-city settings, made it difficult to provide quiet locations in which children could focus on the program. The requirement that the program be used only in the clinic, and our limited ability to compensate families for attending, also led to fewer visits and less intense use of the program than would have been desirable.
For this program to be effective by itself, either enhanced child self-management must improve outcomes, or the program must affect parental behavior as well. The impact on parental behavior would have to occur either through parents' direct involvement in the educational activity or through the child's ability to alter their parents' behavior. In retrospect, it seems unlikely that enhancing child self-management alone, without affecting parental behaviors as well, would affect outcomes, particularly among younger children. Improved asthma care involves a broader partnership than child activities alone. We had anticipated that parents of younger children with asthma would actually use the program themselves, but we found that the program did not consistently engage parents—except the more didactic video components. The physical and cognitive tasks of the program were too complex for younger children (eg, under 6 years) to use the program alone. Among families with older children, parents did not always respond to the suggestions raised by their children based on the child's experience with the program. Some parents could not afford or were unable to remove environmental factors such as wall-to-wall carpeting in their public housing residences. Others who were capable of removing some allergens or irritants sometimes denied that a particular environmental exposure, such as their smoking, was harmful to the child. These observations strongly suggest that parallel educational activities focusing on parents as well as their children may be needed, as reflected in some asthma educational curricula, eg, A+ asthma club.10
The potential effectiveness of an educational program targeting patients with asthma is also bounded by the quality of care provided to these children and their families' real ability to alter their environment. Thus, even though the program demonstrates that the asthma hero functions better with antiinflammatory medicines, the child needs to be prescribed that medication to be able to take it. Similarly, even though the program recommends extermination of cockroaches, inner-city children are dependent on their landlord's willingness to implement these activities.
Interactive, educational software may yet play a useful adjunctive role in improving outcomes for children with chronic conditions. Such software, properly designed, is effective in conveying information and in providing opportunities for children to safely experience the consequences of different self-management activities. Use of such programs could be cost-effective if it offsets time spent by clinical staff on educational activities. Nonetheless, such educational programs are but one small part of the solution for complex medical and social problems such as asthma in the inner city. More comprehensive approaches that may incorporate such innovative components, but also address parental behaviors and skills, medical care, and the physical environment, are more likely to achieve success in addressing this important health problem.
This project was supported by Grant U01-AI31541 and Grant U19-AI31541 from the National Institute of Allergy and Infectious Disease, National Institutes of Health.
We gratefully acknowledge the contributions of the following to the performance of this study: Terry Fenton, PhD, biostatistics; Cordelia Mgoqi, research assistant; Rachel Bird and Judy Shaw, RN, MPH, chart review and completion of software; Raif Geha, MD, overall principal investigator; Robert Goldstein, MD, Ernestine Smart, RN, Marshal Plaut, MD, and Peter Gergen, MD, MPH, NIAID, for their support; Pat Bartels, RN, Bill Bithoney, MD, and the staff of CCHP; Karen Darcy, Joseph Carillo, MD, and the staff of Martha Elliot Health Center; the Pierce School, Brookline, MA (Kris Field, Principal) for assistance in pilot testing; Janet Hafler, EdD, for assistance in qualitative research; and Carmi Margolis, MD, Kevin Weiss, MD, Donald A. Goldmann, MD, and Judith Palfrey, MD, for their many suggestions and support throughout the study.
- Received May 19, 1999.
- Accepted March 17, 2000.
- Address correspondence to Charles Homer, MD, MPH, National Initiative for Children's Healthcare Quality, Institute for Healthcare Improvement, 135 Francis St, Boston, MA 02215.
↵† Dr Susskind is a consultant and on the faculty of the Institute for Healthcare Improvement.
- CHQ =
- Child Health Questionnaire •
- NHLBI =
- National Heart, Lung, and Blood Institute
- ↵National Asthma Education and Prevention Program. Guidelines for the Diagnosis and Management of Asthma, Report 2. Bethesda, MD: National Heart, Lung and Blood Institute; 1997. NIH Publication No. 98-4051
- Clark N,
- Gong M,
- Schork MA,
- et al.
- ↵Bandura A. Self-Efficacy in Changing Societies. New York, NY: Cambridge University Press; 1995
- Rubin D,
- Leventhal J,
- Sadock R,
- et al.
- ↵Landgraf J, Abetz L, Ware J Jr. Children Health Questionnaire (CHQ): A User's Manual. Boston, MA: The Health Institute, New England Medical Center; 1996
- ↵National Asthma Education Program (NAEP). Guidelines for the Diagnosis and Management of Asthma. Bethesda, MD: Office of Prevention and Control, National Heart, Lung, and Blood Institute; 1991. NIH Publication No. 91-3042
- Copyright © 2000 American Academy of Pediatrics