BACKGROUND AND OBJECTIVE: Monitoring health-related quality of life (HRQoL) by using electronic patient-reported outcomes (ePROs) has been only minimally evaluated in pediatrics. Children with juvenile idiopathic arthritis (JIA) are at risk for HRQoL problems. The aim of this study was to investigate the effectiveness of ePROs in clinical pediatric rheumatology care.
METHODS: All children (aged 0–18 years) with JIA visiting any of the 4 pediatric rheumatology clinics in Amsterdam between February 2009 and February 2010 were eligible for this sequential cohort intervention study. Before an outpatient consultation, children (aged 8–18 years) or parents (of children aged 0–7 years) completed web-based questionnaires. The resulting ePROfile was provided to the pediatric rheumatologist (PR). The study was divided into a control period in which the ePROfile was not discussed during consultation, and an intervention period in which the ePROfile was provided and discussed during consultation. Effectiveness was evaluated in terms of communication about different HRQoL topics, referral to a psychologist, and satisfaction with the consultations.
RESULTS: Out of the eligible JIA patients, 176 (65%) participated in the study. Use of the ePROfile increased discussion of psychosocial topics (P < .01), as well as the PR’s satisfaction with provided care during consultation (P < .01). The use of ePROfiles did not affect referrals to a psychologist or parental satisfaction. Parents and PRs evaluated the use of the ePROfile as positive in 80% to 100% of the consultations.
CONCLUSIONS: Our web-based application to systemically monitor HRQoL problems in pediatric rheumatology contributed significantly to communication about psychosocial issues in a positive way. We recommend implementation of ePROs in pediatric clinical practice.
- health-related quality of life
- electronic patient reported outcomes
- juvenile idiopathic arthritis
- clinical practice
- CHAQ —
- Childhood Health Assessment Questionnaire
- ePRO —
- electronic patient-reported outcome
- HRQoL —
- health-related quality of life
- JIA —
- juvenile idiopathic arthritis
- PedsQL —
- Pediatric Quality of Life Inventory
- PR —
- pediatric rheumatologist
- PRO —
- patient-reported outcome
- TAPQOL —
- TNO-AZL Preschool Children's Health-Related Quality of Life
- VAS —
- visual analog scale
What’s Known on This Subject:
Monitoring and discussion of patient-reported outcomes in clinical practice facilitates good communication between the patient and treating physician. However, studies on the use of health-related quality of life patient-reported outcomes in pediatric clinical practice are scarce.
What This Study Adds:
A web-based application to systemically monitor health-related quality of life problems in pediatric rheumatology is effective in increasing awareness of psychosocial topics of the patient and the patient’s family, contributing to increased satisfaction of the pediatric rheumatologist with the care provided during consultation.
Juvenile idiopathic arthritis (JIA) is arthritis of unknown etiology with onset before age 16. It is the most common rheumatic disease in childhood, affecting 0.07 to 4.01 per 1000 children worldwide1 and is a major cause of childhood disability. These children are at risk for a lower health-related quality of life (HRQoL) compared with healthy peers.2–7 To obtain comprehensive insight into the consequences of their disease, the use of physical measures is not sufficient. To provide effective treatment and support, it seems essential to evaluate the patient’s HRQoL in clinical practice.5
The use of HRQoL patient-reported outcomes (PROs) in daily clinical practice has received increased attention. Routine assessment of HRQoL in a clinical setting facilitates early recognition of lowered HRQoL in adult patients.8–14 Physicians generally regard the use of PROs as a valuable addition to daily health care.10 Research on the use of PROs to improve patient satisfaction with the care they receive is still challenging.8,11,13,15
In the context of a chronically ill child’s development, all domains of HRQoL (physical, social, emotional, and cognitive functioning) need to be addressed, and the developmental trajectory needs monitoring. Using PROs in daily clinical practice can be a helpful tool to discuss these issues systematically and provide tailored interventions if needed. However, research on the use of PROs in pediatrics is still scarce.16–18 One study showed that the use of PROs improved care satisfaction in adolescents with diabetes.18 Engelen et al demonstrated that PROs increased discussion of emotional and psychosocial functioning and improved identification of emotional problems in pediatric oncology.17,19,20
The aim of this study was to investigate the effectiveness of a web-based intervention that provided an electronic PRO (ePRO) to the pediatric rheumatologist (PR) during consultation. The ePRO which was completed at home just before consultation includes generic HRQoL scores and disease-specific information on the functional ability of the patient with JIA. Outcomes of effectiveness were communication about HRQoL, referrals, and satisfaction with the provided care. In addition, users evaluated the ePRO.
All children (aged 0–18 years) with JIA who visited the following pediatric rheumatology centers in Amsterdam were approached: Emma Children’s Hospital AMC, VU Medical Centre, Reade (Jan van Breemen location), and the Sint Lucas Andreas Hospital. All 5 PRs at the 4 centers participated.
The intervention consisted of providing HRQoL scores of the patient (ePROfile, Fig 1) and scores on functional ability (ePROfile, Fig 2) to the PR during consultation, focusing on identifying, monitoring, and discussing HRQoL problems. For that purpose, the child or parent completed online questionnaires just before consultation, at home by using the KLIK Web site (www.hetklikt.nu). The answers were then automatically converted to an ePROfile and keyed to colors, with red (“often” and “almost always”) indicating that a child experienced problems with an issue, orange (“sometimes”) representing mild problems, or green (“never” and “almost never”) indicating no reported problems. The development of the ePROfile has taken several steps, as described in Engelen et al 201019 and Haverman et al 2011.21
The PR retrieved the patient’s ePROfile directly from the Web site. The items and sum scores were shown on the computer screen and provided the opportunity to discuss the results directly with the child and parents.
Design and Procedure
A sequential cohort design was used.19,21 Patients took part in either the control or intervention group, depending on the date of consultation; if a patient participated in the control group, that patient was no longer eligible for participation in the intervention group. All PRs first participated in the control group (control period: February 2009–April 2009; online questionnaire completed by the patient, but the ePROfile was not provided to the PR) and thereafter in the intervention group (intervention period: May 2009–February 2010; online questionnaire completed by the patient, and the ePROfile was provided to the PR). The intervention period was longer than the control period because we had the opportunity to extend the research project. As a result, more patients were included in the intervention group compared with the control group, and therefore in the intervention group, the ePROfile could be discussed in 2 consecutive consultations.
Children and parents were invited by mail to participate in the study when the consultation date was planned. They received a username and password by e-mail to log in on the KLIK Web site.21 Each user of the Web site has a unique login name and is automatically given access to a specific secure section of the Web site. The Web site only allows the pediatricians to see the results for their own patients, and therefore, the privacy of the patients is guaranteed. The security and functions of the KLIK Web site are extensively described in Haverman et al 2011.21
To assess the effectiveness of the intervention, shortly after the consultation, the parents and the PR completed an online questionnaire about the HRQoL topics discussed, referrals, and their satisfaction with the consultation, again by using the Web site. In addition, parents and PRs evaluated the ePROfile in the intervention period.
All patients and/or parents gave informed consent, and the medical ethics committees of all participating centers approved the study.
The Netherlands Organisation for Applied Scientific Research-Academisch Ziekenhuis Leiden (TNO-AZL) Preschool Children Quality of Life (TAPQOL)22,23 was used to measure parent-reported HRQoL in children aged 0 to 5 years. The TAPQOL is a Dutch proxy report, generic, and multidimensional HRQoL instrument. The 43 items are clustered into 12 multi-item scales, with higher scores indicating better HRQoL. The completion time is ∼5 to 10 minutes, and the psychometric properties are satisfactory. The Pediatric Quality of Life Inventory (PedsQL) Generic Core Scale24,25 was used to measure generic HRQoL in older children: PedsQL self-report form (children aged 8–18) and PedsQL parent-report form (children aged 6 and 7).
Disease-specific symptoms were assessed with a self-composed questionnaire (Table 1) based on the DISABKIDS arthritis module.26 The answers to these items were only reported in the ePROfile. The data were not used for sum scores or group comparisons.
The Dutch version of the Childhood Health Assessment Questionnaire (CHAQ)27 was used to measure functional ability.28 A 100-mm visual analog scale (VAS) was used for the evaluation of pain and overall well-being.
Communication About HRQoL
After every consultation, the PRs and the parents scored whether the following 12 HRQoL topics were discussed (yes/no): Condition/Energy, Sports, Autonomy, Pain/Physical Problems, Use of Medicine, Sleep, Eating/Drinking, Emotions, Family, Contact With Peers, School and Cognition, and Attendance at School. These topics represented the 4 HRQoL domains in the ePROfile.
PRs reported referral to a psychologist after each consultation. Scores were on a dichotomy scale (0 = no referral, 1 = referral).
After the consultation, the satisfaction of the PR and the parents was assessed with the adapted version of the Patient Satisfaction Questionnaire.19,29,30 The questions, answered on a 100-mm VAS (0 = not at all satisfied to 100 = very satisfied), referred to satisfaction with respect to (1) meeting needs, (2) active involvement, (3) provided information, (4) (emotional) support, and (5) overall satisfaction with care provided during the consultation. Parents completed the 5 questions about the functioning of the PR and the overall consultation; PRs completed these questions for both child (3 items) and parent (3 items) and overall satisfaction (1 item).
Evaluation of the ePROfile
After the first and second consultations in the intervention period, the parents, children, and PRs completed an ad hoc designed 5-point (PRs and parents) or 3-point (child) scale questionnaire about their experience with the ePROfile. Parents and PRs were asked if they regarded the ePROfile as useful, unnecessary, clarifying, practical, emotional, or unpleasant (5-point scale; “totally agree” to “disagree”). Children (≥8 years) were asked if discussing their answers with the pediatrician was nice, unpleasant, normal, tense, stupid, delightful, or sad (3-point scale; “yes,” “a little,” and “no”).
Parents were also asked if the ePROfile helped them or their child gain insight into the child’s HRQoL (5-point scale; “totally agree” to “disagree”) and if the ePROfile correctly reflected the HRQoL of their child (“yes” or “no”). In addition, we asked them to rate the use of the ePROfile during the consultation on 10-point scale: 0 = very negative to 10 = very positive.
Sociodemographic and Medical Information
Parents completed an online questionnaire about sociodemographics at the start of the study. Medical data were assessed by the PR during the consultation. Some sociodemographics (child age and gender, parental country of birth) and medical information about nonparticipants were retrieved from medical files.
SPSS version 18.0 (IBM SPSS Statistics, IBM Corporation, Armonk, NY) was used. First, sociodemographic and medical differences between participants and nonparticipants were tested as well as differences between the control and intervention group, by using independent t tests (age), χ2 test (age groups, gender, parental country of birth, JIA subtype, medication, PRs), and Mann-Whitney tests (physician disease activity rating: VAS score, disease duration). Differences in HRQoL scores between control and intervention groups were analyzed with independent t tests and the difference in CHAQ scores with the Mann-Whitney test.
Second, the effect of the intervention (ePROfile) was examined by testing differences between the control and intervention group on the outcome measures. χ2 tests were used for communication (topics discussed) and referrals; satisfaction was analyzed by using analysis of variance by group and age. Descriptive statistics were used for the evaluation of the ePROfile. To adjust for multiple testing, we used a P value of <.01.
Two hundred seventy-three children with JIA and their parents were approached to participate in this study. The patients were included if the questionnaires were completed before consultation and either the parents or the PR completed the questionnaires after consultation. Eventually, 176 (64.5%) patients completed the online questionnaires; the mean age for patients was 11.6 (SD 4.5) years and 66.8% were girls. Participating patients were younger than the nonparticipants (mean 11.6 vs 13.22; P < .01), and, as a result, disease duration (years) in the participants was shorter (4.0 vs 4.8; P < .05). The participating parents were born in the Netherlands more often (79.5% vs 66.0%; P < .01) compared with the nonparticipants.7
During the control period, 67 patients participated. In the intervention period, 109 patients participated; 40 patients had 1 consultation, and 69 patients had 2 consultations.
Despite differences in age, the distribution of the different age categories between the control and intervention group did not differ significantly (Table 2).
Regarding HRQoL in patients aged 0 to 5 years, no differences were found between the control and intervention groups. However, the patients aged 6 to 18 years in the intervention group reported better functioning in school compared with the patients aged 6 to 18 in the control group (mean 63.9 vs 71.7; P < .05).
Regarding disease specific symptoms of JIA, functional ability (CHAQ), and pain score (VAS), no differences were found between the control and intervention group. However, the control group reported a worse overall well-being score (VAS; mean 27.7 vs 26.6; P < .05).
All PRs participated in the control as well as the intervention groups. The 5 PRs managed, respectively, 9%, 16%, 20%, 21%, and 34% of the patients. No significant differences in the distribution of the patients over the PRs were found between the control and the intervention group.
Communication About HRQoL
According to the parents, the topics of “emotions” and “family” were discussed more often in the intervention group (i2) than in the control group (Table 3).
In addition, according to the PRs, the topics “family” and “contact with peers” were more often discussed in the intervention group (i1 and i2) than in the control group. “Emotions” were discussed more often in the first consultation compared with the control group.
The PR referred 3.0% of the control patients to a psychologist compared with 9.2% of the patients during the first intervention consultation (i1) and 4.3% during the second intervention consultation (i2).
Parents’ satisfaction with the care provided during consultation in the intervention group did not differ from that in the control group (Fig 3).
The PRs reported that they were more satisfied with the provided care during the consultations in the domains of emotional support for parents and child, meeting the needs of the child, and in overall satisfaction with the provided care during the consultation in the first intervention consultation (i1) and the second intervention consultation (i2; Fig 4) compared with the control period. The PRs also reported to be more satisfied about the provided information to the child in the second intervention consultation (i2) compared with the control period.
Evaluation of the ePROfile
Overall, the evaluation of the use of the ePROs was positive. In 88% (i1) and 80% (i2) of the consultations, the parents regarded the ePROfile as useful. In 57%, they regarded the ePROfile as helpful for themselves and in 60% of the consultations as helpful for their child. In 94% of the first intervention consultation (i1) and in 91% of the second intervention consultation (i2), parents judged that the PROfile reflected their child’s HRQoL adequately. The parents (i1 and i2 combined) evaluated the PROfile with a median of 8 on a 10-point scale (range 4–10).
The PRs regarded the provided PROfile as useful in 95.1% of intervention consultations 1 (i1) and 100% of intervention consultations 2 (i2).
The children reported that discussing the ePROfile with the PR was “normal” in 97.9% of the first intervention consultations (i1) and in 82.8% of the second intervention consultations (i2) (Table 4).
Our study shows that providing information to PRs on patient HRQoL, disease-specific symptoms, and functional ability with an ePROfile leads to significantly more discussion of emotional and social functioning during consultation and improves the PR’s satisfaction with the provided care. The use of the ePROfile during consultation seems to result in more psychosocial referrals. Overall parents, children, and PRs evaluate the use of the ePROfile positively.
Our findings of the use of the ePROfile regarding communication correspond with the findings in previous PRO studies in adult8,10,31 and pediatric17 medical settings. With the use of an ePROfile, emotions, family, and contact with peers were discussed significantly more often during consultations than without the use of an ePROfile. It is important to discuss HRQoL systematically in daily clinical practice to identify problems early. Timely, tailored interventions by psychologists or social workers can then be offered when needed. Although solving problems in social and emotional functioning is not the main scope of a pediatrician, the discussion of HRQoL topics can itself lead to a decrease in negative feelings or feelings of insecurity.32
Despite a trend toward more psychosocial referrals (3% of the control consultations compared with 9% of the intervention consultations), the use of the ePROfile did not result in a significant increase in the percentage of referrals. This corresponds with the findings in earlier studies.8,10,17 Overall, based on systematic reviews, PROs are seen as most efficient and effective in improving communication about psychosocial issues.33,34
The PRs reported increased satisfaction with the provided care during consultation if the ePROfile was used. This finding is confirmed by the results showing that PRs regarded the ePROfile as useful in almost all consultations (95%–100%). This positive evaluation of the ePROfile by health care workers is in line with former findings in adult health care,8,9,35 but to date, this has not been reported in pediatric clinical care.
The results regarding parental satisfaction with the provided care during consultation were not unequivocal. Although the majority of the parents regarded the ePROfile as useful in the consultation (80%–88%) and parents evaluated the ePROfile with a high score (8 out of 10), the use of the ePROfile did not seem to influence parental satisfaction. This may be due to a ceiling effect,10,15 which is indicated by the high satisfaction score in the control group. This phenomenon was also observed in other studies evaluating the use of PROs on patient satisfaction,10,17 whereas the study of De Wit et al (2008) recorded improved satisfaction with care in adolescents with diabetes by including HRQoL scores in routine clinical practice.18
Taken together, our results show that using ePROs for children with JIA was appreciated by parents and PRs and that its use has a positive effect on communication about psychosocial issues and on PR satisfaction with provided care. A strength of our ePROfile is that it includes all relevant domains for the provision of clinical care to children with JIA.36,37 It is well documented that children with JIA are at risk for a low HRQoL and that pain, physical functioning, school absence, and the subjective burden of medication use are the main predictors of their HRQoL.7 The ePROfile incorporated all these aspects as well as information on specific symptoms of JIA and general well-being.
A limitation of our study was the possible confounding influence of the provision of the questionnaires to the control group. Therefore, our control group can also be considered an attention control group. In future research, it would be interesting to add a control group under only standard care without the use of questionnaires.8,9 Furthermore, it should be considered that we did not include HRQoL as an outcome measure. In our opinion, the duration of our study was too short to establish any differences in HRQoL. In addition, to avoid contamination, we did not randomize our study participants. A sequential cohort design was considered most suitable for the KLIK study.17 A fourth limitation is that we did not determine the association between HRQoL scores and communication. In future research, it would be interesting to investigate the influence of HRQoL scores on the items discussed by, for example, audiotaping the consultations.
Because all patients with JIA in care in the district of Amsterdam were invited to participate in this study and only small differences in characteristics were found between participants and nonparticipants, we believe that our findings can be generalized to all children with JIA. Furthermore, our findings are in line with previous results in pediatrics17,18 and in adult oncology.8,10,12 We therefore think that the implementation of the ePROfile in pediatrics is justified in daily clinical practice for children with different types of diseases.
The ePROfile is now being implemented in clinical care in our hospitals. The implementation of a web-based ePROfile in clinical practice creates new challenges and opportunities for care, as is extensively described in the International Society for Quality of Life Research guidelines and in other studies.38 In our study, the use of the ePROfile was feasible. In the Netherlands, which has the highest rate of Internet access in Europe, the use of the Internet to provide PROs39 is an efficient medium for monitoring HRQoL in daily clinical practice. The completion of the HRQoL questionnaire takes only 10 minutes,40 and discussing HRQoL issues in clinical practice does not seem to increase the duration of the consultation.10,17,31 Depending on a patient’s health care needs, members of a multidisciplinary team can use the ePROfile individually or during multidisciplinary team consultation. In addition, parent-reported outcomes can be useful supplementary components in the care of chronically ill children and their families.41,42
Our multicenter study is a valuable addition to the few studies on feedback using ePROs in pediatric settings. In conclusion, a web-based application to systemically monitor HRQoL problems in pediatric rheumatology is effective in increasing discussion about psychosocial topics and satisfaction with the provided care by the PR. Therefore, we recommend implementation of the ePROfile in daily clinical practice, to detect HRQoL problems at an early stage and to provide timely and tailored interventions.
We thank all the children, parents, nurses and nurse practitioners, and the other medical staff who participated in the KLIK study. In addition, we thank Heleen Maurice-Stam for her statistical support and her valuable comments on the manuscript, BioMedia/Artsen voor Kinderen for the web design used in this study, and Hedy van Oers and Perrine Limperg for supporting the implementation of the ePROfile in daily clinical practice.
- Accepted September 25, 2012.
- Address correspondence to Lotte Haverman, MSc, Emma Children’s Hospital AMC, Meibergdreef 9, Room A3-241, 1105 AZ Amsterdam, Netherlands. E-mail:
All authors made substantial contributions to the conception and design of the study and to the interpretation of data. Ms Haverman collected the data. The analyses were performed by Ms Haverman and Dr Grootenhuis. All authors helped to draft the article or review it critically for important intellectual content. All authors gave final approval of the version to be published.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by Pfizer BV Pharmaceuticals and AGIS Healthcare.
- Prince FH,
- Otten MH,
- van Suijlekom-Smit LW
- Gutiérrez-Suárez R,
- Pistorio A,
- Cespedes Cruz A,
- et al.,
- Pediatric Rheumatology International Trials Organisation (PRINTO)
- LeBovidge JS,
- Lavigne JV,
- Donenberg GR,
- Miller ML
- Sawyer MG,
- Whitham JN,
- Roberton DM,
- Taplin JE,
- Varni JW,
- Baghurst PA
- Haverman L,
- Grootenhuis MA,
- van den Berg JM,
- et al
- Velikova G,
- Booth L,
- Smith AB,
- et al
- Velikova G,
- Keding A,
- Harley C,
- et al
- Gutteling JJ,
- Darlington AS,
- Janssen HL,
- Duivenvoorden HJ,
- Busschbach JJ,
- de Man RA
- de Wit M,
- Delemarre-van de Waal HA,
- Bokma JA,
- et al
- Fekkes M,
- Bruil J,
- Vogels T
- Wulffraat N,
- van der Net JJ,
- Ruperto N,
- et al.,
- Paediatric Rheumatology International Trials Organisation
- Filocamo G,
- Consolaro A,
- Schiappapietra B,
- et al
- Guyatt GH,
- Ferrans CE,
- Halyard MY,
- et al.,
- Clinical Significance Consensus Meeting Group
- Varni JW,
- Limbers CA,
- Burwinkle TM,
- Bryant WP,
- Wilson DP
- Committee on Hospital Care and Institute for Patient- and Family-Centered Care
- Copyright © 2013 by the American Academy of Pediatrics