Automated Conversation System Before Pediatric Primary Care Visits: A Randomized Trial
BACKGROUND AND OBJECTIVES: Interactive voice response systems integrated with electronic health records have the potential to improve primary care by engaging parents outside clinical settings via spoken language. The objective of this study was to determine whether use of an interactive voice response system, the Personal Health Partner (PHP), before routine health care maintenance visits could improve the quality of primary care visits and be well accepted by parents and clinicians.
METHODS: English-speaking parents of children aged 4 months to 11 years called PHP before routine visits and were randomly assigned to groups by the system at the time of the call. Parents’ spoken responses were used to provide tailored counseling and support goal setting for the upcoming visit. Data were transferred to the electronic health records for review during visits. The study occurred in an urban hospital-based pediatric primary care center. Participants were called after the visit to assess (1) comprehensiveness of screening and counseling, (2) assessment of medications and their management, and (3) parent and clinician satisfaction.
RESULTS: PHP was able to identify and counsel in multiple areas. A total of 9.7% of parents responded to the mailed invitation. Intervention parents were more likely to report discussing important issues such as depression (42.6% vs 25.4%; P < .01) and prescription medication use (85.7% vs 72.6%; P = .04) and to report being better prepared for visits. One hundred percent of clinicians reported that PHP improved the quality of their care.
CONCLUSIONS: Systems like PHP have the potential to improve clinical screening, counseling, and medication management.
- interactive voice response
- electronic health record
- patient activation
- health information technology
- primary care
- routine health care maintenance
- well-child visit
- EHR —
- electronic health record
- GEE —
- generalized estimating equations
- IVR —
- interactive voice response
- PHP —
- Personal Health Partner
- RHCM —
- routine health care maintenance
What's Known on This Subject:
A substantial gap exists between what is recommended for effective primary care of children and what takes place. Patient-centered health information technologies have been used to gather information and counsel parents, however, have not been integrated directly with electronic health records nor been speech-based to improve decision-making at the point-of-care.
What This Study Adds:
This study shows that a ubiquitous technology, the telephone, can be successfully used to automatically assess and counsel parents before pediatric primary care visits as well as inform their primary care clinicians in a way that is feasible and effective for multiple important issues.
A substantial gap exists between what is recommended for effective primary care of children and what actually takes place in pediatric primary care settings.1–4 Patient-centered health information technologies seek to provide solutions that are respectful of and responsive to individual patient preferences, needs, and values and have been used successfully to gather information and counsel parents.5,6 However, such systems have not been integrated directly with an electronic health record (EHR) system to improve clinical decision-making and efficiency at the point of care.7–10
Interactive voice response (IVR) technologies offer a potentially effective, patient-centered communication modality by guiding parents at home through interactive, speech-based discussions that can gather information and reinforce recommendations and treatments. IVR systems are particularly well suited for use by vulnerable populations, because access to the telephone is nearly universal, and such systems can be used by many people who would have difficulty with printed materials.11,12
For this project, we developed and evaluated an IVR-based patient-centered health information technologies system, the Personal Health Partner (PHP). PHP uses synthetic speech and spoken responses and is designed to engage patients outside the clinical environment via telephone.
The primary purpose of this study was to determine whether use of PHP was associated with significant differences in parental report of primary care visit content. Additional goals included evaluating the intervention effect on (1) medication management, (2) asthma care, and (3) parent and clinician satisfaction.13
PHP is an automated conversational system used before routine health care maintenance (RHCM) clinic visits. Details of the system have been presented previously.14 The PHP uses high-quality synthetic speech and automatic speech recognition. PHP conversations use clinical data from the EHR and personal health data gathered during conversations to provide tailored counseling for parents before RHCM visits. Synthetic speech was used in all areas except medication reconciliation, for which synthetic speech accuracy was found to be insufficient. Medication reconciliation was implemented with speech files recorded by the project physician (20 hours estimated effort). Use of synthetic speech for the majority of the system substantially improved the flexibility, scalability, and time required to implement the system.
PHP scripts are published as voice-XML and hosted using open-source web-application server technology and voice-over Internet protocol technology.15 The system can accommodate as many simultaneous calls as needed. Calls were designed to be interactive and conversational. For example, if a parent screens positive for depression, PHP reminds the parent that their child’s doctor is interested in helping and asks the parent if they could agree to bring up their feelings of sadness during the upcoming visit. Each content area was developed as an independent module with a question set, decision rules for directing the conversation, and counseling topics that include activation messages so modules could easily be added or removed.
Data from PHP were transferred to the EHR (Centricity, General Electric, Fairfield, CT) via “patient-entered data” variables that are processed by the EHR at the time of the visit. All pediatric primary care visits at the study center use a single-visit EHR template, which was programmed to look for PHP data within the previous 30 days whenever the RHCM form was initiated and, when available, to (1) add the “Patient-Entered Data Review Form” to the EHR, (2) load PHP data into the form, and (3) prompt the clinician that “Patient-entered data are available” (Fig 1A). By clicking “Go,” the clinician can review and edit if needed (Fig 1B) and when done, click “Accept All” (Fig 1C) to add data to the clinic note. Accepted data were visible within standard RHCM templates, where they could be viewed and edited (Fig 1D). After 30 days, PHP data were no longer viewable by clinicians.
Selection and Description of Participants
Subjects were recruited from the Pediatric Primary Care Center at Boston Medical Center (BMC). The center serves an urban, predominantly lower socioeconomic status, ethnically diverse population. Children aged 4 months to 11 years who had an RHCM or well-child visit were eligible for the study. Parents and children had to speak English and could not be planning to move away from the Boston area within 3 months. All patients with an eligible visit between June 2009 and February 2011 were invited to participate by mailed brochure with a letter from the child’s primary care provider. The brochure and letter were written below a sixth-grade reading level and included (1) a description of the study, (2) the written consent form, (3) their study password to complete the call, and (4) staff contact information. Acceptance of the written consent was confirmed during the automated call; if requested, the consent was read over the phone to subjects. Recruitment letters were sent by mail 14 to 21 days before the visit. Parents who had not called by the week before the visit were contacted by phone once to determine whether the brochure had been received and to answer any questions. Parents were given gift cards for participation in the study. The Consort Diagram is shown in Fig 2.
Procedure and Design
This study began as a 3-arm study in which subjects were randomized to Control (IVR safety survey only), PHP without counseling, or PHP. However, after the first 205 patients, recruitment rates were not sufficient to achieve sufficient power in all 3 arms, so the PHP without counseling arm was discontinued and going forward patients who would have been assigned to the PHP without counseling were assigned to the full PHP intervention. Data for the PHP without counseling group has not been analyzed nor is this group included in any analyses in this report. Participants were asked to call the automated system at a time convenient to them and were randomly assigned to groups at the start of each call. Study staff called participants to complete follow-up data collection 5 to 7 days after the visit to assess outcomes. Study staff members were not aware of allocation group at the time of interviews. All parents who had depressive symptoms were called by the study physician after the visit to ensure adequate follow-up and care.
Attending pediatricians at BMC were introduced to the study with a brief presentation at a clinic provider meeting. All pediatricians agreed to participate. Recruitment activities occurred during a 22-month period during April 2009 to February 2011.
The study was approved by the Boston University Medical Center Internal Review Board.
PHP tailors call content based on the patient’s age and prescription of asthma medication. Call content was based on American Academy of Pediatrics Bright Futures topics reflected in the EHR templates at the study site as well as Medicaid-recommended health risk questions (Table 1) for RHCM, asthma symptoms, and medication safety.4 When available, PHP scripts were based on validated tools. RHCM areas include (1) general health supervision,16 (2) developmental screening,17 (3) diet and physical activity,4 (4) tuberculosis risk assessment,4 (5) smoking risk assessment,13 and (6) maternal depression screening.18 Each call also addressed medication safety examining (1) what medications on the EHR medication list the child was actually taking,19,20 (2) age-appropriate medication use,21 and (3) proper use of asthma controller and reliever medication if applicable. The day before each scheduled visit, PHP data were transferred to the EHR. PHP questions yielding actionable data generate an “Alert” displayed within the “Alerts” section of the “Patient Entered Data Review” form (Fig 1B).
Control parents also completed a single automated call but the content was limited to the 18-question Framingham Safety Survey.22 At the completion of the call, parents in the control group received tailored advice related to unsafe behaviors reported during the call. Because the Framingham Safety Survey was not part of routine primary care at BMC, data from these calls were not shared with the EHR.
Evaluation of the clinical effectiveness of PHP focused on analyses of post-visit parent interviews (Table 1). Because formal developmental screening was not possible via the telephone, parental report of developmental concerns was used as the primary developmental outcome. Assessments of safety-related behaviors (sleep position, seat belt use, and bike helmets) were not included in post-visit interviews. Prescription medication adherence was assessed by using 2 questions from a previously validated Medication Adherence Scale.21 Results from this assessment will be presented in a future manuscript. For medication reconciliation, PHP asked parents to gather all their child’s medications and PHP read the medication list from the medical record as parents reviewed bottle labels.20 During post-visit interviews, accuracy of the medication list was determined using a structured form that was preprinted with EHR data and then updated manually during the interview. Within the EHR, all changes were visible so research staff could identify what was recorded in the EHR as of the end of the visit. Parents were asked about all medication recorded in the EHR and whether the child was taking any additional medication not mentioned.
Parent and clinician satisfaction with PHP was assessed via questionnaire. For each question respondents were asked if they (1) strongly agreed, (2) agreed, (3) were neutral, (4) disagreed, or (5) strongly disagreed with each statement. For parents, questions were administered by research staff over the phone. Clinicians completed web-based surveys.
Generalized estimating equations (GEE) with a compound symmetric covariance structure and robust standard errors were used to compare child and parent demographic information between the Control and PHP user groups and to assess the efficacy of the PHP intervention with respect to post-visit assessments of parent-reported visit content. GEE methods were used to account for the intragroup correlation that resulted from measurements of multiple children per parent, whereas robust standard errors were used to protect against the potential misspecification of the covariance structure. The success of randomization was evaluated for each demographic characteristic. As a confirmatory analysis, GEE models were adjusted by any factors identified as unbalanced post-randomization. To assess usability and satisfaction, parental satisfaction with the system was also compared between the study conditions with χ2 tests. All analyses were conducted in Stata version 12 (StataCorp, College Station, TX) using intent to treat analyses and a significance level of 0.05.
There were 6910 potentially eligible pediatric patients who were mailed a letter inviting the patient’s parent (or primary caregiver) to participate. Six percent of letters were returned by the post office for wrong address. Of those not returned, 667 (9.7%) children had a parent call the PHP system and consent to participate, and 593 (8.6%) were included in the study allocation (Fig 2). Of those children whose parent called, 485 (82%) attended their appointment within 3 weeks of calling the PHP system, and 475 (98%) of these parents completed the follow-up interview (Intervention, n = 293; Control, n = 182). The average duration of Intervention calls (29.3 minutes, SD = 5.9) was longer than Control calls (17.3 minutes, SD = 5) (P < .001). Of those who completed the follow-up survey, no significant demographic differences were noted between PHP and Control parents. Most parents were female, 47% were African American, 44% had a college degree, and 48% were employed (Table 2). The mean age of children was 4.7 years (SD = 3.5). All demographic characteristics, other than child gender, were balanced by randomization.
PHP generated 658 alerts for a variety of primary care-related health issues with an average (SD) number of alerts per child of 1.4 (1.6) (range, 0–7) (Table 1). Parents varied in their willingness to endorse agreement to follow PHP advice. For dietary measures, 40% of children had an alert for watching >2 hours per day of television and 13% of parents agreed to try to reduce television time. Eighty-seven percent of parents reported that their children consumed <5 fruits or vegetables per day. Eighteen percent of parents screened positive for smoking and 22% agreed to call the local or national quit line. Twenty percent of parents screened were positive for depression; 18% of these parents agreed to bring up their feelings of sadness during the child’s visit. Twenty-four percent had increased risk for tuberculosis and 75% of the parents agreed to remind their doctor about tuberculosis screening. Eighty-six percent of parents reported that the EHR medication list included at least 1 medication their child wasn’t taking.
Parent-Reported Visit Content
PHP and Control parents reported similar rates of discussing television viewing and juice intake with their pediatrician (Table 3). PHP parents were significantly more likely to report discussion of depressive symptoms when the parent responded they sometimes felt depressed during the PHP call (52% vs 30%; P = .01) and tuberculosis risk when the child was at risk (37% vs 20%; P < .01). For children taking prescription medications, PHP parents were more likely to report discussing medication (86% vs 73%; P = .04) and to have brought their medication to the visit (19% vs 10%; P = .19). PHP and Control children did not differ with regard to the accuracy of the EHR medication list for current medications (Table 3). Supplementary analyses, adjusting for child gender (the sole characteristic found to be unbalanced after randomization), confirmed all findings (data not shown).
The majority of parents gave favorable reviews of both the PHP and Control IVR systems (Table 4). Parents liked each system because it could be used at home and was telephone-based. Only 40% would have preferred a web-based approach. Fewer PHP parents felt the length of the call was reasonable compared with Control parents (74% vs 90%; P < .001). However, PHP parents were significantly more likely to report feeling “more prepared” for the visit (81% vs 67%; P = .001). PHP parents were also significantly more likely to report that use of PHP reduced their visit time (63% vs 45%; P < .001). Nearly all parents in both groups would use the IVR systems in the future.
Clinicians were also very positive about using PHP (Table 5). Nearly all clinicians reported that PHP was easy to use, increased completeness of documentation, reminded them to do things they might otherwise forget, and improved efficiency during visits. Clinicians provided more mixed responses for medication management. Only 40% agreed that PHP helped with medication management and only 30% agreed that PHP improves medication safety.
In this study, we assessed an automated conversational system tailored to the challenges of an urban pediatric population. The PHP system successfully assessed many primary care domains that busy pediatric primary care clinicians seek to address on a routine basis. PHP detected important issues, engaged parents in pre-visit behavior change efforts, and shared information and alerts with clinicians and changed the content of the primary care provider encounter. Although PHP use was not associated with increased likelihood of parents reporting discussions in all domains, it was significantly associated with improvement in several very important areas, such as parental depression and tuberculosis screening.
There is a need to provide better, safer primary care in an affordable, sustainable, and effective manner. The great majority of American parents (94%) report unmet needs for parenting guidance, education, or screening by pediatric clinicians.23 Only 1 in 5 Medicaid-insured children receive adequate preventive services.24 To our knowledge, the PHP System is the first fully conversational system that supports pediatric assessment and parental counseling before visits in a way that is fully integrated with an EHR to support clinician communication and efficiency during office visits. The system was very well received by patients and clinicians and can serve as a model for linking patient-reported data collected outside clinical setting with the medical home via the EHR. The use of speech rather than text lowered the literacy burden of the system.25 Although the prevalence of internet connectivity is rapidly expanding, the telephone is nearly ubiquitous.
We compared the PHP System to an attention control group that also experienced an IVR system delivering the Framingham Safety Survey in a manner that was not linked to the EHR. Although parents welcomed both IVR conditions, PHP parents felt better prepared than Control parents for primary care provider visits. Importantly, control parents felt that the duration of calls was more reasonable compared with PHP parents. In the future we plan to shorten the duration of PHP calls by removing components that were not effective.
We believe that the next-generation of patient-centered primary care lies in technologies that engage patients outside clinical settings in a way that is tightly integrated with EHRs and primary care in a way that supports and activates patients. Systems that use spoken language rather than text on a screen (ie, a web page) have the potential to play an important role, because they use a format that many users find more engaging, motivating, and usable.26,27,28
An important limitation of this study is that primary outcomes were assessed by parent report. We did not directly monitor primary care provider - parent dialogue. Of note, parents did not report increased likelihood of discussing all topics assessed by PHP. This suggests that some combination of parent activation and/or clinician prompting led to an increased likelihood of addressing important topics like maternal depression. Furthermore, our primary focus was on assessing what a parent takes away from a visit rather than what a clinician documents as discussed. Another limitation was the low response rate. Although we cannot know with certainty, the low rate may have been attrubutable to 1 or more of the following: (1) use of direct mail, (2) complexity of informed consent language, or (3) general reluctance to participate in research. Studies in which patients use similar systems in the waiting room have had much higher response rates. One potential solution to improving response rates would be to evaluate systems like PHP in actual practice in which a phone call could be requested before visits and then, if not done, again at the time of visit registration and used in the waiting room.
The capacity of clinicians to provide all of the recommended services in the clinical setting has been exceeded. Systems like PHP can enhance adherence to guidelines and improve the availability of needed patient information. Counseling, which is tailored and based on existing clinical and patient-entered data, has the potential to promote and encourage healthy behaviors that in turn could support national efforts to address multiple evidenced-based primary care topics.
We acknowledge Bonnie Watson, MPH, Shikha Anand, MD, Michela George, MPH, and Robert Friedman, MD for assistance in study design.
- Accepted June 16, 2014.
- Address correspondence to William G. Adams, MD, Division of General Pediatrics, Boston University School of Medicine, 88 E Newton St, Vose 303, Boston, MA 02118. E-mail:
Dr Adams conceptualized the study, oversaw all activities during the study, performed initial analyses, and lead all aspects of manuscript preparation Dr Phillips participated in subject recruitment, data collection, analysis, and manuscript preparation Ms Bacic performed all final statistical analyses for the paper and lead the statistical content preparation for the manuscript Dr Walsh conceptualized, with Dr Adams, the portions of this study that focused on medication safety and participated in analyses and manuscript preparation in this area Dr Shanahan conceptualized, with Dr Adams, the portions of this study that focused on EHR-integration and participated in analyses and manuscript preparation all aspects of the manuscript Dr Paasche-Orlow conceptualized, with Dr Adams, the portions of this study that focused on patient-engagement, health literacy, and patient-activation and served as senior author for final manuscript preparation.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Agency for Healthcare Research and Quality, grant R18HS017248.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Adams WG,
- Mann AM,
- Bauchner H
- American Academy of Pediatrics
- Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents. Pocket Guide. 2nd ed, revised. Arlington, VA: National Center for Education in Maternal and Child Health; 2002
- Hagan JF,
- Shaw JS,
- Duncan PM
- Tucker JA,
- Simpson CA,
- Huang J,
- Roth DL,
- Stewart KE
- Christakis DA,
- Zimmerman FJ,
- Rivara FP,
- Ebel B
- Sturner R. The Child Health and Development Interactive System (CHADIS). Paper presented at the 7th Annual NIH SBIR/STTR Conference; July 28–29, 2005; Bethesda, MD
- Biondich PG, Downs SM, Anand V, Carroll AE. Automating the recognition and prioritization of needed preventive services: early results from the CHICA system. Paper presented at the AMIA Annual Symposium Proceedings; Washington, DC; 2005
- Olson AL,
- Gaffney CA,
- Hedberg VA,
- et al
- Handley MA,
- Shumway M,
- Schillinger D
- ↵US Preventive Services Task Force. Child and Adolescent Recommendations. Available at: www.uspreventiveservicestaskforce.org/tfchildcat.htm. Accessed July 10, 2014
- ↵Adams WG, Phillips BD, George M, et al. The Personal Health Partner: Conversational IT for Better, Safer Pediatric Primary Care. Poster presentation for the American Medical Informatics Association Conference. Washington, DC; 2010
- ↵Squires J, Bricker D. Ages & Stages Questionnaires [R], (ASQ-3 [TM]): A Parent-Completed Child-Monitoring System. ERIC; 2009
- Bethell C,
- Reuland CH,
- Halfon N,
- Schor EL
- ↵Bickmore TW, Paasche-Orlow MK. The role of information technology in health literacy research. J Health Comm. 2012;17(suppl 3):23–29
- ↵Adams WG, Fuhlbrigge AL, Miller CW, et al. TLC-Asthma: an integrated information system for patient-centered monitoring, case management, and point-of-care decision support. Paper presented at the AMIA Annual Symposium Proceedings; Washington, DC; 2003
- ↵Medhi I, Sagar A, Toyama K. Text-free user interfaces for illiterate and semi-literate users. Paper presented at the Information and Communication Technologies and Development, 2006 International Conference on Information and Communications Technologies and Development; Washington, DC; 2006
- Copyright © 2014 by the American Academy of Pediatrics