BACKGROUND: Making national comparisons of family experience of inpatient pediatric care has been limited by the lack of a publicly available survey. The Agency for Healthcare Research and Quality and Centers for Medicare & Medicaid Services commissioned development of the Child Hospital Consumer Assessment of Healthcare Providers and Systems Survey to address this gap. Using Child Hospital Consumer Assessment of Healthcare Providers and Systems Survey, we measured performance of hospitals in a national field test.
METHODS: We analyzed 17 727 surveys completed from December 2012 to February 2014 by parents of children (<18 years) hospitalized at 69 hospitals in 34 states. For each of 18 survey measures, we calculated a case-mix-adjusted hospital “top-box” score (ie, percentage of respondents selecting the most positive response option). We quantified variation across hospitals by estimating hospital-level SDs for each item with a hierarchical linear probability model. We examined associations of family experience with patient, parent, and hospital characteristics. We compared aggregate performance on each measure across participating hospitals.
RESULTS: Mean hospital top-box scores ranged from 55% (“Preventing mistakes and helping you report concerns”) to 84% (“Keeping you informed about your child’s care in the emergency department”). The mean for overall rating of hospital stay was 73% (SD 7%). “Quietness of hospital room” scores varied most across hospitals (SD 8%). Overall top-box scores were higher for freestanding children’s hospitals (74%) and children’s hospitals within a hospital (73%) than for pediatric wards within hospitals (68%, P = .007).
CONCLUSIONS: Family experience of pediatric inpatient care shows substantial room for improvement and varies considerably across hospitals and measures.
- HCAHPS —
- Hospital Consumer Assessment of Healthcare Providers and Systems
- KID —
- Kids’ Inpatient Database
What’s Known on This Subject:
Although patient experience has been recognized as a key aspect of health care quality and is positively associated with other clinical outcomes, little is known about how families experience inpatient pediatric care nationally or in individual hospitals.
What This Study Adds:
Child Hospital Consumer Assessment of Healthcare Providers and Systems measures show substantial variation across hospitals and room for improvement in family experience of inpatient pediatric care. Child Hospital Consumer Assessment of Healthcare Providers and Systems can help hospitals to identify targets for improvement efforts.
Patient-centered care is a critical component of high-quality health care and is associated with positive health care outcomes, such as treatment adherence, receipt of preventive care, improved clinical outcomes, and lower health care utilization.1–3 Although less work has been done in pediatrics, family-centered care has also been associated with positive clinical outcomes, including reduced nonurgent emergency department visits, improved receipt of anticipatory guidance, and reduced unmet needs.4,5 Whether patients and families experience patient-centered care is often measured through surveys. Stronger associations have been found between patient-centeredness and health outcomes when patient-centeredness is measured by patient report than when it is measured by provider or researcher assessment.6–10
Measures of adult patient experience have demonstrated variation in performance across hospitals, health plans, and providers.11,12 In addition, variation in hospital performance on patient experience has been shown to positively correlate with variation in hospital performance on such quality measures as 30-day readmission rates, patient safety indicators (eg, infection due to medical care and postoperative complications), and inpatient mortality for acute myocardial infarction.13–15 Adult Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) studies have shown that scores can be improved16,17 and that positive characteristics of hospitals and providers, such as greater cultural competency, collaborative cultures, and higher physician engagement, are associated with better scores.18–21 Initiatives such as nurse manager rounding, postdischarge telephone follow-up, improved discharge teaching skills, and physician education and feedback have been associated with improved Adult HCAHPS scores.22,23 Given the importance of patient-centered care and the known variation in performance across hospitals serving adults, the Centers for Medicare & Medicaid Services includes measures from the Adult HCAHPS Survey in its Value-Based Purchasing Program.24 In addition, many providers, hospitals, and health systems incorporate patient experience measures into performance goals and increasingly participate in financial risk arrangements that include such measures.
Little is known about the performance of hospitals that serve pediatric patients. A major impediment has been the lack of a nationally developed, publicly available survey of pediatric inpatient experience of care. To address this gap, the Pediatric Quality Measures Program, a joint program of the Agency for Healthcare Research and Quality and Centers for Medicare & Medicaid Services, commissioned the Center of Excellence for Pediatric Quality Measurement to develop the Child HCAHPS Survey. Using Child HCAHPS, we assessed the performance of hospitals participating in Child HCAHPS national field test, determined whether performance varied across hospitals, and examined whether performance differed in association with hospital characteristics.
Development of Child HCAHPS included an extensive review of the literature and existing patient experience quality measures, expert interviews, focus groups, cognitive testing, pilot testing of the draft survey, a national field test with 69 hospitals in 34 states, psychometric analysis, validity and reliability testing, and end-user testing of the final survey.25 The final Child HCAHPS Survey instrument has 62 items, including 39 patient experience items (which comprise 18 composite and single-item measures), 10 screening questions, 12 demographic/descriptive items, and 1 open-ended item. We found that hospital-level reliability for our composite and single-item measures would be good to excellent at the recommended sample size of 300, comparable to Adult HCAHPS26 and reaching recommended levels for comparison.27
Study Population and Survey Administration
In the national field test, the survey was administered to parents/guardians (henceforth “parents”) of patients aged <18 years who had ≥1 overnight stay at a participating hospital. The survey’s standard exclusion criteria were used: “No-Publicity” patients (ie, parents who do not want to be contacted), court/law enforcement patients, wards of the state, observation patients, healthy newborns, obstetric patients, patients with a foreign home address, patients excluded because of state regulations, patients admitted for a psychiatric diagnosis, patients discharged to another health care facility, and deceased patients. Data were collected over various intervals for each hospital during a 14-month period (December 2012–February 2014). Survey vendors already contracted by the participating hospitals administered questionnaires by mail or telephone in English or Spanish. No incentives were offered. The Boston Children’s Hospital Institutional Review Board approved the study.
Child HCAHPS Measures
The Child HCAHPS Survey contains 18 measures, each of which is composed of ≥1 survey items (see Supplemental Table 5 for the items within each measure and a comparison with Adult HCAHPS). The measures are categorized into 5 overarching groups: communication with parent, communication with child, attention to safety and comfort, hospital environment, and global ratings. The measures are also combined into a summary score to create an overall summative measure of performance. Child and Adult HCAHPS both have measures that address communication with nurses, communication with doctors, responsiveness of hospital staff, hospital environment, overall rating of hospital, and willingness to recommend the hospital. Even when composites address the same topic, their component items in some cases vary between the child and adult surveys (eg, Child HCAHPS responsiveness of hospital staff measure does not include such Adult HCAHPS items as help getting to the bathroom or using a bedpan). Child HCAHPS contains 3 domains not included in Adult HCAHPS: privacy, patient safety, and age-appropriateness of care.
Child HCAHPS scores use case-mix adjustment, allowing for meaningful comparison of inpatient pediatric patient experience of care across hospitals nationwide. During the development of the Child HCAHPS Survey, we tested the effects of variables available from the survey and hospitals’ administrative data and identified those that were predictive of responses and also had unequal distributions at different hospitals.25,28 In our final model, child covariates are age (<1, 1–4, 5–8, 9–12, ≥13 years) and parent-reported global health status (excellent, very good, good, fair, poor). Parent covariates include age (<25, 25–34, 35–44, ≥45 years); relationship to child (mother, father, other); education (≤8th grade, some high school, high school diploma or general educational development, some college or 2-year degree, 4-year college degree, >4-year college degree); and preferred language (English, Spanish, other).
We also examined hospital-level characteristics including hospital type (freestanding children’s hospital, children’s hospital within a hospital, pediatric ward within a hospital); teaching status (teaching, nonteaching); and number of eligible pediatric discharges per month (<300, 300–599, ≥600).
We compared the child characteristics (age, sex, race/ethnicity) from our national field test to those of 2009 Kids’ Inpatient Database (KID) subjects to examine the representativeness of our patient population.29 KID is the largest nationally representative all-payer US pediatric inpatient database.
For each survey item, we calculated a case-mix-adjusted hospital “top-box” score, defined as the percentage of respondents selecting the most positive response options, that is, “always,” “yes, definitely,” or a response of 9 or 10 on the overall hospital rating’s 10-point scale.30 Child HCAHPS uses top-box scoring rather than linear means because top-box scores, which are also used for Adult HCAHPS, result in higher hospital-level reliability25 and have been shown to be easier for patients and families to understand.30 Composite scores were defined as the mean of the scores for the component items at the hospital level.
We calculated the mean, SD, and range of hospital top-box scores across hospitals. We quantified variation across hospitals by estimating the hospital-level SD for each item, excluding patient-level sampling variation, using a hierarchical linear probability model.31 For the 53 hospitals with ≥100 completed surveys, we analyzed whether hospitals performed above, at, or below the overall mean on each measure: we calculated the overall mean across all hospitals (ie, the mean of unadjusted hospital rates), weighting hospitals equally, then calculated the adjusted score and SE for each hospital and tested whether the each individual hospital score differed significantly from the overall mean (P < .05 for statistical significance).
We examined associations of patient experience scores with patient and parent characteristics (case-mix adjustment variables) using a series of regression models predicting each of the 39 survey item scores that comprise the final 18 measures. We counted the number of survey items with which each patient or parent characteristic was significantly associated. Moreover, to evaluate the unique contribution of each case-mix adjustment variable to the adjustment of hospital means, for each survey measure, we performed a series of ordinary linear regressions, omitting 1 patient or parent characteristic at a time and recalculating hospital-level estimates. To evaluate the association of patient experience scores with hospital characteristics, we used linear regression models with adjusted hospital-level means as the outcome variable and hospital characteristics as the predictors. To avoid confounding between patient-level and hospital-level effects, we used separate models to evaluate the contribution of case-mix adjustment variables and the association of patient experience scores with hospital characteristics.
We analyzed 17 727 completed surveys. Child age and sex were similar between our survey sample and 2009 KIDS subjects, but the 2 groups differed in racial/ethnic composition (Table 1). Freestanding children’s hospitals and children’s hospitals within a hospital each comprised 41% of the field test hospitals, with the rest (19%) consisting of pediatric wards within a general hospital. The overall response rate for hospitals that administered ≥1000 surveys was 17.7% (interquartile range 8.0%), which is comparable to that attained for proprietary pediatric patient experience surveys. Responses averaged 257 per hospital (median of 207), with broad representation with respect to child and parent characteristics (Table 1).
The Child HCAHPS Survey items distinguished performance across hospitals with an average reliability of 0.81 on the basis of 300 responses, which is comparable to Adult HCAHPS26 and consistent with levels recommended for valid comparison.27 Mean hospital top-box scores ranged from 55% (“Preventing mistakes and helping you report concerns”) to 84% (“Keeping you informed about your child’s care in the emergency room”; Table 2). The mean top-box score for overall rating of hospital stay was 73%.
Meaningful variation in performance across hospitals was demonstrated for all measure scores. For instance, the SD for overall rating was 7%. The measure with the greatest variation across hospitals was quietness of hospital room (SD 8%). Hospital performance varied significantly; among hospitals with ≥100 completed surveys, the worst-performing hospital scored significantly below the mean for 15 of 18 measures and the best-performing hospital scored significantly above the mean for 16 of 18 measures (Fig 1). Please refer to Supplemental Table 6 for detailed findings on variation in hospital performance.
Top-box scores for the 39 reported items were generally higher for patients with higher parent-reported global health status, higher parent age, and lower parent education. The average summary top-box score was higher for freestanding children’s hospitals (74%) and children’s hospitals within a hospital (73%) than for pediatric wards within hospitals (68%, P = .007; Table 3). The average summary top-box score was also higher for teaching hospitals (74%) than nonteaching hospitals (70%, P = .04) and for hospitals with ≥600 (74%) or 300 to 599 (75%) versus <300 (70%) eligible discharges per month (P = .02).
We found similar associations for hospital type, teaching status, and volume for the composite “Preparing to leave the hospital.” We likewise observed similar patterns by hospital type and volume for “Communication about medicines,” “Hospital rating,” and “Recommend hospital.” For the single item “Involving teens in care,” average top-box scores were higher for freestanding children’s hospitals and children’s hospitals within a hospital (73% and 70%, respectively) than for pediatric wards (62%, P = .003). Altogether, freestanding children’s hospitals and children’s hospitals within a hospital significantly outperformed pediatric wards on seven Child HCAHPS measures, with only 1 significant difference in the opposite direction (“Responsiveness to the call button”). Several of these differences were large (>0.8 SD; Table 4).32 Teaching versus nonteaching hospitals had higher top-box scores for “Communication between you and your child’s doctors” (82% vs 78%, respectively, P = .03); “How well nurses communicate with your child” (71% vs 65%, respectively, P = .01); “How well doctors communicate with your child” (67% vs 61%, respectively, P = .048); and “Child comfort” (68% vs 63%, respectively, P = .006) but lower top-box scores for “Quietness of hospital room” (61% vs 69%, respectively, P = .01).
Our study demonstrates that despite growing recognition of the importance of patient experience, the 69 hospitals in our national field test have room for improvement on the experience of care they provide for pediatric inpatients and their families. Mean hospital top-box scores ranged widely, with a mean of 73% for overall rating and hospital performance varied significantly, with some hospitals performing above average on most measures and others performing below average on most measures. However, even the former demonstrated room to improve. For the 6 composite measures composed of the same items in Child versus Adult HCAHPS (ie, communication with nurses, communication with doctors, responsiveness of hospital staff, hospital environment, overall rating of hospital, and willingness to recommend), mean hospital top-box scores for Child HCAHPS were similar to national means for the corresponding the Adult HCAHPS measures.33
We found that aspects of patient experience differed by hospital characteristics. Average hospital top-box scores for global rating measures were notably higher for freestanding children’s hospitals and children’s hospitals within a hospital than for pediatric wards. In addition, freestanding hospitals outperformed pediatric wards on several measures of interpersonal communication, such as “Communication between you and your child’s doctors,” “Preparing you and your child to leave the hospital,” and “Involving teens in their care.” These findings could reflect that freestanding hospitals focus on pediatric care and use pediatric-specific services such as child life specialists. Our results are contrary to those of an older study, but methodological differences limit comparability.34 Compared with nonteaching hospitals, teaching hospitals had higher performance on child comfort; preparing to leave the hospital; and doctor-parent, nurse-child, and doctor-child communication measures, with lower performance only on quietness. In addition, hospitals with the lowest number of eligible discharges (<300) had poorer performance than those with greater numbers of eligible discharges.
Variability in hospital performance has likewise been observed for Adult HCAHPS scores, both among individual hospitals and by hospital type, size, and location. For Adult HCAHPS, performance tends to be better for non–safety-net versus safety-net hospitals, specialty-care versus general hospitals, and nonprofit or public versus for-profit hospitals.35–37 In contrast to our findings, teaching status in adult studies has not been associated with differences in global rating of the hospital, but nonteaching hospitals perform better on measures of experience with doctor and nurse communication, nursing services, pain control, and the cleanliness of the hospital room.37–39 The notable differences between Child and Adult HCAHPS findings may all be related to the significance of the distinction between pediatric wards in general hospitals and more freestanding models in pediatrics, a distinction without a direct parallel in adult inpatient care. Small hospitals (<100 beds), nonurban hospitals, those with Magnet status for nursing, and those with higher nurse-to-patient-days ratios also tend to perform better on Adult HCAHPS measures.32,34–36 The contrasting findings regarding volume for Child versus Adult HCAHPS might be explained by the lower numbers of eligible discharges from pediatric wards within large general hospitals than from freestanding children’s hospitals or children’s hospitals within a hospital.
Our findings show potential targets for widespread change. The lowest hospital average top-box score was for the composite measure “Preventing mistakes and helping you report concerns” (55% “Always”), which has noteworthy implications for patient safety. The items within the composite—whether hospital staff checked wristbands before administering medicines and whether they told parents how to report any concerns about their child’s care—are components of patient safety that parents can uniformly observe. In addition, reports on nurse and physician communication with the child were less favorable than those on communication with the parent, regardless of hospital type. Our cognitive interviews demonstrated that parents were able make this distinction consistently.25 Another potential target is responsiveness to the call button, which Adult HCAHPS studies have shown is important to adult patient experience.40 In our study, the mean top-box score for responsiveness was only 60%. Hospitals have shown that interventions such as hourly rounding improve both Adult HCAHPS staff responsiveness scores and clinical outcomes such as falls.41–43 Additional research is needed to demonstrate whether interventions in hospitals serving pediatric patients are likewise associated with improvement in patient experience. Furthermore, some interventions might be unique to the pediatric setting, such as those addressing the particular challenges of communicating with children. In addition to evaluating whether interventions are associated with improvements in patient experience, future work might examine associations between patient experience and clinical outcomes (eg, medication adherence, pediatric readmission rates).
Although this is the largest study of pediatric inpatient experience to date, it has limitations. Hospitals volunteered to participate and might not be representative of all hospitals serving children (eg, volunteers might be more likely to have higher scores than nonvolunteers). It is possible that some differences among hospitals are due to incomplete adjustment for case mix. In addition, response rates were low. As a result, responses might not have been fully representative of patient populations at participating hospitals. However, response rates were comparable to those attained for proprietary pediatric patient experience surveys and for younger adults in other CAHPS surveys.44 Moreover, high standards of hospital-level reliability were achieved at the recommended number of completed surveys. Young adults, including parents of children receiving care, are a particularly challenging group with which to achieve high response rates.44 Ongoing research is examining alternative strategies to raise response rates, such as alternative survey modes that might be found to be more effective with young adults.
As a publicly available survey developed specifically for pediatric use, Child HCAHPS enables hospitals serving pediatric patients to assess their performance on patient experience and benchmark it against the growing number of hospitals fielding the survey. It adds domains important to pediatric care that are not covered in Adult HCAHPS, such as privacy, age-appropriate care, and safety. Child HCAHPS national field test has demonstrated that many hospitals are not high performers on pediatric patient experience. Hospital-level variation exists for pediatric patient experience, and the survey can be used as a tool to identify targets for improvement. Public reporting of Adult HCAHPS scores has been associated with significant improvement in scores, particularly among initially low-performing hospitals, helping to potentially reduce disparities in patient experience between hospitals.16,17 Proposed mechanisms by which public reporting can promote improved performance include pay-for-performance, reputational incentives, and helping hospitals identify areas needing improvement.25 As implementation of Child HCAHPS spreads, we hope to see similar improvements in pediatric patient experience.
We thank Sarah Onorato, BA, for editorial work and assistance in manuscript preparation.
- Accepted January 23, 2017.
- Address correspondence to Sara L. Toomey, MD, MPhil, MPH, MSc, Division of General Pediatrics, Boston Children’s Hospital, 300 Longwood Ave, Boston, MA 02115. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Support for this work was provided by the US Department of Health and Human Services Agency for Healthcare Research and Quality and Centers for Medicare & Medicaid Services, Child Health Insurance Program Reauthorization Act (CHIPRA) Pediatric Quality Measures Program Centers of Excellence under grant U18 HS 020513 (principal investigator: Schuster). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. This work was also conducted with the support of a Mentored Career Development/Catalyst Medical Research Investigator Training award (KL2 award) from Harvard Catalyst/the Harvard Clinical and Translational Science Center (National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health Award KL2 TR001100).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Doyle C,
- Lennox L,
- Bell D
- Anhang Price R,
- Elliott MN,
- Zaslavsky AM, et al
- Brousseau DC,
- Hoffmann RG,
- Nattinger AB,
- Flores G,
- Zhang Y,
- Gorelick M
- Patient-Centered Care: What It Means and How to Get There—Health Affairs Blog. Available at: http://healthaffairs.org/blog/2012/01/24/patient-centered-care-what-it-means-and-how-to-get-there. Accessed January 13, 2014
- Hudon C,
- Fortin M,
- Haggerty JL,
- Lambert M,
- Poitras M-E
- Little P,
- Everitt H,
- Williamson I, et al
- Glickman SW,
- Boulding W,
- Manary M, et al
- Elliott MN,
- Lehrman WG,
- Goldstein EH, et al
- Manary M,
- Staelin R,
- Kosel K,
- Schulman KA,
- Glickman SW
- Hospital Value-Based Purchasing Program
- Toomey SL,
- Zaslavsky AM,
- Elliott MN, et al
- Elliott MN,
- Lehrman WG,
- Goldstein E,
- Hambarsoomian K,
- Beckett MK,
- Giordano LA
- Lyratzopoulos G,
- Elliott M,
- Barbiere JM, et al
- Zaslavsky AM
- Agency for Healthcare Research and Quality
- Giordano LA,
- Elliott MN,
- Goldstein E,
- Lehrman WG,
- Spencer PA
- Hospital Consumer Assessment of Healthcare Providers and Systems
- Cohen J
- Centers for Medicare & Medicaid Services
- Co JPT,
- Ferris TG,
- Marino BL,
- Homer CJ,
- Perrin JM
- Siddiqui ZK,
- Wu AW,
- Kurbanova N,
- Qayyum R
- Lehrman WG,
- Elliott MN,
- Goldstein E,
- Beckett MK,
- Klein DJ,
- Giordano LA
- Copyright © 2017 by the American Academy of Pediatrics