January 2017, VOLUME139 /ISSUE 1

A Population Intervention to Improve Outcomes in Children With Medical Complexity

  1. Garey Noritz, MD, FAAP, FACPa,b,c,
  2. Melissa Madden, MPHa,
  3. Dina Roldan, BAa,
  4. T. Arthur Wheeler, MS, MSES, MBAa,
  5. Kimberly Conkol, RNa,c,
  6. Richard J. Brilli, MD, FAAP, MCCMa,b,
  7. John Barnard, MDa,b, and
  8. Sean Gleeson, MD, MBAa,b,c
  1. aNationwide Children’s Hospital, Columbus, Ohio;
  2. bDepartment of Pediatrics, The Ohio State University, Columbus, Ohio; and
  3. cPartners For Kids, Columbus, Ohio
  1. Dr Noritz and Ms Madden conceptualized and designed the study, drafted the initial manuscript, and critically reviewed the manuscript; Ms Roldan conducted the initial analyses and reviewed and revised the manuscript; Mr Wheeler conducted the initial analyses, contributed to the interpretation of data, and reviewed and revised the manuscript; Ms Conkol conceptualized and designed the study, contributed to the interpretation of data, and critically reviewed the manuscript; Drs Brilli and Barnard contributed to the interpretation of data and critically reviewed the manuscript; and Dr Gleeson conceptualized and designed the study, drafted the initial manuscript, and critically reviewed the manuscript. All authors approved the final manuscript as submitted.


BACKGROUND AND OBJECTIVES: Children with medical complexity experience frequent interactions with the medical system and often receive care that is costly, duplicative, and inefficient. The growth of value-based contracting creates incentives for systems to improve their care. This project was designed to improve the health, health care value, and utilization for a population-based cohort of children with neurologic impairment and feeding tubes.

METHODS: A freestanding children’s hospital and affiliated accountable care organization jointly developed a quality improvement initiative. Children with a percutaneous feeding tube, a neurologic diagnosis, and Medicaid, were targeted for intervention within a catchment area of >300 000 children receiving Medicaid. Initiatives included standardizing feeding tube management, improving family education, and implementing a care coordination program.

RESULTS: Between January 2011 and December 2014, there was an 18.0% decrease (P < .001) in admissions and a 31.9% decrease (P < .001) in the average length of stay for children in the cohort. Total inpatient charges were reduced by $11 764 856. There was an 8.2% increase (P < .001) in the percentage of children with weights between the fifth and 95th percentiles. The care coordination program enrolled 58.3% of the cohort.

CONCLUSIONS: This population-based initiative to improve the care of children with medical complexity showed promising results, including a reduction in charges while improving weight status and implementing a care coordination program. A concerted institutional initiative, in the context of an accountable care organization, can be part of the solution for improving outcomes and health care value for children with medical complexity.

  • Abbreviations:
    accountable care organization
    children with medical complexity
    electronic health record
    length of stay
    Nationwide Children’s Hospital
    children with neurologic impairment and a feeding tube
    Pediatrics Health Information System
    Partners for Kids
    quality improvement
  • Improving outcomes, including the value of care for children with medical complexity (CMC), represents a unique and important challenge. Physicians of different specialties care for these patients because of the broad range of underlying medical conditions, varying symptoms, and nuances of individual hospital practices. The extensive needs of CMC challenge traditional communication and care coordination models.13 Harmonizing care among multiple disciplines requires focused attention and the development of innovative approaches.4

    Quality improvement (QI) techniques have been successfully applied to drive change in pediatric chronic diseases.510 In these analyses, the target patient population included a well-defined medical disorder and subspecialty specific oversight. These characteristics do not apply to CMC, a group which frequently uses multiple hospital services, experiences disparate medical conditions, and may not have a “home” physician subspecialty. QI methods, especially efforts to reduce care variability and increase health care value, may also improve the care of CMC but need to be implemented among multiple subspecialists.

    Due to their underlying fragility, CMC are more likely to require inpatient hospital care or die prematurely.1113 Without thoughtful coordination, care of CMC may focus more on urgent needs than a holistic, preventive approach.14 Although this group comprises 6% of children receiving Medicaid, they consumes 40% of Medicaid dollars spent on children.1517 Berry et al18 studied children with neurologic impairment and found they accounted for 5.3% of pediatric hospitalizations and 21.6% of charges. The number of CMC is increasing,19 generating greater need for effective management of this population.

    The present article describes the work of 1 children’s hospital and affiliated accountable care organization (ACO) to improve health care value for a subset of CMC (ie, those with neurologic impairment and a percutaneous feeding tube [NI/FT cohort]).



    Nationwide Children’s Hospital (NCH) is a freestanding quaternary children’s hospital. In 1997, NCH formed Partners For Kids (PFK), a physician hospital organization accountable for children covered by Medicaid managed care in the hospital’s catchment area. Through capitated risk arrangements, PFK serves >300 000 children, including those with medical complexity.20

    This project was identified by the NCH Institutional Review Board as a QI study and exempt from review.

    Identification of the Cohort

    CMC targeted for this project were those with NI/FT. During the observation period (January 2011–December 2014), a rolling 12-month methodology identified the monthly cohort by querying NCH’s Enterprise Data Warehouse. To be included in the cohort, a child had to meet all of the following: (1) age 0 to18 years; (2) covered by Ohio Medicaid; (3) have a visit to NCH since January 2010 in which the billing data included 1 of the neurologic codes found in Supplemental Table 2; and (4) had a visit to NCH in the prior 12 months in which the billing data included 1 of the feeding tube codes found in Supplemental Table 3. Thus, children with a neurologic code entered the cohort upon their first visit in which a tube feeding code was captured (either new or existing tubes). Children exited the cohort 12 months after their last visit in which a tube feeding code was captured. The list of codes was created by clinicians with experience caring for CMC and included neurologic and genetic diagnoses likely to cause significant functional impairment.


    A multidisciplinary feeding tube task force began in July 2012, creating aim statements and key driver diagrams to address each of the triple aim goals of the project (Supplemental Figs 6–8). Aim 1 was to reduce charges by decreasing admissions and the average length of stay (LOS) for those admissions. Aim 2 was to improve health by improving nutrition, increasing the proportion of children with a weight between the fifth and 95th percentiles for age on a standard growth chart.21 Aim 3 was to improve care quality and efficiency by providing proactive care coordination.

    Interventions fell into 3 main categories: standardizing percutaneous feeding tube management, improving family education, and implementing a care coordination program. A rapid cycle improvement strategy was used to allow for continuous intervention refinement and to integrate with workflow changes in different departments.22

    Standardization interventions focused on tool development to manage data related to a child’s feeding tube, documented within the electronic health record (EHR). Providers making a referral for feeding tube insertion were asked to proactively delineate responsibilities among providers managing the child’s feeding tube. Identification of these care team members sought to improve continuity and clarify who the family should contact if problems arose. A form was created to discretely house care team information, the current feeding regimen, and progress toward tube removal, which was easily accessible in each child’s EHR.

    Education interventions sought to empower families and address inconsistencies in the education process. A bundle of materials was developed, including a Care Journey Board outlining educational milestones and a printed educational workbook. One-on-one education with a nurse educator included a tablet-based preteaching/postteaching assessment. Educational materials were made available over the Internet and through a mobile app.

    Based on feedback from staff and families, rapid cycle changes were made to the care coordination model during the project. These changes included altering outreach/engagement tactics, the staffing model, content of the health risk assessment, format of the care plan, the contact schedule, and the duration of the intervention. The final model prioritized outreach to those members of the cohort with higher rates of emergency department and inpatient utilization. Those agreeing to participate were assigned to a registered nurse or social worker in a 1:40–60 ratio. A health risk assessment, completed at enrollment, included a patient/family interview, review of the medical record, input from providers, and synthetization of the patient’s treatment plans. Individualized goals with interventions were established to address each identified need. The assessment was updated quarterly and with significant changes in patient status. Enrolled families were contacted at least monthly. Face-to-face visits took place at least every 90 days in various locations (home, physician office, or other). Children remained engaged in care coordination until established goals were met and no new needs were identified.

    Data Collection and Measures

    Monthly charge data were collected from NCH’s Enterprise Data Warehouse, with utilization measured by tracking admissions per 100 patients, average LOS for admissions, and average charge per admission. Comparison was made between the cohort’s average total inpatient charges per month during the baseline year (preceding project initiation) and each subsequent year of the observation period. The first 6 months of year 3 were compared with 50% of the baseline year charges. The hospital admission during which the feeding tube was placed was excluded.

    Impact on the health of the NI/FT cohort was measured by tracking the percentage of patients weighed during the reporting month with a weight between the fifth and 95th percentiles on a standard growth curve. Weight was measured by using the standardized procedure for all patients at NCH, either directly, or by weighing the child in the wheelchair and subtracting the wheelchair weight. Unexpectedly high or low weights trigger warnings requiring verification. Later, a dietitian was hired to review individual patient charts and determine if the child was at an appropriate weight considering age, condition, and disease state, even if outside the fifth to 95th percentile range. All patients outside that range were reviewed, as well as those patients who were deemed “at risk” by a clinical dietitian in the course of their usual care. Factors considered by the dietitian included underlying diagnosis, degree of disability, comparison with height when available, and examination of growth over time.

    Engagement in the care coordination program was tracked by using tools built into the EHR. Children in the cohort in December 2014 were categorized as enrolled, declined, unable to contact, or ineligible (deceased, living in a nursing facility, or moved out of state).

    Statistical Analysis

    Data were analyzed by using statistical process control methods. A u-chart, which assumes a Poisson data distribution, was used to track admission rates. The random and relatively rare nature of hospital admissions makes it likely that they would follow a Poisson distribution, and a Poisson fit test (Minitab 17; Minitab Inc, State College, PA) provided confirmation with a P value of .912 (indicating no evidence of deviation from Poisson).

    X-bar charts were used to track average LOS and average charge per inpatient stay. Because LOS and charges data deviated substantially from a normal distribution, a Box-Cox logarithmic data transformation was used to determine appropriate control limits for the X-bar and S control charts. (Control limit calculations for these charts assume that data are at least approximately normal.) The resulting means, variances, and control limits for the transformed data were then reverse-transformed and scaled to yield statistically equivalent control charts in terms of the original LOS and charges, rather than transformed measures.23 Only the control limits were affected by this transformation. All data remain unchanged, and statistical tests were performed on original (not transformed) data. A p-chart was used to track the percentage of patients with weights between the fifth and 95th percentiles.

    The Pediatric Health Information System (PHIS) database case-mix indices, particularly the associated expected charges and expected LOS, were used to account for potential differences in hospital charges and LOS that might result from variation in patient severity. Adjustment consisted of revising the preintervention results to coincide with what they would have been, had the case-mix been the same as the postintervention. The initial (January 2011–May 2012) average values of charges and LOS were adjusted downward (because postintervention case-mix was less severe) by the same percentage as the reductions in the corresponding average PHIS expected charges and LOS.24 The improved (September 2013–December 2014) averages were compared with the downward-adjusted initial averages. This method ensured that any significant results would indicate that we achieved savings beyond reductions due to case-mix differences. Two-sample t tests were used for the statistical comparisons. The t tests were justified by application of the central limit theorem, given that the LOS and charges were group averages of >1000 values.

    Hospital admission comparisons were performed using a 2-sample Poisson rate test (in line with the aforementioned u-chart justification). Because this population is medically fragile and every patient might realistically be admitted in any month, zero-inflated Poisson or negative binomial regression was not performed.

    Acceptable weight comparisons were conducted by using a 2 proportions (Fisher’s exact) test. All tests were performed in Minitab 17.


    Demographic data for children in the NI/FT cohort during the observation period (January 2011–December 2014) are shown in Table 1 (N = 1070). The average monthly cohort size was 548 (range, 437–630). Hospital admissions per 100 patients decreased from 15.0 to 12.3, a reduction of 18.0% (P < .001) (Fig 1).

    TABLE 1

    Demographic Characteristics of the NI/FT Cohort

    FIGURE 1

    Inpatient admissions per 100 patients in the NI/FT cohort. RN, registered nurse.

    Average LOS is shown in Fig 2A, with SDs in Fig 2B. Average LOS decreased from 7.2 days to 4.9 days, a 31.9% reduction (P < .001). Outlier points indicating possible special cause variation (eg, April 2013) were excluded in calculating control limits but retained in statistical significance calculations to be conservative in measuring program impact.

    FIGURE 2

    (A) Average LOS for patients in the NI/FT cohort. (B) SD of LOS for patients in the NI/FT cohort. An In(x) transformation was used to determine control limits. Indication of potential special cause may appear on any of the 4 control charts. aCenter lines reflect specified baselines. Some outlier points (on this or other charts) were excluded; thus, plotted center lines will not reflect these points. LOS, Length of Stay; NI/FT, children with neurologic impairment and a feeding tube; RN, Registered Nurse; SD, Standard Deviation.

    Average charges per inpatient stay decreased from $65 152 to $54 442, a reduction of 16.4% (P = .03) (Fig 3). Centerline shifts were observed in May 2012 and August 2013 for admissions, average LOS, and charges.

    FIGURE 3

    Average charge per inpatient stay for patients in the NI/FT cohort. An In(x) transformation was used to determine control limits. Indication of potential special cause may appear on any of the 4 control charts. aCenter lines reflect specified baselines. Some outlier points (on this or other charts) were excluded; thus, the plotted center lines will not reflect these points.

    PHIS benchmarking showed that the decrease in hospital charges and LOS were only fractionally due to case-mix changes (7.6% and 5.6%, respectively). Although the difference in the case-mix–adjusted hospital charges did not reach statistical significance (P = .23), the difference in the case-mix–adjusted LOS remained significant (P < .001).

    Average total inpatient charges per month for the cohort decreased each year of the observation period (Fig 4). The cohort’s average total inpatient charges per month decreased from $4 512 016 during the baseline year to $3 834 416 during the first 6 months of year 3. This change was a decrease from the baseline of 2.1% in year 1, 12.2% in year 2, and 15.0% in the first 6 months of year 3, resulting in an estimated total savings of $11 764 856 over the 30 months. Given our hospital’s 0.44 ratio of cost to charges, the estimated cost savings were $5 176 537. This amount compares favorably to the program costs of $663 658, resulting in a positive return on investment of 680%.

    FIGURE 4

    Average total inpatient charges for the NI/FT cohort per month. NI/FT, children with neurologic impairment and a feeding tube; SD, Standard Deviation.

    To measure the health of the cohort, weight percentiles were tracked on standard growth curves (Fig 5). The percentage of children with a weight between the fifth and 95th percentiles increased from 56.1% to 60.7%, an 8.2% increase (P < .001). Over the last 15 months of the project, a dietitian determined that 75.8% of children were at an appropriate weight, even if they were outside the fifth to 95th percentile range.

    FIGURE 5

    Percentage of patients in the NI/FT cohort with acceptable weights. NI/FT, children with neurologic impairment and a feeding tube; RD, registered dietician; RN, registered nurse.

    Of the 544 children in the December 2014 cohort, 58.3% enrolled in care coordination (aim 3), 16.5% declined care coordination services, 15.1% were unable to be contacted, and 2.6% were deemed ineligible. As of December 2014, 7.5% of the cohort had not been offered care coordination services.


    The definition of CMC used here (a child with an NI/FT) creates a clinically meaningful, well-defined subset of CMC that can be easily identified from basic administrative data and targeted for QI. Hospital admissions, LOS, and average charge per inpatient stay decreased for children in this cohort during the observation period, even after adjustment for changes in severity. The marked reduction in hospital admissions, coupled with reduced (although not significant) charges per inpatient stay, supports a conclusion that total charges also decreased because inpatient care is the most costly component of care for CMC.16 Because inpatient hospitalization is stressful for families,25,26 and there are significant safety risks when CMC are hospitalized,27 future research may find that the intervention helped improve the quality of life for children in the cohort and their families.

    The use of PHIS to adjust for the less severe case-mix is a conservative approach, in that this method makes no allowance for the possibility that the reduced case-mix was the result of the program. Interventions such as care coordination and proactive feeding tube management were designed to reduce the illness severity of patients needing hospitalization. The adjusted results likely underestimate the significance of the improvement in charges and LOS.

    The reduction in hospital charges likely underrepresented the true impact of the changes. Inflation of medical care costs (5.7% increase in Consumer Price Index–Medical Care28 over the reported time period) would counter any project-associated reductions observed for charges. This impact could be formally assessed in a subsequent study. Although the adjusted hospital charges did not reach statistical significance, these findings are relevant to organizations caring for this at-risk population.

    Coller et al29 reviewed 17 studies that described interventions to reduce hospitalizations among CMC. Several described the value of medical home interventions to improve the care of CMC,15,3032 whereas others describe population-based interventions.33,34 As in our study, these programs focused on reductions in inpatient utilization, but we also tracked a health outcome (ie, weight status). We used new defining criteria for CMC (NI/FT), which can be used to automate patient identification.

    In children with disabilities, nutrition is intimately related to health, but it can be difficult to accurately determine nutritional status. Although Brooks et al35 constructed growth charts for children with cerebral palsy stratified according to severity of motor disability, the EHR did not allow practical retrieval of this information. For this reason, standard growth curves were used. There is potential bias inherent in the dietitian’s subjective review; however, this information was still valuable for examining the nutritional status of these complex children who are not easily assessed according to purely objective means. Improvement of weight status is only 1 of the possible benefits of tube feeding in the NI/FT population; others, such as ease of feeding, reduction in aspiration, and improved developmental gains, are important goals for families but less easily extracted from the EHR.36,37 An additional limitation includes difficulty in accurately measuring weight in children with NI/FT. However, the weighing process was standardized and consistent during the study.

    Our interventions involved multiple components over time. The tools were available organization-wide, but adoption and integration into standard workflows differed. These factors make it difficult to attribute outcomes to any specific intervention. Care coordination is the only intervention in which enrollment and engagement data were tracked at the patient level. The evolution of the care coordination model over the course of the project contributed to the complexity of a subgroup analysis for those who received the intervention, declined the intervention, or were unable to be reached. An article evaluating the impact of a standardized care coordination intervention on utilization outcomes for a cohort of high-risk children covered by PFK is forthcoming.

    Lack of claims data limited our ability to quantify costs of care outside of NCH, but given high market penetration, it is unlikely that much care for the children with NI/FT was provided elsewhere or that the proportion would have changed over the course of the study.

    The extent to which our interventions can be adopted in other settings is unclear. This research was performed in a children’s hospital with a mature ACO, a wide catchment area, and high market penetration. Hospitals interested in similar initiatives will need to assess their own institutional structure, identify areas for improvement, and work with stakeholders to design interventions within the aforementioned themes. The set of interventions we describe were effective in achieving improvement at NCH, and we believe they could be implemented in other organizations.


    This study describes a multifaceted QI initiative to improve the health, health care value, and utilization for a population-based cohort of NI/FT. Significant decreases in inpatient utilization and improvement in the growth status of the children were observed. In the setting of an ACO, in which NCH carried financial risk for children in our catchment area, we had robust institutional support to implement programs with the aim of improving care while reducing costs and thus improving value. The financial incentives for providers aligned with delivering the highest value care. This outcome is a much-needed shift in motivation for systems of health care in the United States, and we suggest that our outcomes demonstrate that ACOs can be part of the solution for improving the care of these most vulnerable children.


    We thank the entire Health Care Innovation team for their contributions to this project and care of the patients. We thank the following persons for their contributions to the manuscript: Deena Chisolm, PhD, Elaine Damo, MBA, RHIA, Kelly Kelleher, MD, MPH, Naomi Makni, MHA, Michael Slaper, MHSA, and Ellen Marie Whalen, PhD, CRNP.


      • Accepted July 5, 2016.
    • Address correspondence to Garey Noritz, MD, FAAP, FACP, Nationwide Children’s Hospital, Complex Health Care Program, 700 Children’s Dr, Columbus, OH 43205. E-mail: garey.noritz{at}
    • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

    • FUNDING: Supported by Funding Opportunity Number CMS-1c1-12-0001 from the Centers for Medicare & Medicaid Services, Center for Medicare and Medicaid Innovation. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the US Department of Health and Human Services or any of its agencies.

    • POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.