OBJECTIVES: To assess whether children with medical complexity (CMC) exposed to a hospital-based comprehensive case management service (CCMS) experience improved health care quality, improved functional status, reduced hospital-based utilization, and/or reduced overall health care costs.
METHODS: Eligible CMC at Seattle Children’s Hospital were enrolled in a cluster randomized controlled trial between December 1, 2010, and September 29, 2014. Participating primary care providers (PCPs) were randomly assigned, and CMC either had access to an outpatient hospital-based CCMS or usual care directed by their PCP. The CCMS included visits to a multidisciplinary clinic ≥ every 6 months for 1.5 years, an individualized shared care plan, and access to CCMS providers. Differences between control and intervention groups in change from baseline to 12 months and baseline to 18 months (difference of differences) were tested.
RESULTS: Two hundred PCPs caring for 331 CMC were randomly assigned. Intervention group (n = 181) parents reported more improvement in the Consumer Assessment of Healthcare Providers and Systems version 4.0 Child Health Plan Survey global health care quality ratings than control group parents (6.7 [95% confidence interval (CI): 3.5–9.8] vs 1.3 [95% CI: 1.9–4.6] at 12 months). We did not detect significant differences in child functional status and most hospital-based utilization between groups. The difference in change of overall health care costs was higher in the intervention group (+$8233 [95% CI: $1701–$16 937]) at 18 months). CCMS clinic costs averaged $3847 per child-year.
CONCLUSIONS: Access to a CCMS generally improved health care quality, but was not associated with changes in child functional status or hospital-based utilization, and increased overall health care costs among CMC.
- CAHPS —
- Consumer Assessment of Healthcare Providers and Systems
- CCMS —
- comprehensive case management service
- CI —
- confidence interval
- CMC —
- children with medical complexity
- ED —
- emergency department
- LOS —
- length of stay
- NP —
- nurse practitioner
- PCP —
- primary care provider
- SCH —
- Seattle Children’s Hospital
What’s Known on This Subject:
At this study’s outset, preliminary studies had revealed that multidisciplinary hospital-based comprehensive care programs may improve health care outcomes and reduce inpatient utilization for children with medical complexity, but few trials had been conducted.
What This Study Adds:
In a randomized controlled trial, access to a hospital-based comprehensive case management service generally improved health care quality, but was not associated with change in child functional status or hospital-based utilization, and increased overall health care costs among children with medical complexity.
Children with medical complexity (CMC) require more health care services compared with children without chronic illness.1–3 When sick, CMC can be challenging for primary care providers (PCPs) to care for because of their complex conditions and limited resources in the outpatient setting.4 In response, some children’s hospitals have created multidisciplinary hospital-based comprehensive care programs to address the needs of CMC.3,5,6 The small but growing body of research using mainly uncontrolled study designs suggests that programs such as the medical home,7–12 hospital-based programs,13–17 home care,18–20telehealth,21 and disease-specific specialty clinics22–24may improve health care outcomes for CMC.5 At this study’s outset, we were unable to identify previous trials of pediatric complex care service provision, but preliminary studies had revealed the potential for care coordination interventions to substantially reduce the costs of care for CMC by decreasing inpatient utilization.13,14,17,25,26
Seattle Children’s Hospital (SCH) developed a comprehensive case management service (CCMS), a multidisciplinary, hospital-based service focused on improving care coordination for CMC. SCH does not provide primary care but has subspecialty outpatient services and a dedicated inpatient service for CMC. With the creation of our CCMS, we had a unique opportunity to examine its effect. We conducted a cluster randomized controlled trial to assess whether exposure to our CCMS would improve health care quality and functional status, reduce hospital-based utilization, and/or reduce health care costs among CMC.
Trial Design and Setting
A cluster randomized controlled trial of the CCMS clinic at SCH was conducted from December 1, 2010, to September 29, 2014. The SCH Institutional Review Board provided approval.
Eligible children were obtained from 2 sources: (1) application of a validated algorithm to classify CMC27 by using 3M Clinical Risk Group categories 5b, 6, 7, or 9 from the previous 3 years of administrative data and (2) identification by participating PCPs. Potentially eligible children were screened for meeting 2 inclusion criteria: (1) hospitalization or emergency department (ED) visit at SCH 1 or more times in 2009 or 2010, or in 2011 until the end of the enrollment period, and (2) having a dominant chronic condition, defined as a “potentially lifelong, serious chronic medical condition that often results in progressive deterioration of health and that contributes to individual debility, death, or future need for medical services.”27 We then excluded children who were (1) cared for in a SCH subspecialty clinic that comprehensively addresses the needs of CMC; (2) age ≤90 days or (3) age ≥18 years; (4) from out-of-state; (5) expired; (6) other than English or Spanish speaking; or (7) unwilling to participate in research. Finally, we excluded children with a length of stay (LOS) ≥27 days who were felt to be in the palliative care phase of illness and unlikely to benefit from CCMS. Final confirmation of inclusion and exclusion criteria was performed by review of the child’s medical record by a research team physician, CCMS physician, or nurse practitioner (NP).
Randomization, Recruitment, and Blinding
Once the pool of 1036 eligible children was established, we identified their community-based PCP by using SCH databases (Fig 1). We conducted a cluster randomized controlled trial with PCPs as the units of randomization to address logistic and contamination concerns. To further balance cluster sizes, we stratified 587 PCPs into those with ≥5 and <5 eligible CMC, and we performed random assignments within each stratum by using the “sample” command in Stata.
Eligible PCPs were approached by mailed letter invitation followed 3 days later by recruitment by telephone call(s). Up to 10 recruitment calls were made before assuming a passive declination. If the PCP refused to participate, his or her patients were removed from the pool of eligible children. If the PCP agreed to participate, we reviewed his or her list of identified CMC with the PCP to confirm that he or she was the PCP for each listed child and that the PCP agreed with the CMC designation. We also solicited the names of additional patients, whom the PCPs believed were eligible, from the PCPs’ practices.
Once the PCP for an eligible child was enrolled, the child’s parents were approached by mailed letter invitation cosigned by the PCP, followed 2 weeks later by recruitment by telephone call(s). If the parent expressed interest in participating, he or she was sent a consent and Health Insurance Portability and Accountability Act authorization form. If the parent chose not to participate, his or her child was removed from the pool of eligible children. Once the consent forms were returned, parents were informed of their study group assignment.
PCPs and parents were blinded to their assignment until their patients/children were enrolled in the study. Study staff remained unblinded to group assignment throughout the study.
The intervention group had an initial 3-hour intake visit in the multidisciplinary SCH-based CCMS clinic staffed by several hospital medicine physicians who specialized in the care of CMC: 1 NP, 2 registered nurse case managers, medical assistants, 1 social worker, 1 registered dietician, and 1 dedicated scheduler. Potential experience gaps for hospital medicine providers who may have been less familiar with obtaining and maintaining outpatient care needs were addressed by collaboration with the numerous ancillary providers involved with the clinic. At this first visit, the child’s parent(s) worked with the CCMS team to develop an individualized shared care plan that included the child’s routine health care needs and steps to take when the child was sick, including at home, in the PCP’s office, and in the ED.28–30 A CCMS physician or NP reviewed the shared care plan with the child’s PCP by telephone to incorporate his or her feedback (see the CCMS Care Plan in the Supplemental Information). The shared care plan resided in the electronic medical record and was readily available to all SCH providers and staff. Paper copies of the plan were provided to the parent, the child’s PCP, and the state’s electronic database for EDs. When an intervention group child became ill, the parent, PCP, and CCMS providers followed the processes established by the CCMS and outlined in the child’s shared care plan. Intervention group PCPs and families also had access to CCMS providers to help manage care coordination needs and acute health issues. If a child presented to the SCH ED, a CCMS provider notified the ED of the patient and the reason for the visit; and if a child was admitted to SCH, a CCMS provider saw the patient and contacted the inpatient team. As a maintenance measure, CCMS visits were scheduled for each child every 6 to 9 months to prospectively update the shared care plan. Some patients in the intervention group eventually used the CCMS clinic more regularly for routine and sick care.
The control group continued to receive the usual care from their PCP, SCH, and other providers31 but did not have access to the CCMS clinic or providers.
We assessed health care quality by using the composite and global rating scores included in the Consumer Assessment of Healthcare Providers and Systems (CAHPS) version 4.0 Child Health Plan Survey,32 including supplemental CAHPS items related to care coordination for children with chronic conditions. Composite measure scores were constructed as described in the 2008 CAHPS Health Plan Survey and Reporting Kit.33 Parent reports of child functional status were assessed by using the 14-item Functional Status II-Revised measure.34 All enrolled parents were invited to complete surveys at baseline, 12 months, and 18 months by e-mail, regular mail, or telephone, on the basis of parental preference.
We used SCH administrative discharge data to assess hospital-based utilization including ED visits, hospital admissions, total hospital days, average LOS, ICU admissions at baseline, 12, and 18 months. Death of enrolled patients was assessed by using discharge data at 12 and 18 months. All outcomes were assessed per child-year, which was derived from the amount of time each child spent in the study. Patients with no SCH-based usage at 12 and 18 months were assumed to have had no usage; patients who moved out of state or died during the study period were censored at the date of the event if known. To minimize bias from extreme values and/or outliers on regression analyses, the highest 1% of usage outcomes for ED visits, hospital admissions, and ICU visits were winsorized at the 99th percentile.35 Average LOS was additionally log-transformed before analysis.36
We measured 2 economic outcomes: CCMS program implementation costs and costs attributable to overall health care usage. CCMS program costs represent net costs after accounting for costs offset by clinic revenue, and they include staff and physician salaries and benefits, as well as medical supplies. Overall health care costs were based on total health plan costs of care obtained directly from 5 payers who insured this patient population during the study period, and included the subtypes listed in Table 1 All costs were inflation-adjusted to 2014 US dollars by using the medical care services component of the consumer price index.37 Total intervention costs were calculated as the sum of CCMS program costs (intervention group only) and total health plan costs. Not all payers were willing to provide data and some only provided data at baseline and 12 months, resulting in missing cost data for some participants.
We estimated that 100 PCPs per group with 2 patients per PCP (200 children per group) would achieve 80% power to detect an absolute 10% reduction in ED or hospitalization risk at 12 months in the intervention group, compared with the 20% risk in the control group, assuming an intraclass correlation coefficient of 0.02 and α level of .05. Assuming at 12-month follow-up that the average control group “getting needed care” CAHPS score is 83 (SD 21) and the average “getting care quickly” score is 83 (SD 22),38 sample sizes of 100 PCPs per arm provide 80% power (intraclass correlation coefficient 0.02, α level .05) to detect a minimum increase in scores of 6.0 points and 6.3 points (considered moderately large increases: 0–100 scale)39,40 for the intervention compared with the control group.
Characteristics of enrolled patients and caregivers, as reported at baseline, were descriptively summarized. All analyses were conducted by using the intent-to-treat population of patients.
For health care quality, functional status, and hospital-based utilization outcomes, changes over time between intervention and control groups were assessed by using a difference-of-differences approach. Specifically, we fitted a linear or generalized linear mixed model to each outcome, with the main effects study group, time, study group by time interaction, and link function being based on outcome distributions. The model included PCP-level random effects, and repeated measures were modeled by using a spatial power covariance structure to accommodate unequally spaced time points. From the models, we estimated the mean change in outcomes from baseline to 12 months and from baseline to 18 months for the intervention and control groups, and we used Fisher’s exact tests generated by contrasts to assess whether the changes were significantly different between the intervention and control groups. Mortality was compared by using a 2-sample test of binomial proportions.
To address the skewness in cost data, we obtained bootstrap estimates of health care costs and bias-corrected accelerated 95% confidence intervals(CI) from 1000 bootstrap replicates drawn randomly with replacement.41 The highest 1% of costs was winsorized at the 99th percentile.35 Additionally, 8 extreme outliers were removed from subsequent analyses. Missing costs were multiply imputed (5 imputations) within each study arm and replicate by the Markov chain Monte Carlo method by using time-dependent outcomes and patient age at baseline.42 Change scores (baseline to 12 months and baseline to 18 months) were calculated for each patient. Change scores were aggregated over imputations; mean change scores were calculated by study arm; and differences between the control and intervention groups in change from baseline to 12 months and from baseline to 18 months (difference of differences) were calculated. Significance of the difference of differences was obtained from the bootstrap distribution, with missing outcomes imputed. We also performed a subanalysis on the children whose insurers were willing to provide complete billing data.
All analyses were conducted by using SAS version 9.4 (SAS Institute Inc, Cary, NC), with some data management performed by using Stata version 12 (Stata Corp, College Station, TX). To control for multiple comparisons, we controlled for false discovery rate by using q values,43,44 the proportion of false positives incurred if a particular comparison is called significant. We considered a comparison significant if the q value is ≤0.1 (ie, false discovery rate ≤10%).45
A total of 200 PCPs who cared for 331 CMC were randomly assigned, and 181 children were allocated to receive the intervention (Fig 1). Survey respondents at each time point are shown in the Consolidated Standards of Reporting Trials diagram. All living children were included in the analysis. Recruitment started on December 1, 2010, and ended on February 27, 2013; follow-up continued through the end of the fiscal year on September 29, 2014.
No differences in patient, caregiver, or PCP characteristics were noted between the intervention and control groups (Table 2). The majority of caregivers were female (n = 277, 84%) and had some college education (n = 233, 70%), and the majority of children were white (n = 208, 63%). There was no difference in SCH-based utilization, including ED visits, hospital admissions, total hospital days, average LOS, and ICU admissions, at baseline between the intervention and control groups.
We observed improvements in reports of receiving help with coordinating multiple providers relative to baseline in the intervention group compared with control group at 18 months (Table 3). We also observed improvements in the getting-needed-care-quickly composite scores and in the health care quality global ratings, relative to baseline, in the intervention group compared with the control group at 12 months only. We found no differences in doctor or specialist global ratings, other composite scores, or child functional status between the intervention and control groups at any time point.
We did not detect significant differences in SCH-based usage between the intervention and control groups, with 2 exceptions (Table 4). Although both study groups had decreases in hospital days per child-year and hospital stays over 7 days per child-year between the baseline and both subsequent time points, the observed decreases were larger in the control group than in the intervention group.
Change in overall health care costs over time, including health plan and clinic costs, was higher in the intervention group at 18 months (Table 1). The observed average cost of CCMS clinic usage was $3847 per child-year. Among subtypes of health plan costs, only the change in pharmaceutical costs over time was higher in the intervention group than in the control group at both 12 and 18 months. Similar findings were observed in the subanalysis that included children whose complete health plan cost data were provided (Supplemental Table 5).
CCMS, a multidisciplinary hospital-based service focused on improving care coordination for CMC, was designed to develop and support the implementation of an individualized shared care plan that empowered the child’s PCP and family. We observed that exposure to the CCMS generally improved quality of care, did not change functional status or hospital-based utilization, and did increase overall health care costs among CMC.
In the group receiving CCMS, we observed increases in both health care quality and costs relative to the decline in health care costs experienced by the control group and other similar cohorts.46 Because the service focused on care coordination, we believe that the CCMS potentially improved access to care, supplies, and services that were not available47–49 before exposure to the CCMS, which would naturally drive costs up. CCMS providers reported informally that many patients had health care issues that had not been addressed because families were overwhelmed. CCMS services illuminated existing gaps in care, helped families organize and/or prioritize care needs, and addressed gaps caused by insurance barriers and/or denials. This assertion is supported by parents also reporting improved coordination among multiple providers. Over time, as a child experiences higher quality and access to needed outpatient services, we hypothesize that both their hospital-based utilization and costs of care would decrease; had we been able to follow these CMC for a longer period, we might have observed improvements in hospital-based utilization. Alternately, creating an optimal care environment for CMC that addresses previously unmet needs and coordinates care with a whole-child approach may simply be more expensive.
Subsequent to the initiation of this study, Mosquera et al12 reported on a controlled trial of an enhanced medical home caring comprehensively for high-risk children. With this trial, they demonstrated a reduction in both the primary outcome of serious illness (defined as death, ICU stay, or hospital stay over 7 days) and costs. Our CCMS trial differed from the Mosquera et al12 trial in 3 critical ways. First, the intensity of the CCMS intervention was intermittent and focused on creation of an individualized standard shared care plan that supported the medical home. In contrast, Mosquera et al12 created an enhanced medical home for a small group of CMC. Second, the children that we targeted through CCMS had substantially lower illness severity. Only 51 (15%) of the CCMS population met Mosquera et al’s12 usage entry criteria of ≥3 ED visits, ≥2 hospitalizations, or ≥1 ICU visit in the previous year. Finally, in contrast to the cost data in Mosquera et al,12 CCMS cost data were (1) obtained in a comparable manner between intervention and control groups; (2) obtained directly from insurers, which may more accurately reflect real-world costs and better inform the development of a reimbursement model for private payers; and (3) inclusive of pharmacy, speech therapy, and home services. The findings of the 2 studies taken together reveal that intensive care coordination services directed by physicians might be most cost-effective when applied to the most severely ill CMC (including those in the palliative phase), whereas less resource-intensive care coordination services could be applied to less ill CMC (ie, “tiered care”50,51).
Our study has several limitations. It was conducted among CMC with PCPs at a single center with a unique approach to the management of CMC. However, the setting provided an opportunity to test this intervention. In addition, enrollment was challenging and did not achieve desired sample sizes, so the trial’s power was limited for utilization outcomes. In addition, given the study’s limited power, we were unable to conduct subgroup analyses to assess differences in outcomes by intervention intensity or by child characteristics such as race and/or ethnicity or language. PCPs and CMC who enrolled in the study may differ from those who declined to participate, and payers who did not provide data may differ from those who did. Those differences are unmeasured. Our analysis was conducted from the payer perspective and does not include out-of-pocket or indirect costs incurred by families, which can be significant.52 Finally, our analysis of hospital usage was limited to SCH, and inpatient care might have been obtained elsewhere. However, we performed analysis of health plan cost data that allowed for an indirect assessment of usage outside of our hospital.
Exposure to a hospital-based CCMS focused on care coordination for CMC generally improved quality of care, did not change functional status or hospital-based utilization, and increased overall health care costs. Tiered care providing care coordination services commensurate with disease severity may be a more cost-effective approach for CMC. The authors of future studies of interventions for CMC and their families should focus on refining (1) the population of CMC who most benefit from intervention(s), (2) the intervention approach, (3) the intensity and duration of intervention needed, and (4) outcomes to include not only utilization and cost but also similar quality of care measures.
We appreciate the efforts of all the providers in the CCMS clinic, including Andrea Barry-Smith, MSW, LICSW, JD; Mary Ehlenbach, MD; Susan Hunt, MD; Stacey Klontz, ARNP; Brett Leggett, MD; Nataline Lingren, ARNP; Emily Moon; Trina Roufs; and Barb York. All providers received compensation for their professional effort. We also appreciate the efforts of Daksha Ranade in obtaining SCH utilization data and Laurie Cawthon, MD, MPH, of the Washington Department of Social and Health Services in obtaining Washington state Medicaid data for study participants.
Pat Hagan, Mark Del Beccaro, David Fisher, Eileen O’Connor-King, and Margrette Ramirez were instrumental in executing agreements with health plans to provide cost data.
- Accepted July 25, 2017.
- Address correspondence to Tamara D. Simon, MD, MSPH, Seattle Children’s Research Institute, 1900 Ninth Ave, JMB 946, M/S JMB 9, Seattle, WA 98101. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by Seattle Children’s Hospital. Seattle Children’s Hospital had no role in the design of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. The funder/sponsor was involved in the decision to end the study. Washington state Medicaid, Regence BlueShield, Premera Blue Cross, Molina Healthcare, and the Community Health Plan of Washington provided insurance claims and payment data. The contents of this manuscript are solely the responsibility of the authors and do not necessarily reflect the official views of the funder/sponsor.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2017-2860.
- Simon TD,
- Berry J,
- Feudtner C, et al
- Lucile Packard Foundation for Children’s Health
- Cohen E,
- Kuo DZ,
- Agrawal R, et al
- Cooley WC; American Academy of Pediatrics Committee on Children With Disabilities
- Cooley WC,
- McAllister JW,
- Sherrieb K,
- Kuhlthau K
- Berman S,
- Rannie M,
- Moore L,
- Elias E,
- Dryer LJ,
- Jones MD Jr
- Stein RE,
- Jessop DJ
- Jessop DJ,
- Stein RE
- Jessop DJ,
- Stein RE
- Rahimy MC,
- Gangbo A,
- Ahouignan G, et al
- American Academy of Pediatrics
- Committee on Pediatric Emergency Medicine and Council on Clinical Information Technology
- American College of Emergency Physicians
- Pediatric Emergency Medicine Committee
- Foster CC,
- Mangione-Smith R,
- Simon TD
- Consumer Assessment of Healthcare Provider and Systems
- Bureau of Labor Statistics
- Schafer JL
- Agrawal R,
- Hall M,
- Cohen E, et al
- Kuo DZ,
- Goudie A,
- Cohen E, et al
- Johnson TL,
- Vostrejs M,
- Brewer D, et al
- Peltz A,
- Hall M,
- Rubin DM, et al
- Drummond MF,
- Sculpher MJ,
- Claxton K,
- Stoddart GL,
- Torrance GW
- Copyright © 2017 by the American Academy of Pediatrics