OBJECTIVES: We sought to develop and validate a method to identify social complexity risk factors (eg, limited English proficiency) using Minnesota state administrative data. A secondary objective was to examine the relationship between social complexity and caregiver-reported need for care coordination.
METHODS: A total of 460 caregivers of children with noncomplex chronic conditions enrolled in a Minnesota public health care program were surveyed and administrative data on these caregivers and children were obtained. We validated the administrative measures by examining their concordance with caregiver-reported indicators of social complexity risk factors using tetrachoric correlations. Logistic regression analyses subsequently assessed the association between social complexity risk factors identified using Minnesota’s state administrative data and caregiver-reported need for care coordination, adjusting for child demographics.
RESULTS: Concordance between administrative and caregiver-reported data was moderate to high (correlation range 0.31–0.94, all P values <.01), with only current homelessness (r = –0.01, P = .95) failing to align significantly between the data sources. The presence of any social complexity risk factor was significantly associated with need for care coordination before (unadjusted odds ratio = 1.65; 95% confidence interval, 1.07–2.53) but not after adjusting for child demographic factors (adjusted odds ratio = 1.53; 95% confidence interval, 0.98–2.37).
CONCLUSIONS: Social complexity risk factors may be accurately obtained from state administrative data. The presence of these risk factors may heighten a family’s need for care coordination and/or other services for children with chronic illness, even those not considered medically complex.
- ACE —
- adverse childhood experience
- CI —
- confidence interval
- COE4CCN —
- Center of Excellence on Quality of Care Measures for Children with Complex Needs
- CPS —
- child protective services
- ED —
- emergency department
- OR —
- odds ratio
- MHCP —
- Minnesota Health Care Programs
What’s Known on This Subject:
There is growing consensus that medically complex children may benefit from care coordination. However, children with noncomplex chronic disease may similarly benefit, particularly those children with social risk factors that can impede effective disease management.
What This Study Adds:
Social complexity risk factors can be successfully identified using state administrative data. Additionally, social complexity may be associated with increased need for care coordination among children with noncomplex chronic disease.
In 2011, the Centers for Medicare and Medicaid Services and the Agency for Healthcare Research and Quality identified a need to develop quality measures related to care coordination for children with medical complexity. This task was assigned to the Center of Excellence on Quality of Care Measures for Children with Complex Needs (COE4CCN), 1 of 7 Centers that constitute the national Pediatric Quality Measures Program mandated by the Children’s Health Insurance Program Reauthorization Act.1 The COE4CCN was simultaneously charged with developing a method to identify children with medical complexity using Medicaid administrative data, which eventually resulted in the creation of an algorithm that separates children into 3 categories: complex chronic disease, noncomplex chronic disease, and no chronic disease (healthy).2
Although the initial charge to the COE4CCN focused on care coordination for children with complex chronic disease, COE4CCN members also identified the need to determine whether children with chronic but noncomplex diseases, such as asthma, would have increased need for care coordination if they had social risk factors that might impede effective disease management and control. Recent work based on national survey data examining the role of adverse childhood experiences (ACEs) has shown that children with 2 or more ACEs are more likely to have problems getting referrals and needed care coordination.3 However, family characteristics and/or other contextual factors that may not be considered adverse experiences (eg, limited English proficiency) could also result in greater need for assistance managing the care of a noncomplex chronically ill child. Such families may be considered to have “social complexity,” as an analog to medical complexity that reflects a potentially heightened need for additional support due to social risk factors.
In a previous study, the COE4CCN found that social complexity risk factors identified using Washington state Medicaid client administrative data were associated with increased risk of emergency department (ED) visits and decreased timely adherence to recommended well-child care visits independent of a child’s level of medical complexity.4 However, as it is unclear whether administrative data accurately represent social complexity risk factors, the current COE4CCN study sought to assess the validity of identifying such risk factors using this data source. Validating a method to identify children with social complexity risk factors using state administrative data rather than surveys would extend the previous literature that relied on caregiver surveys to measure social risk factors, including ACEs,3 and would help to identify more children who might benefit from interventions, such as care coordination.
The current study had 2 aims. First, we sought to test the COE4CCN’s previously developed methodology for identifying social complexity risk factors, established using Washington state data, with a second state administrative database.4 Second, we sought to validate the accuracy of these data for correctly identifying children with social complexity risk factors, including examining the association between social complexity risk factors and need for care coordination, using caregiver survey reports as the gold standard.
The current study relies on a consensus definition established by the 43 COE4CCN members through an iterative process. Hereafter, social complexity is defined as a set of co-occurring individual, family or community characteristics that can have a direct impact on health outcomes or an indirect impact by affecting a child’s access to care and/or a family’s ability to engage in recommended medical and mental health treatments. To address the study aims, we compiled a variety of administrative social risk factor indicators and assessed whether the risk factors found in state administrative data were correlated with comparable social complexity risk factors obtained via caregiver report. Secondarily, we conducted a regression analysis to investigate whether social complexity, as operationalized in state administrative data, was associated with caregiver-reported need for care coordination.
Using the Minnesota Health Care Programs (MHCP) public health care claims and encounter database, we identified children who had visited 1 of 50 pediatric clinics in the previous 14 months and whose providers were members of the Minnesota chapter of the American Academy of Pediatrics, a member organization of the COE4CCN. Children seen by these providers were eligible for inclusion in the study if they were age 0 to 17 years, had been enrolled at least 3 months of the previous year in a public health care program, lived with at least 1 custodial parent who was also enrolled in a public health care program for at least 3 months, and who were identified as having a “noncomplex chronic” condition based on the Pediatric Medical Complexity Algorithm.2 Among the 2966 children identified in this process, 88.5% (2625) were deemed eligible to participate in the study. Most were enrolled in Medical Assistance, Minnesota’s Medicaid program, although some were enrolled in MinnesotaCare, Minnesota’s state-funded program for participants whose income is low but higher than the Medicaid cut-off.
Figure 1 presents the CONSORT flow diagram depicting study recruitment and attrition. We invited 2625 eligible families to participate in the study by mail and provided them the option of returning a study refusal card. A total of 176 of the families (7%) returned the refusal card, and 1546 (59%) were unreachable before the close of study enrollment, which ran from August to December 2013. We attempted to contact by phone the 903 remaining families for recruitment/enrollment, of which 460 (51%) agreed to participate, 374 (41%) explicitly or passively declined to participate, and an additional 69 (7%) were unreachable after 10 calls. In all, 460 complete surveys were obtained, yielding an American Association of Public Opinion Research response rate of 17.5%, cooperation rate of 45.5%, refusal rate of 21%, and contact rate of 38.5%. Surveys were administered in English or Spanish via telephone to caregivers of children in the study population. Verbal consent by phone was obtained before survey administration. All study procedures were approved by the institutional review boards at the Minnesota Department of Human Services, Children’s Hospital Los Angeles, and Seattle Children’s Research Institute.
Table 1 lists the operational definitions of social complexity risk factors in the state administrative database and in the caregiver survey.
Social Complexity Risk Factors Measured in Minnesota Administrative Data
The MHCP claims and encounter database used to identify participants also provided administrative measures of social complexity risk factors. We included 5 variables previously identified as predictors of increased ED visits and decreased timely adherence to recommended well-child visits: severe poverty (defined as income ≤50% of the federal poverty level), the involvement of child protective services (CPS), caregiver with severe and persistent mental illness, caregiver with limited English proficiency, and homelessness.4 We included 3 additional risk factors (caregiver chemical dependency,3,5 caregiver physical disability,6 and discontinuous child insurance coverage7,8) previously identified in the literature review but which could not be examined in the Washington study. For analytic purposes, all social complexity risk factors were constructed as indicator variables (1 = risk; 0 = otherwise), and a binary measure of any social complexity was created from these indicators (1 = any social complexity risk factor; 0 = none). Analyses were based on all available use data for eligible children and their parents, from all Medicaid-covered services.
Social Complexity Risk Factors Measured by Caregiver Survey
Of the 8 social complexity risk factors identified in the state administrative database, 6 were also measured by a caregiver survey assessing health and sociodemographic status, including housing and neighborhood, family composition, and demographics (Table 1). Eight survey items related to caregiver physical and mental health were adapted from the 12-Item Short Form Health Survey.9 Limited English proficiency was assessed with a combination of items capturing language spoken at home and self-reported ability to speak English on a 4-point response scale, based on the US Census framework.10 De novo items assessed insurance discontinuity of at least 3 months, homelessness, and CPS involvement. The COE4CCN parent advisory panel from Minnesota Family Voices, which reviewed the proposed survey, considered 2 proposed items related to income level and parent chemical dependency to be too sensitive to ask in a telephone survey and unlikely to be measured reliably due to social desirability bias. These items were removed from the survey and were not considered further in the study. Cognitive interviews to assess item understandability were conducted with 3 caregivers of children with noncomplex chronic conditions cared for in a safety-net clinic in Seattle, WA. These interviews did not result in any additional changes to the survey. Caregiver-reported need for care coordination, which was examined as a secondary validation measure, was operationalized as a binary score based on a positive response to any of 3 items (2 adapted from the National Survey of Children with Special Health Care Needs11 and a third item related to assistance with obtaining community services developed for this study).
Tetrachoric correlations were used to quantify concordance between the binary measures of social complexity obtainable in the Minnesota state administrative data and caregiver-reported social complexity risk factors. A logistic regression analysis subsequently examined the association between the presence of ≥1 social complexity risk factor and need for care coordination, controlling for the child’s age in years (range, 0–17 years), gender (1 = male, 0 = female), and race/ethnicity (categorized as non-Hispanic white, non-Hispanic black, Hispanic, and other, with non-Hispanic white as the omitted reference category).
Table 2 presents descriptive statistics reflecting the demographic makeup and prevalence of social complexity risk factors among the sample. Based on the Minnesota state administrative data, 326 families (71% percent of the sample) had at least 1 social complexity risk factor, compared with 335 (73%) identified by the caregiver survey. The most common documented risk factor was poverty (ie, income ≤50% federal poverty level), identified in 53% of the sample.
Validation of Social Complexity Risk Factors Identified Using State Administrative Data
We were able to examine concordance for 5 of the 8 social complexity risk factors present in the administrative database. As noted earlier, caregiver chemical dependency and poverty were not assessed in the survey, and we were unable to examine concordance for the CPS involvement risk factor as the final sample included only 1 foster caregiver. Significant correlations were found between administrative and survey-based measures of caregiver physical disability, caregiver serious and persistent mental illness, child insurance discontinuity of at least 3 months, limited English proficiency, and a history of homelessness in the previous 2 years (r values ranging from 0.31 [P < .01] to 0.94 [P < .001]; Table 3). However, there was no association between the administrative indicator of whether the parent was currently homeless and the parent’s self-reported precarious housing (ie, endorsing a housing option other than owning or renting one’s home; P = .95).
Logistic regression results are presented in Table 4. The presence of any social complexity risk factor was associated with a higher likelihood of caregiver-reported need for care coordination (unadjusted odds ratio [OR] = 1.65; 95% confidence interval [CI], 1.07–2.53). After controlling for demographic characteristics, social complexity trended toward an association with need for care coordination but was no longer a statistically significant predictor (P = .06; adjusted OR = 1.53; 95% CI, 0.98–2.37).
This study complements and extends previous work on ACEs by demonstrating that valid measures of social complexity risk factors may be obtained in state administrative data not originally collected for this purpose. We found a high degree of concordance between social complexity risk factors identified using administrative data and the same risk factors assessed using caregiver survey reports, including caregiver physical disability and serious and persistent mental illness, child insurance discontinuity of at least 3 months, limited English proficiency, and a history of homelessness. Our results lend credibility to the idea that Medicaid agencies can validly identify children with a subset of important social complexity risk factors using the administrative data already available, without having to collect additional information from families beyond the documents already required to enroll in Medicaid coverage. These results may inform future research efforts (eg, identifying children with social complexity risk factors for secondary data analyses) as well as policies for children and families served by public insurance, who may benefit from provision of targeted services at the state or local level.
Furthermore, results suggest that this sample of children relying on Minnesota public health care programs have been exposed to many social risk factors in addition to financial need, which generally triggers their eligibility. It is especially concerning that 10% of children in our sample had a parent with a recent chemical dependency diagnosis, 8% had a parent with a severe and persistent mental illness, and 16% had an open CPS case within the past 3 years. Such information could be invaluable to Medicaid agencies as they try to identify and support the physical and mental health of children at greatest risk for poor health outcomes.
Results also suggests that for children with noncomplex chronic conditions, the presence of at least 1 social complexity risk factor may be associated with increased caregiver-reported need for care coordination, although this relationship was no longer statistically significant after controlling for demographic variables, particularly race/ethnicity. As caregivers of non-Hispanic black children were more likely than caregivers of non-Hispanic white children both to experience at least 1 social complexity risk factor (76% vs 68%) and to endorse heightened need for care coordination (51% vs 32%), the attenuated association between social complexity and need for care coordination may be evidence of a statistical suppression effect12 after the introduction of the race/ethnicity variables.
Importantly, the results of this study, which is based on a sample of families with public insurance and known providers, also suggest that although families with social complexity may technically have “access” to care (eg, the ability to secure an appointment with a care provider), they may nonetheless lack the ability or resources to fully use these providers or to coordinate the services rendered by these providers. Previous work suggests that providing “potential access” does not equate to “realized access,” particularly for socially at-risk families.13 We may need to consider broadening how patient access to care is measured or evaluated, particularly among families with social complexity.
The availability of public program data differs from state to state (and database to database), which raises questions about the ease of generalizing results. Whereas claims and encounter data are standard across states and are likely compiled within the agency or organization that administers the programs, there is substantial variation among Medicaid enrollment forms across states. In Washington, for example, indicators of parental death and intimate partner violence are available that were not assessed in Minnesota; conversely, we were able to include measures of chemical dependency, disability, and insurance discontinuity using Minnesota data that could not be modeled in the Washington study. Other states may have additional measures of social complexity risk factors that should be considered beyond those discussed in the current study.
Furthermore, individual state agencies may or may not have access to these data. The Minnesota Medicaid agency, for example, had access to child protection data because the Minnesota Department of Human Services administers both types of programs. Replicating the current results in other states may be more or less challenging depending on how state services are organized, what types of program data are available, and how social complexity risk factors are operationalized. Collectively, our results using the comprehensive Minnesota administrative database highlight the opportunity to obtain a much richer understanding of social complexity if data from multiple sources are combined at the state or federal level.
A challenge to generalizability inherent to our study design is that this sample was selected based on the child having recently accessed health care from an American Academy of Pediatrics member physician. This may have greatly limited our ability to identify problems with access to care and biased associations between social complexity and need for care coordination toward the null. Additionally, given that we were unable to formally validate 3 of the social complexity risk factors available in the administrative data, the identification of families with “any social complexity” may be overly broad. This too may have biased regression results toward the null, if we included families identified administratively as socially complex who would not have identified themselves that way. Alternately, the small difference in families identified by each method (326 in the MHCP database vs 335 via the survey) suggest that administrative data may slightly underestimate families with social complexity risk factors.
Finally, the small percentage of the Minnesota sample reporting some risk factors (eg, homelessness) makes it difficult to get a clear picture of their impact; results might differ for populations in which these risk factors are more prevalent. Because our sampling approach may have disproportionately excluded homeless families by requiring a known, stable mailing address and phone number, alternative methods may be needed to study the impact of social complexity among the most heavily disadvantaged families. Similarly, our sample may be biased toward the inclusion of higher-functioning caregivers and those not currently experiencing an exacerbation of physical or mental conditions that may have precluded some potentially eligible respondents from opting in to study participation.
Although we found high concordance overall between the survey and administrative data, a different caregiver may have completed the survey than had been included in the administrative dataset. This may partially explain disagreements between the 2 data sources, particularly for the caregiver health measures, but may also challenge the interpretability of our results. Additionally, we did not examine connections between social complexity and child health outcomes (eg, ED/hospital use), and this remains an important direction for future study.
Despite growing consensus that medically complex children may benefit from care coordination, less is known about the needs of children with noncomplex chronic disease, particularly those with social risk factors that may impede effective management of their conditions. We found that social complexity may be accurately ascertained from state administrative data and trended toward association with greater caregiver-reported need for care coordination. Children with noncomplex chronic disease may be vulnerable to social risks that increase their family’s need for care coordination or other services, and future work to identify a common set of indicators in administrative databases, including data belonging to commercial payers and other health systems, may help to identify and include all families who may benefit from these interventions.
We thank the COE4CCN and in particular the Social Complexity Working Group for their contributions to this study.
- Accepted June 30, 2016.
- Address correspondence to Sheree Schrager, PhD, MS, Division of Hospital Medicine, Children's Hospital Los Angeles, 4650 Sunset Blvd, MS 94, Los Angeles, CA 90027. E-mail:
FINANCIAL DISCLOSURES: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This study was funded by a cooperative agreement with the Agency for Healthcare Research and Quality and the Centers for Medicare and Medicaid Services, grant #U18HS020506, part of the Children's Health Insurance Program Reauthorization Act Pediatric Quality Measures Program.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Simon TD,
- Cawthon ML,
- Stanford S, et al; Center of Excellence on Quality of Care Measures for Children with Complex Needs (COE4CCN) Medical Complexity Working Group
- Bethell CD,
- Newacheck P,
- Hawes E,
- Halfon N
- Arthur KC,
- Lucenko BA,
- Sharkova IV, et al
- Kogan MD,
- Newacheck PW,
- Honberg L,
- Strickland B
- Benedict RE
- National Center for Health Statistics
- Copyright © 2016 by the American Academy of Pediatrics