OBJECTIVE: Using structural equation modeling, we sought to assess the system of services for children with special health care needs (CSHCN) and their families by using 14 key indicators of functional abilities of CSHCN, health insurance coverage, access to care, and the impact of their conditions on their families.
METHODS: With data from the 2001 and 2005–2006 National Surveys of Children With Special Health Care Needs, we used confirmatory factor analysis for ordered-categorical measures to model the relationship between an indirectly observed (ie, latent) variable and the key indicators and evaluate changes in this relationship over time.
RESULTS: For both survey periods, a single-factor model fit well. The latent construct was defined as the well-being of the health care environment for CSHCN and their families. Family financial problems caused by the child's condition, unmet needs for family support services, and negative impact on employment were most strongly related to the latent well-being construct. The lowest levels of the well-being construct were associated with families that had unmet needs for support services, CSHCN who lacked a usual place for care, and families that spent ≥11 hours/week providing or coordinating care. CSHCN and their families with family-centered care and with adequate health insurance were likely to have average or better levels of the well-being construct. Mean levels of the well-being construct were unchanged over time.
CONCLUSIONS: The 14 key indicators can be used to reliably assess a single latent construct. The relative ordering of the indicators' thresholds (a model parameter) may be useful for guiding pediatricians' evaluations of the health care environment for CSHCN and their families. Researchers may use the scores available from the latent-variable model to assess outcomes related to the health care environment and the system of services for CSHCN and their families.
- special health care needs
- health care environment
- service system
- latent variables
- structural equation modeling
A successful system of services for children with special health care needs (CSHCN) results in high levels of child and family health and well-being.1 The Maternal and Child Health Bureau (MCHB) has identified 15 key indicators of functional abilities, health insurance coverage, access to care, and family impact that are available from the National Survey of Children With Special Health Care Needs (NS-CSHCN).2,3 These indicators have been described generally as measures of the well-being of CSHCN and their families,4–7 but perhaps they are better described as child- and family-level measures of the performance of the system of services. Health care–related problems experienced by CSHCN and their families provide information about the service system.1
In this report, we examine whether these indicators can be considered collectively, perhaps as part of an overall assessment of service-system performance. National and state-level estimates for these indicators are currently available, and much has been written about individual indicators. Yet, little research has been performed to describe the relationships among the indicators. We propose that a single latent variable can reasonably account for these relationships among the indicators and that, therefore, these indicators can be used to assess the health care environment.
A LATENT-VARIABLE APPROACH
A few notes about our terminology are in order. There are many phenomena that are not directly observable. Researchers call these latent variables.8 The fact that one cannot directly observe latent variables does not make them unmeasurable. For example, because pain is not directly observable, physicians routinely ask about patients' pain levels and observe their behavior. By asking questions about pain, physicians indirectly measure pain. When physicians ask several slightly different but related questions about pain, the physician is able to use the answers to more accurately measure the pain. In the same way, we propose to assess the health care environment by examining responses to and correlations among indicators of functional abilities, health insurance coverage, access to care, and the impact of their conditions. We accomplished this cross-sectionally and across time by using a mathematical model: multiple-group confirmatory factor analyses for ordered-categorical measures.9,10
We describe this latent variable as representing the “well-being of the health care environment” rather than the functioning or performance of the service system. Well-being implies an outcome, whereas functioning and performance describe process-related variables. The indicators may reflect the performance of the system, but they first and foremost reflect outcomes of the system's performance (eg, impaired functioning, inability to pay for or otherwise access needed care, family struggles caused by the child's health). The outcomes available from the NS-CSHCN do not include community health and well-being, which are critical outcomes when assessing a system of services.1 Instead, we assess the health care environment, a network of services that is unique for each child and is smaller than the community-based system but broader than any single clinic or practice.
We examined whether a single latent variable reasonably accounts for the covariances (ie, relationships) among the indicators. Next, we tested whether CSHCN with high scores on this latent variable correspond to CSHCN who are generally doing well on each indicator and vice versa. Finally, we hypothesized that the indicators do not relate to the latent variable equally and that, across time, the relation of specific indicators with the latent variable can change.
Data for this study come from the responses of parents and guardians of 79589 CSHCN. These parents were interviewed as part of either the 2001 or 2005–2006 NS-CSHCN. The survey design for 2005–2006 is described briefly by Kogan et al in this supplemental issue of Pediatrics11; more in-depth information about both the recent and previous surveys can be found elsewhere.12,13
From the NS-CSHCN, the MCHB identified 15 key indicators of the functional abilities of CSHCN, health insurance coverage, access to care, and the impact of their conditions on their families.2,3 For this study, we examined the relation between the latent variable and all but 1 of these key indicators. “Uninsured at time of the survey” was not included, because this indicator was nearly redundant with “uninsured at some time during the previous year.” The 14 indicators are listed in Table 1. Specifications for these indicators can be found elsewhere.14,15
Multiple-group confirmatory factor analyses for ordered-categorical measures involves specifying a measurement model (Fig 1). Model parameters (eg, loadings and thresholds) describe how the latent variable and indicators relate.16 Loadings and thresholds describe different kinds of information. A loading, like a correlation, describes how strongly an indicator relates to the latent variable. Thresholds describe the level of the latent variable necessary before an affirmative indicator response becomes likely (probability ≥ 50%). One describes the strength of the relationship; the other describes the “location.” Two indicators could be equally strongly associated with the latent variable (equal loadings), but affirmative responses could reflect very different average levels of the latent variable (unequal thresholds). Similarly, 2 indicators could reflect similar average levels of the latent variable (equal thresholds), but 1 indicator could be a clear marker of that level, whereas the other provides only a weak suggestion of that level (unequal loadings).
First, we tested the measurement model's fit in the 2 CSHCN cohorts. Fit describes the measurement model's ability to reasonably account for the covariances (relationships) among the indicators in the model.17 We examined several goodness-of-fit indices: the root-mean-square error of approximation (RMSEA); comparative fit index (CFI); Tucker-Lewis index (TLI); gamma hat; and McDonald's noncentrality index (NCI). For the RMSEA, values of <0.05 suggest ideal fit. For the CFI, TLI, and NCI, guidelines favor values near or greater than 0.90. For gamma hat, guidelines suggest that values of >0.95 indicate good fit.17
Second, we examined whether the relationship between the indicators and well-being remained the same across the 2 survey time periods by incorporating mathematical constraints into the model. If constraining a parameter to equality across time led to fit deterioration, we could conclude that the relationship between the latent variable and the indicator differed across time. Because our model included a mean for each of the 2 groups in our analysis (the 2001 and 2005–2006 CSHCN cohorts), we could also evaluate changes in the mean level of the latent variable across time. By setting the mean of the 2001 cohort to 0, we can evaluate change across time in the 2005–2006 cohort's mean relative to 0. Negative values reflected poorer levels of the latent variable.
The majority of goodness-of-fit indices indicated that the measurement model (Fig 1) fit well (RMSEA: 0.03; McDonald's NCI: 0.98; gamma hat: 0.99; TLI: 0.87; CFI: 0.84). This result suggested that a single latent factor accounted for the relationships among the indicators. Next, we examined the equality of loadings and thresholds across time. We first compared a model with unconstrained loadings and thresholds to one that constrained the loadings to equality across time. The added constraints led to statistically significant misfit (Δχ2 = 89.76; degrees of freedom = 12; P < .001). Parameter-level investigations that controlled for inflated type 1 errors18 indicated that allowing the loadings for 5 indicators (referral difficulty, usual place for care, personal doctor/nurse, family-centered care, and high medical expenses) to vary across time provided better fit (RMSEA: 0.027; McDonald's NCI: 0.98; gamma hat: 0.99; TLI: 0.88; CFI: 0.86).
We then compared the constrained loadings and unconstrained thresholds model to one that constrained the thresholds to equality. The equivalence constraints were not supported (Δχ2 = 631.540; degrees of freedom = 17; P < .001). A model that allowed the thresholds for 10 indicators (uninsured in the previous year, unmet health care needs, inadequate insurance, usual place for care, personal doctor/nurse, family-centered care, high medical expenses, family financial problems, time burden providing care, and employment impact) to vary across time provided better fit (RMSEA: 0.027; McDonald's NCI: 0.98; gamma hat: 0.99; TLI: 0.88; CFI: 0.86).
In sum, our analyses indicated that a single-factor measurement model that allowed some loadings and thresholds to differ across time best fit the data. As noted earlier, we chose to describe this single latent construct as the well-being of the health care environment for CSHCN and their families (hereafter, “well-being”). Additional analyses (not detailed because of space constraints) revealed that (1) the model remained robust to different statistical identification, (2) deleting any single indicator did not improve fit, and (3) a 2-factor model that separated indicators into family and child groupings did not provide better fit.
Table 1 presents the parameters associated with the final model. With respect to the loadings, across time, 3 indicators had the largest loadings: family financial problems caused by the child's conditions; unmet needs for family support services; and employment impact of the child's conditions. These 3 indicators were the most strongly related to the well-being construct at either time. The relatively small value of most other loadings indicated that, although the well-being construct was related to responses to these indicators, the relationship was not as strong.
The model also shows that the strength of the relationship between some of the indicators and the well-being construct changed across time. Lack of a personal doctor/nurse, lack of family-centered care, difficulty obtaining needed referrals, and high medical expenditures were not as strongly related to the well-being construct in 2005–2006. Conversely, having no usual place for care demonstrated a slightly stronger relationship with the well-being construct in 2005–2006. The remaining loadings maintained equivalent relationship strength across time.
With regard to the thresholds, in 2005–2006, the lowest levels of the well-being construct were indicated by unmet needs for family support services, families that spent ≥11 hours/week providing or coordinating care, lack of a usual place for care, and lack of a personal doctor or nurse. Relatively better levels of the well-being construct were indicated by receipt of family-centered care and having adequate insurance.
Some mean shifts in the level of the well-being construct occurred across time. Reports of out-of-pocket medical expenses greater than $1000 indicated substantially poorer levels of the well-being construct in 2001 than in 2005–2006. Conversely, lack of a personal doctor/nurse, lack of a usual place for care, employment impact of the child's conditions, and reports of families spending ≥11 hours/week providing or coordinating care all indicated appreciably poorer levels of the well-being construct in 2005–2006 than in 2001. Three indicators were associated with equivalent levels of the well-being construct across time: unmet needs for family support services; difficulty obtaining needed referrals; and ≥11 days of school absences.
Mean levels of the well-being construct did not change significantly across time (2001 mean: 0; 2005–2006 mean: 0.01; z = 0.62; P = .54; d = 0.02).
Health care–related problems experienced by CSHCN and their families provide information about the service system.1 Our analyses support the hypothesis that these 14 indicators of health care–related problems can be used to reliably assess a single latent construct. We interpret this construct as representing the well-being of the health care environment for CSHCN and their families.
This construct, as assessed by using these indicators, forms a continuum. The thresholds show that each indicator corresponds to a slightly different level on this continuum. CSHCN whose families had unmet needs for support services such as family counseling, respite care, or genetic counseling had the lowest well-being levels. Lack of a usual place for care and/or a family that spent ≥11 hours providing or coordinating care characterized the next lowest well-being levels.
The relative ordering of the thresholds for each indicator has clinical implications. It offers a method for assisting diagnosis when pediatricians or social workers attempt to assess the well-being of the health care environment for CSHCN and their families. If a parent reports problems relevant to a particular indicator, the likelihood that problems with greater (more positive) threshold levels are also present is generally >50%. For example, if a parent reports that family members have cut back or stopped working because of the child's health, there is a strong likelihood that the child has high medical expenses, activity limitations, and inadequate insurance and a strong likelihood that the family had difficulty obtaining needed referrals and family-centered care. Given a concern reported by a parent (perhaps during a well-child visit), pediatricians and social workers may find it useful to use the relative ordering of the indicators' thresholds to identify and document other less “severe” problems that are likely present. (Here, less severe problems are defined as those that, when reported, would generally reflect more positive levels of the construct.) Pediatricians and social workers may also wish to ask about slightly more severe problems that could reveal that the “true” level of the well-being construct for CSHCN and their families is worse than that indicated by the parents' reported concern.
The analyses presented here show that the health care environments for CSHCN with adequate insurance and with family-centered care are likely to have average or better scores on the well-being construct. As a result, pediatricians and others could use reported problems with either of these 2 indicators as early warning signs of problems with the health care environment. If the health care environment begins to deteriorate, the first noticeable consequence will likely be complaints about insurance coverage or complaints that the child's doctors do not sufficiently listen, provide needed information, or spend enough time with the child. Interventions at that time could prevent further problems with the health care environment and possibly restore more positive levels of well-being. For example, intervention by social workers could help parents avoid cutting back on work hours and could help parents negotiate coverage for needed care with insurance companies.
As suggested earlier, the most immediate concerns offered by parents may not reflect the most severe problems regarding the health care environment. Clinicians concerned about this problem could choose to formally administer a questionnaire that includes the indicators. We recognize that it is not reasonable to expect most pediatricians to administer dozens of items to parents in their practices and score the results by using sophisticated statistical techniques. Yet, administering the survey items in the clinical setting would standardize assessment and would provide a reliable score for evaluating changes in the well-being of the health care environment over time. One limitation in this approach is worth noting. The indicators addressed in this article do not assess the full range of the well-being construct. All of the thresholds were negative, suggesting that the indicators cannot be used to identify when the health care environment for CSHCN and their families is flourishing.
The Child and the Family
The MCHB has recognized the central role of family-centered care in ensuring positive health care outcomes for CSHCN and their families.6 Indeed, the 3 indicators most strongly associated with the latent well-being construct were related to the impact of the child's health and health care on the family: financial problems caused by the child's conditions; unmet needs for family support services; and employment impact of the child's conditions. Moreover, a 2-factor model that attempted to separate family-oriented from child-oriented indicators fit more poorly than the more parsimonious single-factor model. Therefore, it may be reasonable to conclude that the well-being of the health care environment for the family is largely indistinguishable from the well-being of the health care environment for CSHCN when using these indicators. Such a finding supports the conclusions of the American Academy of Pediatrics Task Force on the Family that pediatricians have a role in supporting both children and families.19
The Ontology of Well-Being
Some may question whether this well-being construct exists independently of the indicators that we used to assess it. The MCHB did not select the 14 indicators for the specific purpose of assessing the well-being of the health care environment. However, the good fit of the single-factor model and the robustness of the model to the selection of indicators suggest that the well-being construct is more than just the sum of its parts.20 Researchers may find that the well-being scores available from the latent variable model are useful in regression models when there is a desire to assess outcomes related to the health care environment and inclusion of all the indicators would yield multicollinearity problems.
Change Over Time
For some, the utility of the model will not only come from these scores but also will depend on the scores' ability to predict future outcomes and on their sensitivity to change as a result of programs or interventions to improve the system of services for CSHCN. The analyses presented here did not address these issues. However, for others, the utility of the measurement model will come from its ability to describe changes in the relation of an indicator or indicators to the well-being of health care environments across time, perhaps as a result of effective policy or economic changes. Our analyses did address sensitivity to change over time in this sense.
Although nationally representative mean levels of the well-being construct did not change from 2001 to 2005–2006, we did see 1 expected change over time in the relationship between some indicators and the latent construct. Health care costs rose substantially from 2001 to 2005–2006,21,22 and the percentage of CSHCN with out-of-pocket expenses greater than $1000 also increased.3 It is not surprising, therefore, that high levels of expenses were indicative of poorer levels of the well-being construct in 2001 than in 2005–2006. All other changes over time were less pronounced. Some of the minor changes in the relationships between indicators and the construct could have occurred as a result of minor changes in the survey questionnaire from 2001 to 2005–2006.23
Finally, we note that, over time, an indicator's relation to the well-being construct could become substantially stronger or substantially weaker. For example, if all CSHCN became adequately insured, we would no longer expect this indicator to relate strongly to the well-being of the health care environment. In contrast, if it became increasingly difficult on average for families to acquire adequate insurance, then successfully acquiring insurance could become a particularly salient and reliable indicator of the well-being of the health care environment. We did not observe substantial changes over time in the size of the loadings (ie, the strength of the relationship) with any of the indicators examined here. Still, researchers may wish to use changes over time in parameter estimates from latent variable models to investigate the results of policy changes for CSHCN and their families.
- Accepted August 3, 2009.
- Address correspondence to Stephen J. Blumberg, PhD, Centers for Disease Control and Prevention, National Center for Health Statistics, 3311 Toledo Rd, Hyattsville, MD 20782. E-mail:
Financial Disclosure: The authors have indicated they have no financial relationships relevant to this article to disclose.
- ↵US Department of Health and Human Services, Health Resources and Services Administration, Maternal and Child Health Bureau. The National Survey of Children With Special Health Care Needs Chartbook 2001. Rockville, MD: US Department of Health and Human Services; 2004
- ↵US Department of Health and Human Services, Health Resources and Services Administration, Maternal and Child Health Bureau. The National Survey of Children With Special Health Care Needs Chartbook 2005–2006. Rockville, MD: US Department of Health and Human Services; 2007
- ↵Denboba D, McPherson MG, Kenney MK, Strickland B, Newacheck PW. Achieving family and provider partnerships for children with special health care needs. Pediatrics.2006;118 (4):1607– 1615
- ↵Margolis LH, Mayer M, Farel AM, Clark KA. State capacity and the well-being of children and youth with special health care needs. Paper presented at: annual meeting of the American Public Health Association; December 10–14, 2005; Philadelphia, PA
- ↵Bollen KA. Structural Equations With Latent Variables. New York, NY: Wiley; 1989
- ↵Kogan MD, Strickland BB, Newacheck PW. Building systems of care: findings from the National Survey of Children With Special Health Care Needs. Pediatrics.2009;124 (suppl 4):S333– S336
- ↵Blumberg SJ, Olson L, Frankel M, et al. Design and operation of the National Survey of Children With Special Health Care Needs, 2001. Vital Health Stat 1.2003;(41):1– 136
- ↵Blumberg SJ, Welch EM, Chowdhury SR, et al. Design and operation of the National Survey of Children With Special Health Care Needs, 2005–2006. Vital Health Stat 1.2008;(45):1– 188
- ↵Child and Adolescent Health Measurement Initiative. 2001 National Survey of Children With Special Health Care Needs SPSS Codebook. Portland, OR: Data Resource Center for Child and Adolescent Health; 2007
- ↵Child and Adolescent Health Measurement Initiative. 2005/2006 National Survey of Children With Special Health Care Needs SPSS Codebook. Portland, OR: Data Resource Center for Child and Adolescent Health; 2008
- ↵Hu L, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling.1999;6 (1):1– 55
- ↵Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol.1995;57 (1):289– 300
- ↵Catlin A, Cowan C, Hartman M, Heffler S; National Health Expenditure Accounts Team. National health spending in 2006: a year of change for prescription drugs [published correction appears in Health Aff (Millwood). 2008;27(2):593]. Health Aff (Millwood).2008;27 (1):14– 29
- ↵Child and Adolescent Health Measurement Initiative. National Chartbook Indicators: Summary of Key Differences in Measurement—2001 to 2005–2006. Portland, OR: Data Resource Center for Child and Adolescent Health; 2007
- Copyright © 2009 by the American Academy of Pediatrics