Background. There currently are few published data evaluating the effect of State Children's Health Insurance Programs on health care outcome measures in children. Colorado's Child Health Plan Plus (CHP+) is a non-Medicaid State Children's Health Insurance Program that began enrollment in April 1998. The objectives of this study were to compare reported (1) access to care, (2) utilization of health care, and (3) quality of care during the year before and the first year after enrollment into CHP+.
Methods. We interviewed 480 randomly selected families by telephone 2 months after their first enrollment into CHP+ (September 1999 to January 2000) and, again, 1 year later. We used generalized linear models to examine the effect of enrollment on health care access, utilization, and quality while controlling for type of previous insurance, length of time uninsured before enrollment, race/ethnicity, and age.
Results. Regarding access to care, the percentage of families who reported a usual site of preventive care did not change significantly, but families reported more often being able to see providers as soon as desired for routine care (incidence ratio [IR]: 2.03; 95% confidence interval [CI]: 1.37–3.02]), for care when sick or injured (IR: 2.77; 95% CI: 1.85–4.16), for specialty care (IR: 1.96; 95% CI: 1.16–3.32), and for all health care (IR: 2.35; 95% CI: 1.81–3.07). Unmet medical needs decreased after versus before enrollment for prescription medications (IR: 0.38; 95% CI: 0.26–0.55), mental health care (IR: 0.63; 95% CI: 0.40–0.97), prescription glasses (IR: 0.44; 95% CI: 0.29–0.65), and dental care (IR: 0.59; 95% CI: 0.47–0.76). Regarding utilization, the proportion who saw a provider for routine care in the past year increased (IR: 1.39; 95% CI: 1.06–1.83), but reported visits for sick, specialty, and emergency department care and hospitalizations did not increase. Regarding quality of care, the proportion who rated their health care as “best” increased (RI: 1.31; 95% CI: 1.04–1.66) after versus before enrollment.
Conclusions. Families who were newly enrolled into CHP+ perceived dramatic increases in access to all types of care and decreases in unmet medical needs, no increase in utilization of emergency department or hospitalization services, and improved overall quality of care in the year after enrollment into CHP+.
In response to the growing problem of uninsured children, the Balanced Budget Act of 1997 created the State Children's Health Insurance Program (SCHIP), providing federal funding to help states expand the provision of health insurance to uninsured low-income children. Currently, 20 states have stand-alone, non-Medicaid SCHIP programs, 14 have Medicaid expansion programs, and 17 are operating programs that are combinations of both types of plans.1 Although enrollment in the first few years was slower than hoped,2–4 by June 2002, it had reached 3.6 million nationally.5,6 Recently, there is also encouraging evidence that SCHIP is making a difference in levels of uninsured children. Cunningham,7 using data from the Community Tracking Study household survey, demonstrated that almost 20% fewer children were uninsured in 2000–2001 than in 1998–1999 nationally. In addition, Elixhauser et al8, using data from the Medical Expenditure Panel Survey, demonstrated a decline in the percentage of children who were uninsured all year from 10.3% in 1996 to 7.8% in 1999.
Since the enactment of SCHIP, most of the published literature regarding the program has focused on issues of enrollment, eligibility, and retention.2–5,9–17 Because SCHIP is a relatively new program, little has been published regarding the effect of SCHIP on important processes of care measures and health outcomes. The major objectives of the present study were to compare reported access to care, utilization of health care, and overall quality of care 1 year before and during the first year after enrollment into Colorado's SCHIP, the Child Health Plan Plus (CHP+). Specifically, we addressed the following questions: Did enrollment in CHP+ increase the percentage of children with a medical home and access to preventive, acute, and subspecialty health services? Did enrollment result in higher utilization of medical services after enrollment than preceding enrollment? Did enrollment result in higher overall quality of care services from the perspective of the enrolling family?
Description of CHP+
Colorado's SCHIP, CHP+, is a stand-alone program, although it shares a common enrollment application with the state's Medicaid program. The program provides medical benefits, including inpatient and outpatient services, prescription drugs, dental care, and mental health care. Services are provided either by a health maintenance organization or by a fee-for-service network, depending on the geographic area in which an enrollee lives. Families are eligible when they earn <185% of the federal poverty level (FPL). Enrolling families pay an initial enrollment fee of $25 for 1 child or $35 for multiple children when their income is >150% of the FPL as well as copayments when their income is >100% of the FPL.
Design and Study Population
We conducted 2 telephone surveys, 1 year apart, for a cohort of families who had recently enrolled a child in CHP+ for the first time. We randomly selected families who had enrolled 2 months before and did an initial survey during September 1999 through January 2000 (N = 711). We surveyed 2 months after enrollment, because >90% of newly enrolling families have been notified of their acceptance to the program and assigned to a provider by this time. A follow-up survey was conducted 1 year later, during November 2000 through February 2001, for the same cohort. Families with both initial enrollment and 1-year follow-up surveys (N = 480) composed the study population. The study protocol was approved by the Colorado Multiple Institutional Review Board.
The survey instrument and method have been previously described in more detail.9 During both surveys, families were asked to report only on the year preceding the interview, corresponding to the year before and the year after enrollment into CHP+. The surveys incorporated standardized questions with minor modifications from the National Health Interview Survey Household survey, the Prototype Children's Health Insurance and Health Care Questionnaire from the State and Local Area Integrated Telephone Surveys of the National Center for Health Statistics, and the Consumer Assessment of Health Plans Child Core that have been previously used by the study team.2,9 The interviews were conducted in English or Spanish, depending on the preference of the interviewee.
Interviews were conducted by Survey Units at the Colorado Department of Public Health and Environment and the AMC Cancer Research Center in Denver, Colorado. Families were called up to 15 times at different calling periods to optimize response rates, and both home and work numbers listed with the program were used. The interview was programmed for Computer Assisted Telephone Interviewing and skip patterns and acceptable range of responses, and consistency checks were programmed into the instrument. A minimum of 10% of all interviews were monitored by supervisors who randomly listened to calls and, using Local Area Network Assist Plus software, monitored interviewers' computer screens at the monitoring station.
Definition of Measures
Measures of access to care included whether there was a usual source of care or an identified primary provider; ease in seeing provider for routine, acute, or subspecialty care (on a 4-point Likert scale); and whether there were unmet prescription, mental health, vision, dental, routine, acute, or subspecialty health care needs. Utilization of care measures included the quantity of routine, acute office, subspecialty or emergency department visits, and hospitalizations. Quality of care was determined by having parents rate the quality of health care received (10-point Likert scale with 0 = worst and 10 = best) and by assessing whether any preventive care had occurred in the previous year. Type of previous insurance was categorized as private, Medicaid, none, or other, and length of time uninsured before enrollment in CHP+ was grouped into 3 levels: no gap in insurance, uninsured <1 year, and uninsured 1 year or more. Race/ethnicity was determined by self-report and was categorized as white, black, Hispanic, or other.
For bivariate analyses, binary categorical data were analyzed using McNemar's test for paired data, and continuous data were analyzed using paired t tests. In comparing access to care for subgroups who perceived a need for preventive, acute illness or injury, or subspecialty care, paired data could be statistically compared only for those who needed these types of care in both years. For multivariate analyses, binary categorical data were analyzed using logistic regression, and continuous outcomes (number of visits) were analyzed using Poisson regression within a Generalized Linear Model. For the multivariate analysis, each child contributed 2 records to the data set, 1 for the year before CHP+ enrollment and 1 for the year during CHP+ enrollment. To account for repeated measures of the same children over time, a generalized linear models analysis was used (SAS PROC GENMOD). We did multivariate modeling using 2 methods. We first ran the model including all of the explanatory variables and, subsequently, using backward elimination, including only explanatory variables that reached a significance level of .25 or less.18 Results are reported for variables with significance levels of <.05.
The dependent variables in our analyses were measures of access to care, utilization of care, and quality of care. The explanatory variables that initially were included in the models were period of measurement (pre-CHP+ enrollment vs post-CHP+ enrollment), type of previous insurance, length of time uninsured before enrollment in CHP+, and reported race/ethnicity. In addition, because age is known to be a significant predictor of utilization, we included age in years as a continuous variable measured at time of enrollment and age2 to account for a possible quadratic contribution of age. The results presented therefore are adjusted for age. We also assessed interaction terms including the pre/post time period by previous insurance and by length of time uninsured. SAS Version 8.2 (SAS Institute, Cary, NC) was used in all analyses.
Of the initial 920 newly enrolling families who were randomly selected from throughout the state, 711 (77%) completed the pre-CHP+ survey.9 Of this cohort, we were able to reinterview 68% (n = 480) 1 year later, whereas 4% (n = 25) refused reinterview, 18% (n = 128) did not have a current telephone number, and 11% (n = 78) were unreachable with 15 or more attempts. As previously reported,17 families whom we were able to resurvey did not differ significantly from those whom we did not resurvey with regard to gender, race, FPL, location of residence, or presence of chronic medical conditions or learning or behavioral difficulty in the enrolling child.
Access to Health Care
As demonstrated in Table 1, the percentages of families who reported a usual site of preventive care or identified a primary care provider did not change for the year before and the first year after CHP+ enrollment. However, the percentage of families who reported that it was very easy or easy to get all of the health care for their child increased from 53.9% to 73.1%, a relative increase of 36%. In addition, the percentages of families who reported “usually” or “always” seeing a provider as soon as desired for routine care, for care when sick or injured, or for subspecialty care increased significantly from preenrollment to postenrollment. The absolute increases shown correspond to relative increases of 13% for routine care, 17% for sick care, and 67% for specialty care. As Fig. 1 demonstrates, there were also dramatic decreases in the percentages of families who reported being unable to obtain needed services because they could not afford them. This was particularly striking for prescription drugs, with a relative decrease of 55%, and for eyeglasses, with a relative decrease of 48%, between the preenrollment and the postenrollment years. There was also a significant decrease in reported unmet dental care, despite the fact that CHP+, during the time of this study, did not provide dental benefits. Overall, 1 year after enrollment into the program, 44% of families (n = 210) still reported some unmet health need, but only 19% of the unmet need was for services other than dental.
Utilization of Health Care
As shown in Table 2, our data demonstrated significant increases in the percentage of families who received any routine care or specialty care in the year after versus the year before CHP+ enrollment. However, the mean number of visits for any type of care, emergency department care, or hospitalizations did not change significantly during the year before versus the year after enrollment. When routine care was stratified into age categories (<3 years of age [n = 43] vs older [n = 437] 1 year after enrollment), the difference in the percentage seen for any routine care was significant only for children 3 years of age or older (64.2% before vs 70.5% after; P = .04) and not for children who were younger than 3 years (88.9% before vs 94.4% after; P = .41). However, as the effect sizes were similar in the 2 groups, the lack of statistical difference in the younger subgroup may have been related to the small number of subjects in this sample. There was a trend for children with chronic illness to have more of an increase in the proportion who received any routine care (71.8% before vs 83.3% after; P = .07) compared with children without a chronic condition (65.0% before vs 70.1% after; P = .11); however, the small size of the subgroup of children with chronic illness in our sample (n = 88) did not allow us to adequately assess this relationship.
Quality of Health Care
In response to the question, “How would you rate all of your child's health care in the previous 12 months?” 34.9% reported a best rating before enrollment versus 41.6% after enrollment (P = .02). The means for the pre- and postenrollment periods were 8.3 (SD 1.9) and 8.7 (SD 1.5), respectively (P < .0001).
Results of multivariate analyses predicting change in post- versus preenrollment responses and controlling simultaneously for child's age and race, the length of time uninsured before CHP+ enrollment, and the type of previous insurance are shown in Table 3. The interaction terms assessed were not found to be meaningful and, therefore, are not included. Multivariate results were similar when all variables were included or when a backward elimination method was used. Because we were interested in reporting incidence ratios for other significant predictors of the effect of CHP+ enrollment, Table 3 shows the results of the backward elimination analyses.
The multivariate analyses confirmed all findings in the bivariate analyses in the areas of access, unmet medical needs, utilization, and quality of care with the exception of changes in specialty care visits that approached but did not reach significance in the multivariate model. Table 3 also demonstrates other significant predictors of the relevant outcomes in addition to CHP+ enrollment, most notably the deleterious effects of length of time uninsured before enrollment on access and utilization, and black or Hispanic race on having a primary care provider and access to subspecialty and routine health care.
When we ran the same multivariate model including only children who, before enrollment, had no medical home or no primary provider (n = 155), more significant changes in utilization data associated with enrollment were evident. Comparing postenrollment and preenrollment, the incidence ratio for seeing a provider for routine care in the past year increased to 2.53 (95% confidence interval: 1.56–4.10) and the incidence ratio for number of visits as a result of illness or injury increased to 1.48 (95% confidence interval: 1.18–1.87).
The current study demonstrates that families who were newly enrolled in Colorado's CHP+ perceived dramatic increases in access to all types of needed health care after enrollment, especially routine, acute illness, and specialty care. Our results show corresponding increases in the percentage of all enrolled children who received preventive health care services and higher utilization of office-based acute visits in the subgroup that did not have a medical home or provider before enrollment. Utilization of emergency department or hospitalization services did not change for the subgroup or for the total sample. In addition, families reported improved overall quality of health care received after enrolling in CHP+.
Since SCHIP's enactment in 1997, little has been published in the medical literature evaluating the effect of the program on health status, access, utilization, and quality of care. Although several studies assessed the effects of state child health plans that predated SCHIP in Colorado,19 New York,20,21 Massachusetts,22 and Florida,23,24 few data directly compare outcome measures before and after enrollment in SCHIP programs.25–28 A study of the North Carolina SCHIP, a stand-alone, fee-for-service program, demonstrated increases in the percentage of children with an identified private physician or clinic and decreases in unmet medical needs for prescription medications and eyeglasses after SCHIP enrollment.27 Another study that compared health status and unmet health needs for children before and after enrollment in Kansas's SCHIP also demonstrated significant decreases in unmet need after SCHIP enrollment, as well as modest increases in the proportion of children whose health was rated as very good to excellent by their parents.25 More recently, a substantially larger and more comprehensive study of New York's SCHIP was published, using similar methods to the present study but also including a comparison group enrolled at the time of the 1-year follow-up interviews to control for secular trends.26 This study demonstrated increases in the percentage of children with a usual source of care; decreases in unmet health care needs for preventive, acute, subspecialty, dental, and vision care; and increases in the proportion of children with preventive care and in continuity of care.
The results of the current study add to the growing evidence that SCHIP has been successful in improving health care access and delivery for enrolled children. Eligible families in Colorado perceived significant barriers to access before enrollment, and CHP+ enrollment was associated with increases in access to all types of health care. The pre- versus postenrollment decreases in unmet needs for prescription drugs, vision care, routine care, and acute care were similar in magnitude in our study to the study of Szilagyi et al,26 and, as in New York, we also demonstrated significant decreases in unmet need in all categories of health care. The reductions by 50% in unmet needs for eyeglasses and for prescription drugs in the current study are particularly notable, as such changes might translate into substantial educational benefits and better management of childhood illnesses for enrolled children. Residual unmet need 1 year after enrollment overall was twice as high in Colorado (44%) than in New York (19%); however, more than half of the residual need in Colorado was related to dental needs, a service not covered by CHP+. When dental services were not included, residual unmet health needs in Colorado and New York 1 year after SCHIP enrollment were very similar. It is interesting that families in Colorado perceived a decrease in barriers to dental care after enrollment, despite that CHP+, during the time of this study, did not provide dental benefits. Although we could not find another report of this in the literature, we speculate that this perception may indicate that, as a result of coverage by CHP+, families had more of their own financial assets to put toward dental care.
Although access was reportedly much improved, the mean rates of visits of all kinds did not increase significantly after enrollment in the total cohort, although the percentage who received any preventive care increased and the number of acute illness visits increased in children who did not have a medical home at the time of enrollment. These data are also consistent with the recent data from New York's evaluation that showed increases only in preventive services, with stable utilization for emergency, acute, and subspecialty care.26 In contrast, the evaluation of Kansas's SCHIP demonstrated increases in office visits of all types.25 However, in the Kansas study, families who did not reenroll in SCHIP for a second year were excluded, perhaps resulting in retention in the sample of families who used more services. In addition, possible confounders, such as age, were not controlled for in the analyses.
The reasons for the apparent mismatch between data showing, on the one hand, uniform increases in perceived access and decreases in unmet need and, on the other hand, very modest changes in utilization are not entirely clear. Our data show that pent-up demand for routine and acute care visits was concentrated in the subgroup of children who had not had a medical home or provider at the time of CHP+ enrollment. Because this group was relatively small, their increased utilization may not have been reflected in the overall analyses. In fact, a previous study in our state demonstrated that families who enrolled in the first few years of the program, “early enrollers,” were more likely to have a primary provider and less likely to have been uninsured before enrollment than eligible uninsured children who did not enroll.9 It is possible that characteristics of families enrolled at the time of the current study may have motivated them to seek health care, despite difficulties, before enrollment in CHP+, thereby blunting differences in utilization associated with enrollment. An alternative explanation, proposed by Szilagyi et al,26 is that SCHIP enrollment may result in more effective coordination of care and more efficient health care delivery as a result of increasing the proportion of care delivered at the usual site of care. In the current study, we did not collect data pertaining to the site of care for all visits and, therefore, are unable to test this plausible theory.
Our study relies on self-reported measures of access and utilization, which are subject to recall bias. However, we did compare 2 sets of data collected in the same manner and requesting recall for the same period 1 year apart. Because our conclusions are based on comparisons of these reported values rather than on absolute reported values, problems with recall bias probably do not have a large impact on our results. Findings regarding satisfaction with quality of care in the previous year may be more subject to reporting bias than other measures, because families are aware that they are being asked to assess the program. In addition, small sample sizes of children who were younger than 3 years and of children with chronic illnesses limited our ability to assess fully the impact of these subgroups on observed utilization patterns. Finally, this was an observational before–after study, and it is not possible, given this study design, to control for secular trends and demonstrate definitively that reported differences were attributable solely to enrollment in CHP+.
The results of this study demonstrate that families who enrolled for the first time in SCHIP in Colorado perceived dramatic increases in access to health care, decreases in financial barriers to care, and improvements in their overall quality of care. These gains were made without apparent increases in the more costly sites of health care, such as emergency department use or hospitalizations. Unfortunately, in the current climate of economic uncertainty and state fiscal downsizing, the success of CHP+ and other SCHIPs in increasing access to care may be unraveling. In November 2003, 6 states, including Colorado, implemented enrollment freezes in their SCHIPs.29 A survey of state SCHIP officials and child health advocates in these states suggests that these freezes are already resulting in tens of thousands of eligible children being turned away and are compounding inequities in insurance by income. In addition to the immediate effects of such freezes on new enrollment, already enrolled children whose families fail to comply promptly with renewal procedures will lose enrollment in these states until enrollment freezes are lifted. In the face of difficult economic times, when the need for insurance in families with low-income children is particularly high, the repercussions of enrollment freezes are likely to be dramatic. It is ironic that, at a time when SCHIP has sufficiently matured to allow us to demonstrate success in decreasing levels of uninsurance, increasing access, and improving quality of health care, the political and economic climate is such that these successes may go unheralded and may, indeed, be rapidly reversed.
This work was supported by grants from the Rose Community Foundation and the Colorado Child Health Foundation.
We are grateful to the Colorado Department of Health Care Policy and Financing for allowing us access to CHP+ data and for their input into the research. We also acknowledge Annie Wohlgenant, Rose Community Foundation, for help in disseminating our findings locally to those involved in influencing health policy and Barbara Stucky for help in preparing this manuscript.
- Accepted July 22, 2004.
- Reprint requests to (A.K.) Children's Hospital, 1056 E 19th Ave, B032, Denver, CO 80218. E-mail:
The views in this article are those of the authors and do not necessarily represent the views or policies of the Colorado Department of Health Care Policy and Financing.
No conflict of interest declared.
- ↵Centers for Medicare and Medicaid Services. Number of Children Ever Enrolled in SCHIP by Program Type. Available at: www.cms.hhs.gov/schip/enrollment/schip03.pdf. Accessed July 1, 2004
- Lutzky AW, Hill I. Has the Jury Reached a Verdict? Early Experiences With Crowd out Under SCHIP. Washington, DC: The Urban Institute; 2001 [Occasional Paper No. 47:1-26]
- ↵Trafton S, Shone LP, Zwanziger J, et al. Evolution of a children's health insurance program: lessons from New York State's Child Health Plus. Pediatrics.2000;105 :692– 696
- ↵Ryan JM. SCHIP turns five: taking stock, moving ahead. NHPF Iss Brief.2002;(781) :1– 12
- ↵Smith VK, Rousseau DM. SCHIP Program Enrollment: June 2002 Update. Washington, DC: Kaiser Commission on Medicaid and the Uninsured; 2002
- ↵Cunningham PJ. SCHIP making progress: increased take-up contributes to coverage gains. Health Aff.2003;22 :163– 172
- ↵Kempe A, Renfrew B, Barrow J, Cherry D, Levinson A, Steiner JF. The first two years of a state child health insurance plan—whom are we reaching? Pediatrics.2003;111 :735– 740
- Berman S. Mandated child health insurance: an approach whose time has come? Pediatrics.2003;111 :893– 895
- Ku L, Ross DC. Staying Covered: The Importance of Retaining Health Insurance for Low-Income Families. New York, NY: The Commonwealth Fund; 2002 [Publication No. 586]
- Kenney G, Haley J. Why Aren't More Uninsured Children Enrolled in Medicaid or SCHIP? New Federalism National Survey of America's Families.Series B, No. B-35. Washington, DC: The Urban Institute; 2001
- VanLandeghem K, Brach C. SCHIP Disenrollment and State Policies, Issue Brief No. 1. Washington, DC: Child Health Insurance Research Initiative; 2002
- Dick AW, Allison RA, Haber SG, Brach C, Shenkman E. The consequences of states' policies for SCHIP disenrollment. Health Care Financ Rev.2002;23 :67– 69
- Kalkines, Arky, Zall & Bernstein LLP. Coverage Gaps: The Problem of Enrollee Churning in Medicaid Managed Care and Child Health Plus. New York, NY: Kalkines, Arky, Zall & Bernstein LLP; 2000
- Riley T, Pernice C, Kannel S, Perry M. Technical Report: Why Eligible Children Lose or Leave SCHIP. Portland, ME: National Academy for State Health Policy; 2002:6
- ↵Kempe A, Beaty BL, Barrow J, Stokstad J, Belman S, Crane LA, et al. Disenrollment from a State Child Health Insurance Plan: are families jumping S(c)HIP? Ambul Pediatr.2004;4 :44– 51
- ↵Hosmer DW, Lemeshow S. Model-building strategies and methods for logistic regression. In Applied Logistic Regression. 2nd ed. New York, NY: John Wiley and Sons; 2000:95
- ↵Kempe A, Beaty B, Englund BP, Roark RJ, Hester N, Steiner JF. Quality of care and use of the medical home in a state-funded capitated primary care plan for low-income children. Pediatrics.2000;105 :1020– 1028
- ↵Keane CR, Lave JR, Ricci EM, LaVallee CP. The impact of a children's health insurance program by age. Pediatrics.1999;104 :1051– 1058
- ↵Szilagyi PG, Zwanziger J, Rodewald LE, et al. Evaluation of a State Health Insurance Program for low-income children: implications for State Child Health Insurance Programs. Pediatrics.2000;105 :363– 371
- ↵Feinberg E, Swartz K, Zaslavsky A, Gardner J, Walker DK. Family income and the impact of a children's health insurance program on reported need for health services and unmet health need. Pediatrics.2002;109(2) . Available at: www.pediatrics.org/cgi/content/full/109/2/e29
- ↵Shenkman E, Pendergast J, Wegener DH, et al. Children's health care use in the Healthy Kids Program. Pediatrics.1997;100 :947– 953
- ↵Szilagyi PG, Dick WE, Klein JD, Shone LP, Zwanziger J, McInerny T. Improved access and quality of care after enrollment in the New York State Children's Health Insurance Program (SCHIP). Pediatrics.2004;113(5) . Available at: www.pediatrics.org/cgi/content/full/113/5/e395
- ↵Eisert S, Gabow P. Effect of child health insurance plan enrollment on the utilization of health care services by children using a public safety net system. Pediatrics.2002;110 :940– 945
- ↵Ross DC, Cox L. Enrollment Freezes in Six State Children's Health Insurance Programs Withhold Coverage from Eligible Children. Washington, DC: The Kaiser Commission on Medicaid and the Uninsured; 2003 [Publication No. 4159]
- Copyright © 2005 by the American Academy of Pediatrics