Abstract
OBJECTIVES: We examined whether exposure to high-performing schools reduces the rates of risky health behaviors among low-income minority adolescents and whether this is due to better academic performance, peer influence, or other factors.
METHODS: By using a natural experimental study design, we used the random admissions lottery into high-performing public charter high schools in low-income Los Angeles neighborhoods to determine whether exposure to successful school environments leads to fewer risky (eg, alcohol, tobacco, drug use, unprotected sex) and very risky health behaviors (eg, binge drinking, substance use at school, risky sex, gang participation). We surveyed 521 ninth- through twelfth-grade students who were offered admission through a random lottery (intervention group) and 409 students who were not offered admission (control group) about their health behaviors and obtained their state-standardized test scores.
RESULTS: The intervention and control groups had similar demographic characteristics and eighth-grade test scores. Being offered admission to a high-performing school (intervention effect) led to improved math (P < .001) and English (P = .04) standard test scores, greater school retention (91% vs 76%; P < .001), and lower rates of engaging in ≥1 very risky behaviors (odds ratio = 0.73, P < .05) but no difference in risky behaviors, such as any recent use of alcohol, tobacco, or drugs. School retention and test scores explained 58.0% and 16.2% of the intervention effect on engagement in very risky behaviors, respectively.
CONCLUSIONS: Increasing performance of public schools in low-income communities may be a powerful mechanism to decrease very risky health behaviors among low-income adolescents and to decrease health disparities across the life span.
- API —
- Academic Performance Index
- CST —
- California Standards Tests
- OR —
- odds ratio
What’s Known on This Subject:
Graduating from high school is associated with better health and health behaviors. However, no rigorous studies have tested whether exposure to a high-performing school improves health or health behaviors, thus the causal relationship is unknown.
What This Study Adds:
Exposure to successful schools can reduce very risky health behaviors among low-income adolescents. The primary mechanism is mostly due to better school retention and also due to better academic achievement.
Many aspects of the school environment may influence adolescent behaviors and health outcomes.1,2 For example, school climate factors such as greater school connectedness and engagement, positive teacher relationships, and respect in school are associated with better academic outcomes, fewer behavioral problems, and better mental health.3–7 Other school-level factors, such as health and disciplinary policies, are associated with better health3,8 but are less well studied. Although these and other studies have been very helpful in understanding how school environments might be modified to improve health, almost all have relied on observational study designs. Thus, selection bias and other confounders, such as parental influence and neighborhood poverty, severely limit the ability of these studies to determine the causal relationship between school environment and adolescent outcomes. A few randomized trials or natural experiments have shown that lower barriers to education and some early childhood programs have economic and health benefits.9–11 Despite recent education reform efforts in the United States, whether better health can be achieved through public education improvements is unproven.
The first charter school in the United States opened in 1992, and since then the charter school movement has spread rapidly.12 These schools receive public funding and are designed to have more autonomy over curriculum and operations to foster innovation and better academic outcomes. Given this autonomy, their educational models can vary substantially, but admission is free and open to all. Although charters are not a panacea for improving education,12 the random admissions lottery of successful charter schools provides an opportunity to observe a natural experiment. The Reducing Health Inequities Through Social and Educational Change (RISE) Study surveyed applicants to 3 high-performing public charter high schools in low-income neighborhoods of Los Angeles to determine if exposure to these schools reduces risky behaviors.
Methods
Study Design and Sampling
In 2010 we identified all 20 public charter high schools run by charter management organizations in low-income neighborhoods of Los Angeles and open since 2007 or earlier. Among 7 of these with a California 2009 Growth Academic Performance Index (API)13,14 in the top tertile of public high schools, 4 had enough applicants to hold an admissions lottery, and 3 of these agreed to participate in our study.
Also in 2010 we sought to obtain a study sample of current ninth- through twelfth-grade students who had applied to a participating charter school for the ninth grade and entered the random admissions lottery. We randomly selected potential subjects from each school’s ninth grade applicant list from 2007 to 2010, with the goal of recruiting into our study equal numbers who were offered admission (intervention group) and who were not offered admission (control group). To minimize contamination of our sample to other high-performing schools, we excluded those who went to another charter or private school for ninth grade, but not those who transferred to another charter or private school after ninth grade. We excluded subjects who could not be contacted or had moved out of the area and siblings receiving admission outside the lottery.
After obtaining written informed parental consent, research assistants conducted a 90-minute face-to-face computer-assisted interview with each subject. An audio-enhanced, computer-assisted self-interview was used to collect information about substance use and sexual behaviors. With parental consent, we also obtained student-level math and English California Standards Tests (CST) scores from the California Department of Education for study participants since eighth grade. The human subjects research review board approved all research activities.
Measures
From surveys, we collected demographics, and self-reported school information (school transfer and dropout), substance use, sexual behaviors, and exposure to violence.15,16 We assessed depression,17 school engagement,18 parenting style,19 and hopelessness.20 To assess subjects’ personal social network, we asked them to name 20 persons whom they know and are most important to them. For each person in their network (alter), we asked subjects (ego) about their relationship (eg, relative, friend), and the alter’s characteristics (relative age, gender) and behaviors (eg, smoking, alcohol, drug use, sexual behaviors). We calculated the proportions of same-aged peers in a subject’s network who used substances or had sexual intercourse. CST scores were categorized into below basic (<300), basic (300–349), and proficient and above (≥350).21
Our primary outcome measures were risky behaviors, defined as any use of tobacco, alcohol, marijuana, and other drugs in the past 30 days. We also examined less common but very risky behaviors, which included binge drinking (≥5 drinks on 1 occasion), alcohol use at school, any drug use (excluding marijuana), carrying a weapon to school in the past 30 days, gang membership in the past 12 months, current pregnancy, multiple sexual partners, sex without condoms, sex without contraception, and alcohol or drug use with sex in the past 90 days. We compared those who engaged in none versus any of these very risky behaviors.
Analytic Methods
We conducted an intent-to-treat analysis comparing those offered admission to a study charter school (intervention group), regardless of whether they accepted the offer, with those not offered admission (control group). We used generalized estimating equations to account for nonindependence of students clustered within schools, adjusting for grade, gender, race/ethnicity, language, parental education, parental employment, parental birthplace, and parenting style.22 The main analyses were limited to 10th through 12th graders because we hypothesized that the “intervention effect” would be absent among ninth graders who had limited exposure to the high-performing schools. We performed a sensitivity analysis examining ninth graders only, but the small sample permitted controlling for gender only. We formally tested for mediation effect by using a method by Breen et al.23 This test does not take into account hierarchical models, so we performed this mediation test by using logistic regression. We imputed missing data by using multiple imputation based on methods developed by Rubin24 and Schafer.25 Stata version 11 was used for all analyses (StataCorp, College Station, TX).26
Results
Among all ninth-grade applicants to the 3 study charter schools between 2007 and 2010, we randomly selected 1432 applicants who were not offered admission (control group) and 952 who were offered admission (intervention group) (Fig 1). We tried to locate applicants to screen them for study eligibility and recruitment but could not find 253 (17.7%) of the potential control sample and 157 (16.5%) of the potential intervention sample using contact information from their admissions application. Of those remaining, 574 (48.7%) of the control and 162 (20.4%) of the intervention group were ineligible, mainly because they attended another charter school for ninth grade. Of remaining eligible subjects, 196 (32.4%) in the control group and 112 (17.7%) in the intervention group refused participation (P < .001). The final sample included 409 and 521 subjects in the control and intervention groups, respectively. Our original target sample was 950 subjects with 50% in each study arm. Because of the higher ineligibility and refusal rate in the control arm, we were unable to obtain equal numbers of subjects, and we had to select from a much larger sample of school applicants to obtain the final control group.
Subject recruitment, eligibility, and participation.
Student demographic characteristics, eighth grade math and English CST scores, and parental education, nativity, and employment were not statistically different between the 2 groups (Table 1). Some differences were noted that might indicate lower socioeconomic status and potentially higher risk for adverse education and health outcomes among the control group. Specifically, the control group was less likely to own a home as reported by the student (40.6% vs 45.0%; P = .02) and parents were slightly less likely to have graduated from high school (48.2% vs 52.6%; P = .15). However, the intervention group was less likely to include native English speakers (42.2% vs 35.8%; P = .07), which might increase their risk for worse academic outcomes. Whether a student had come from a high- or low-performing middle school might influence the impact of being exposed to a high-performing high school. We found the Growth API of the students' previous middle schools were similar between the 2 study arms, and the baseline characteristics of the 2 study arms were similar after stratifying on the performance of their middle school.
Baseline Characteristics of the Control and Intervention Groups
Table 2 shows the characteristics of the subjects’ school and school performance. There was some “contamination” between the control and intervention groups as measured by the Growth API of the actual school attended. Nine percent of the intervention group ended up in a school that performed worse than the study charter schools compared with 82.9% of the control group, and 17.1% of the control group ended up in a school that was equal to or better than the study charter schools compared with 90.8% of the intervention group. The intervention group was less likely to have transferred high schools or dropped out (8.8% vs 24.2%; P < .001) and to have “cut” school in the past 12 months (P < .001). Also, the intervention group was more likely to have attended a higher performing high school (average Growth API of 748 [68th percentile] vs 673 [45th percentile]; P < .001). Student-level CST scores from eighth grade or the first available score were similar in the 2 study arms (Table 1), but according to the most recent CST scores, the Intervention group was more likely to be proficient or above in English (P = .04) and math (P < .001).
School Characteristics and Performance Measures by Intervention and Control Group
Our main outcomes were engagement in risky and very risky behaviors. Because exposure to a high-performing high school would not likely have an immediate impact in ninth grade, we limited our analysis to 10th through 12th graders. Although we observed no statistically significant differences between the control and intervention groups in any of the risky behaviors (tobacco, alcohol, and marijuana use in the past 30 days), the effect of the intervention on cigarette smoking and alcohol was in the direction that we hypothesized and the confidence intervals were relatively wide (Table 3). We also examined 11 very risky behaviors: the most common were sex without contraception (13.4%), inconsistent condom use (11.6%), binge drinking (8.3%), alcohol or drug use with sex (7.1%), marijuana use at school (6.4%), and drug use other than marijuana (6.3%). Prevalence was <5% for each remaining very risky behaviors (alcohol use at school, current pregnancy, multiple sex partners, carrying a weapon to school, and gang membership). The proportion engaging in ≥1 of these 11 very risky behaviors was 41.9% in the control group and 36.3% in the intervention group, with an adjusted odds ratio (OR) of 0.73 (P = .048). The intervention effect on engaging in a very risky behavior was more pronounced among those who had attended a low-performing middle school (OR = 0.67, P = .032) than among those who had attended a high-performing middle school (OR = 0.85, P = .40). However, the interaction effect between being offered charter school admission and middle school performance was not statistically significant (P = .77).
Proportions and Adjusted ORs of Engaging in Risky Health Behaviors Comparing Control With Intervention Groups
We also conducted a sensitivity analysis examining ninth graders only and, as expected, found no statistically significant differences in tobacco, alcohol or marijuana use, or engagement in ≥1 very risky behaviors. Compared with those in the intervention group, ninth graders in the control group were somewhat more likely to smoke cigarettes (9.3% vs 5.3%; P = .22) and slightly less likely to drink alcohol (18.3% vs 21.3%; P = .56) and use marijuana (10.0% vs 11.5%; P = .72) in the past 30 days. Engagement in ≥1 very risky behaviors was almost identical between groups (16.7 vs 16.5%; P = .98).
We examined whether various potential mediators were associated with engaging in ≥1 very risky behaviors, and whether controlling for the potential mediator diminished the intervention effect. Very risky behaviors were more likely if the student was not retained in the same high school (ie, changed schools or dropped out), was less engaged in school, had below basic English and math CST scores on their most recent test, had a lower grade point average, had a greater proportion of peers who use substances or have had sex, was more depressed, or felt more hopeless about the future (Table 4). Having a teacher named in the student’s social network was the only potential mediator not associated with engaging in very risky behaviors. Only school retention and math and English CST scores were significant mediators (P < .001 and P = .06) and reduced the intervention effect on engagement in very risky behaviors after being individually added into the regression model. These 2 mediators explained 58% and 16.2% of the intervention effect, respectively. This analysis suggests that exposure to a high-performing school improves retention and standardized test scores, which then reduces very risky behaviors.
Effect of Potential Mediators on the Intervention Effect on Engaging in ≥1 Very Risky Behaviors
An alternative mechanism may be that high-performing schools reduce very risky behaviors, which then improve test scores and school retention, but this was not supported by additional analyses. We conducted a mediation analysis with retention as the outcome and very risky behaviors as the mediator and found that the intervention effect without any mediators was strong (OR = 5.2, P < .001), but very risky behaviors mediated only 4% of the intervention effect on retention (P = .08). We also performed a similar analysis with test scores as the outcome. The intervention effect was associated with a 27-point increase on standardized test scores (P < .001), but engagement in very risky behaviors mediated only 7% of the intervention effect on test scores (P = .08).
Discussion
A few experimental or quasi-experimental studies have found that early childhood schooling and more primary schooling have important economic and health benefits.9,10,27 However, whether improving the school environment causally results in fewer risky behaviors has not been fully explored with the use of more rigorous study designs. We used a natural experiment design and found that, among minority adolescents from low-income families, exposure to high-performing public charter high schools (intervention effect) led to better standardized test scores, fewer dropouts and school transfers, and lower rates of engagement in very risky behaviors, such as drinking at school, gang participation, or use of alcohol or drugs with sex. However, we found no statistically significant differences in less risky behaviors, such as any recent use of tobacco, alcohol, or marijuana. The use of tobacco and alcohol was slightly lower in the intervention group, but this finding did not reach statistical significance perhaps due to lack of statistical power.
A number of potential mechanisms might explain why successful school environments are associated with fewer risky health behaviors:
Better cognitive skills may lead to better health outcomes,28,29 perhaps through better health knowledge, medical decision making, or health literacy.
Factors that improve educational achievement may also reduce risky behaviors. For example, attending a high-performing school may lower exposure to “risky” peers. Successful school environments may also improve persistence, resiliency, and other noncognitive skills, which may lead to better outcomes.29–32
Better academic achievement may lead to a better future outlook and less risk taking.33
Being in school or doing more homework may leave less time and opportunity to engage in risky behaviors.
We examined a number of factors related to these potential mechanisms and tested them for mediation effects. Although many were associated with very risky behaviors, only math and English CST scores and retention in the same school were significant mediators. This finding has important implications for future interventions. For example, school policy changes that reduce suspensions and expulsions might lower rates of adolescent risky behaviors.
Several study limitations are worth noting. Because the study was not a true randomized trial, ensuring comparability between the 2 study arms is difficult. The control sample was almost twice as likely to refuse participation, potentially introducing sampling bias. Although limited information is available for those who refused, we suspect that they would have more behavioral and academic problems, which would bias the results toward the null. We also excluded many potential intervention and control subjects because they had chosen to attend another high-performing charter or private school in ninth grade. Had we included them, a much greater proportion of the control group would have been “contaminated” by exposure to another high-performing school. As expected, exclusion for this reason was more common among the control group, which might have introduced additional bias. For example, applying to more schools might be associated with differences in risky behaviors, parenting, or academic motivation. Despite these challenges, among study participants, the 2 study arms were similar in demographic characteristics and early test score performance, suggesting that group assignment was random and equal. Still, some unobserved differences that could confound the results might exist due to imperfect randomization or sampling bias.
Additional limitations include the inability to generalize to adolescents who do not apply to charter schools or to other cities, populations, or school environments, including successful noncharter public or private schools. We relied on student self-reported behaviors, and those with better school outcomes might tend to provide more socially acceptable responses. We also did not observe intermediate- or long-term outcomes beyond high school. Finally, the intervention group reported lower tobacco and alcohol use that did not reach statistical significance, but our study may have been underpowered as suggested by the large confidence intervals. Of note, for our ex ante sample size calculation, we used previously reported behavior rates among high school dropouts and graduates for smoking, the least prevalent outcome in our study, and further inflated our sample because the estimated effect size was based on observational studies.15,34 However, the observed differences in our study were much smaller than anticipated. The sampling bias issues mentioned previously may have further hampered the study to achieve adequate power.
Conclusions
The current study encouragingly reveals that successful public charter high schools in low-income neighborhoods might have early beneficial health effects. Future studies will need to determine if the effects are long lasting or can be observed in other populations and school settings. The academic achievement gap in the United States appears to be growing between the wealthy and poor,35 which is concerning not only because of the economic implications but also because of the potential effects on population health and health disparities. The present findings highlight the importance of improving academic achievement and attainment, and in particular school retention, on adolescent health behaviors and suggest that health policy makers may need to pay greater attention to educational policies and trends.
Footnotes
- Accepted April 21, 2014.
- Address correspondence to Mitchell D. Wong, MD, PhD, UCLA Division of General Internal Medicine and Health Services Research, 911 Broxton Ave, Los Angeles, CA 90024. E-mail: mitchellwong{at}mednet.ucla.edu
Dr Wong conceptualized and designed the study, conducted all analyses, and drafted the initial manuscript; Dr Coller assisted with the study design, data collection, and data analysis and reviewed and revised the manuscript; Dr Dudovitz assisted with the conceptual framework for the study, survey content and design, and data analysis and reviewed and revised the manuscript; Dr Kennedy assisted with the study design, particularly in social network data collection and analysis, and reviewed and revised the manuscript; Dr Buddin assisted with the study design in sampling and collection and analysis of educational outcomes data and reviewed and revised the manuscript; Drs Shapiro, Kataoka, and Brown assisted with the study design and reviewed and revised the manuscript; Drs Tseng and Bergman assisted with data analysis, statistical methods, and interpretation and preparation of the data tables of the manuscript and reviewed and revised the manuscript; Dr Chung assisted with the study design, data collection, and data analysis and reviewed and provided critical revisions of the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Support for this study was provided by a grant from National Institutes of Health (NIH)/National Institute on Minority Health and Health Disparities (RC2MD004770) and by additional support from NIH/National Center for Advancing Translational Sciences (UL1TR000124). Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2014 by the American Academy of Pediatrics
In this issue
