Expert Commentary

The impact of high-achieving charter schools on non-test score outcomes

2013 study published by the National Bureau of Economic Research on how attendees of the Promise Academy in the Harlem Children’s Zone did later in life.

Are charter schools better for children? The answer depends on context. And it’s not an unequivocal “yes,” at least based on evidence from test scores: One of the most comprehensive randomized studies to date, published in 2010 by the National Center for Education Evaluation, found that charter schools were no more effective than public ones in raising math and reading outcomes.

Other concerns exist as well: Research from the University of Texas and Rice University found that the attrition rates for black students at some Texas charter schools were three times as high as in public schools. Although concerns that charter schools “push out” low-performing students may be unfounded, many charter schools fail to meet equity goals in the face of market competition (a 2013 analysis in the Oxford Review of Education provides information).

Yet research indicates that some charter schools can produce better results: A 2010 study by the National Bureau of Economic Research found that test scores were higher at Knowledge Is Power Program (KIPP) schools — which employ a “no excuses” approach that has also been tried in public establishments. Superior charter schools often feature an extended school day, a focus on discipline and a curriculum centered on reading and math skills, according to a 2011 MIT study. Still, research indicates that charter-school teachers exit the profession at relatively high rates, something that can have negative effects on students.

As the charter-school movement matures, it has become possible to assess the medium-term impact of specific schools on students. A 2013 study for the National Bureau of Economic Research, “The Medium-Term Impacts of High-Achieving Charter Schools on Non-Test Score Outcomes,” looks at attendees of the Promise Academy in the Harlem Children’s Zone — one of the nation’s most high-profile efforts to reform outcomes in urban schools. The scholars — Will Dobbie of Princeton University and Roland G. Fryer Jr. of Harvard —  examined students’ health, human capital and behaviors five to six years after admission.

To do so, the researchers exploited a natural experiment: The Promise Academy is routinely over-subscribed, so the children who attend are selected by lottery. All live in the same neighborhoods, and the only significant difference between them, on average, is whether or not they were randomly selected to attend the school. The study involves 11th and 12th grade students — 501 in total — who entered the Promise Academy lottery in their 6th grade in 2005 and 2006.

The study’s key findings are:

  • Students who entered the Promise Academy as a result of winning the lottery do 0.4 standard deviations better on math and 0.2 standard deviations better on reading than those who didn’t attend the school. (The tests used are Woodcock-Johnson tests, which assess a broader range of computational and language skills than standardized school tests.)
  • Attendees are more likely to take New York State Regent exams and when they do take them, they’re 31% more likely to pass, especially advanced subjects such as geography and chemistry.
  • Students are 49% more likely to enroll in college in the fall after their senior year. “Attending the Promise Academy increases the probability of enrolling in college by 24.2 percentage points, an 84% increase,” the researchers find. The students are also more likely to attend a four-year college than a two-year college.
  • Both pregnancy and incarceration rates are relatively lower: “Admitted females are 12.1 percentage points less likely to be pregnant in their teens, and males are 4.3 percentage points less likely to be incarcerated.”
  • There was little significant evidence that the Promise Academy affected drug or alcohol use, self-reported criminal behavior, or self-reported health outcomes and eating habits.
  • No evidence was found that non-cognitive skills such as self-esteem, self-control and persistence are better developed at the Promise Academy than at control schools.
  • Peer influence also did not appear to play a strong role: The researchers found no statistically significant differences in the attitudes towards educational success or risky behavior of the social networks of lottery winners compared to lottery losers.
  • One significant difference is Promise Academy students did appear to be significantly more risk-averse than the control group, suggesting that attending the Promise Academy may increase risk aversion and so increase students’ efforts to improve their educational prospects and reduce involvement in risky behaviors.

An important distinction to note is that the Promise Academy is in the Harlem Children’s Zone, which provides a wide range of children’s services that are not available at most other charter schools. The authors attempt to gain broader insight by looking at college enrollment data from four other charter schools around the United States. They find that students who won lotteries to attend these charter schools were also more likely to attend college, and that college was more likely to be four-year and high-standard, compared to children who didn’t attend such schools.

Further research: Search the federal ERIC database for the latest and most in-depth studies on charter schools.

Keywords: charter schools, youth, crime, prisons, poverty

About The Author