Back when I worked at New Schools for New Orleans, we applied for a $30m federal grant to turnaround failing schools in New Orleans and scale the model to Tennessee.
CREDO just came out with a research study on our efforts. Their findings, and my analysis, are below.
The New Schools Were Much Better than the Ones They Replaced
Here’s what CREDO found when they compared the schools we created to the schools we replaced:
In New Orleans, we replaced schools (“closing schools”) that were at 26th percentile in the state with new schools (“CRM schools”) that performed at the ~33rd percentile in the state at the end of the study.
In Tennessee, schools went from the ~17th percentile to the ~23rd percentile by the end of the study.
To quote the CREDO report: “the CRM schools in both New Orleans and Tennessee showed significantly higher academic growth compared to the Closing schools they replaced.”
Translated into days of learning, these are large effects: “Closing school students experience 63 fewer days of learning in reading and 86 fewer days of learning in math when compared to students in non-CRM schools… students in CRM schools make comparable academic growth to non-CRM students.”
The New Schools Performed About the Same as Other Schools in the City
When CREDO compared the new schools to other existing schools (rather than the failing schools they replaced), they found no statistically significant effects:
In other words, the new schools that replaced the failing schools performed no better or worse than other existing schools in the city.
On one hand, this is disappointing. Our most ambitious targets included having the new schools be amongst the highest performing schools in the city.
On the other hand, this is still a major improvement: the new schools replaced failing schools and ended up achieving at the same level of most other schools in the city.
Building a System that Keeps Getting Better
Replacing failing schools with new schools is a process, not a one-time intervention.
Ideally, a subset of the schools you created will do really well, and then, overtime, these schools will continue to grow. The ones that don’t do well will not be supported to do additional turnarounds.
Over the long-haul, gradually increasing the number and scale of high-quality school operators is more important than the average effect of the first wave of replacements.
Here’s what CREDO found across the new schools when they compared them to existing schools:
In New Orleans, 50% of the new schools had positive effects in both Math and Reading. This is really positive: half of our turnaround schools in New Orleans achieved significantly better results than existing schools across the city.
In Tennessee, only one school had positive effects in both Math and Reading, though a few other schools had positive effects in only reading.
This makes me optimistic that the school operator base in New Orleans will continue to have the capacity to replace more failing schools over time.
The early results in Tennessee are a bit more worrying on the operator quality front, and the next few years will be extremely important in ensuring that a healthy operator base emerges.
Lastly: CREDO found that replacing failing schools with fresh start schools (that opened one grade at a time) had a higher success rate than whole school turnarounds. My takeaway here is that you need a mature operator base to do a lot of whole school turnarounds, and no city had enough capacity to really do whole school at scale. In hindsight, we should have done more fresh starts and less whole school turnarounds.
Was the Effort a Success?
At the outset of the project, I remember debating with our research partners at CREDO about how to set-up the evaluation.
I argued that we should ultimately be judged on whether or not the new schools we created were better than the failing schools we replaced.
I didn’t think we should be primarily judged on whether or not the new schools were better than other existing schools that weren’t failing.
Yes, we did include language in the grant application that had goals of schools performing much better than existing schools. And as we executed the project we tried to pick school operators that we thought could deliver top tier results. Our highest aspirations weren’t met. This is disappointing, but it does not mean the project was a failure.
Rather, I consider the project to be a positive step forward in improving public education in these cities.
Making Things Better
The result of the project strikes at the heart of what’s so difficult about education reform: our aspirations for our most at-risk children are incredibly high, but making progress in creating better educational opportunities is very difficult.
In roughly a five year period, we replaced failing schools with new schools that were on average 7 percentile points higher in state performance, which translates to an extra 60-90 days of learning per year.
If the process of opening and replacement continues, what is a modest success right now may eventually become a great success.
I hope that this occurs and that New Orleans continues on its impressive track record of increasing student achievement. As a reminder, the federal grant was just one piece of an overall effort that has radically reduced failing schools in the city:
SaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSave