Harvard researchers just published a study on Newark.
In large part due to Mark Zuckerberg’s $100 million gift to the reform efforts, the city has seen a lot of coverage, including the well done book The Prize.
Most of this coverage has been negative.
The Harvard Study: Teasing Out Annual Effects From 2010-2016
The Harvard study found that the Newark reforms, in the most recent year of the study, had a positive impact on ELA (.07 SD) and no impact on math. In the early years of the reforms, the achievement effects were negative.
Unfortunately, the researchers only looked at annual effects (were there positive or negative effects in any given year) rather than calculating cumulative impacts (what was the total impact over these years). Because of this, we can’t perfectly compare the Newark results with results of other city based studies, such as Doug Harris’ evaluations of the New Orleans efforts.
What Caused the Positive Effects?
While the authors didn’t calculate the cumulative effects of the reforms, they did do something wonderful.
The authors separated out the effects of two different strategies: (1) improving existing schools vs. (2) expanding high-performing schools, closing low-performing schools, and facilitating the transfer of students out of low-performing schools and into high-performing schools.
The improving existing schools approach included replacing large numbers of principals, renegotiating the union contract, implementing new data systems, and extending learning time.
The open / close / shift enrollment approach included adding to the enrollment of high-performing charter and district schools, closing underperforming schools (11 traditional schools and 3 charters), and implementing a unified enrollment system that made choosing schools much easier for families. These reforms increased charter enrollment from 14% to 28%.
The results are striking.
In math, the improve strategy achieved a .08 negative effect in math while the open / close / shift strategy had a positive .04 effect.
In ELA both strategies had positive effects, but the open / close / shift was responsible for 62% of the overall positive effect.
Perhaps most importantly: the open / close / shift strategy achieved positive effects in every year of the study.
Opening and closing schools, and shifting student enrollment, increased student learning.
Could Newark have Achieved New Orleans’ Level Gains?
In roughly the same number of years as the Newark reforms, New Orleans achieved positive .2-.4 standard deviation effects.
New Orleans, of course, relied much more on the open / close / shift than it did on improvement.
As noted above, we can’t do a direct apples-to-apples comparison of the studies. But I’d be surprised if Newark’s cumulative effect is as large as the New Orleans effect given that Newark saw negative effects in 3 of the 5 years of reforms.
We’ll never know if Newark could have achieved similar gains to New Orleans, but in many ways the Newark circumstances were more enviable than those of New Orleans.
Newark is home to one of the highest-performing charter sectors in the country, while New Orleans had no large effective charter operators in the city right after Katrina.
Newark’s initial philanthropic investment was much larger than that received in New Orleans after Katrina. And the public per-pupil in Newark is about 3x that of New Orleans.
Newark students also didn’t have to go through the immense trauma of Katrina.
That being said: perhaps New Orleans style reforms would have not worked in Newark.
Maybe the politics would not have allowed for it… though it’s worth remembering that Shavar Jeffries, now the CEO of Democrats for Education Reform, was only about 2,000 votes away from beating Ras Baraka in the 2014 Newark mayoral race.
Or perhaps the charter sector would not have been able to grow much faster than it did; notably, charter effects did decline as they enrolled more students… though perhaps more charter capacity would have emerged under better conditions.
All told, my guess is that Newark did not increase educational opportunity as much as it could have.
A 2010 Prediction
On this blog, I try to be honest about both my mistakes and successes.
It would be disingenuous for me to write a post arguing that, in 2010, no one could have predicted which strategy would have been better.
The fact is that I did predict this. In 2010, when it became clear that there was going to be major reforms in Newark, I created a powerpoint deck to give advice to national educational leaders. In the deck, I modeled out how quickly I thought you could expand great schools and close failing schools.
Given the relatively small size of Newark public schools (50,000 students), as well as the number of high-quality charter schools, I felt confident that the vast majority of Newark students could attend a high-quality school within ten years.
This ended up being the strategy we took in New Orleans.
It was not the strategy taken in Newark.
I have a lot of love for the philanthropists, political leaders, and educators who spent the good part of a decade trying to make things better for kids. I know many of them well and have had nothing but positive interactions.
In terms of rigorous third party research results, we also knew a lot less in 2010 than we do now. There have been a wave of recent studies documenting the positive effects of expanding the best schools and closing the worst.
And while we still suffer from a dearth of research on citywide turnaround efforts, outside of the early results in Lawrence, Massachusetts, we really haven’t seen a citywide turnaround reform effort significantly raise achievement without a focus on opening and closing schools (research has shown that Washington D.C. improved their public schools, but the city also leaned heavily on opening and closing schools).
So perhaps the better question is not what should have Newark leaders done in 2010, but what should similarly situated civic leaders do in 2017?
Civic leaders should embrace strategies that maximize the expansion of the best schools and close the worst schools.
Of course this need not be the only focus of a city turnaround effort, but the Newark study has increased my belief that it should be the primary focus.