Category Archives: New Orleans

Why did New Orleans public schools improve so much?

Tulane researchers have a new paper that attempts to determine the causal mechanism for New Orleans school improvements.

A similar paper was written by Harvard researchers on the Newark reforms.

Both papers tried to answer the question: did things get better because schools opened and closed, or because existing schools improved?

Both papers come to the same conclusion: opening and closing schools is driving the gains in student learning (as measured by test scores).

The Tulane report came to a particularly strong conclusion. The authors write:

“The average school improved from the first to the second year after it opened, but school performance remained mostly flat afterwards… aside from the improvement when schools first opened, essentially all of the improvement in New Orleans’ average test scores has been due to the state regularly closing or taking over low-performing schools and opening new higher performing charters.”

The below graphic captures this finding in visual form:

Screen Shot 2019-08-29 at 12.45.18 PM.png

The authors end their study with a strategic recommendation and warning:

“The fact that newly opened schools continue to be better than those closed and taken over also suggests that the extreme measure of replacing school operators also still has some potential to generate further gains. At some point, the benefits from this strategy are likely to run out, but it does not appear that we have reached that limit yet.”

New Orleans has had strong government regulation over the past decade. For the most part, the best schools expanded and the government closed or transformed the worst schools.

It is an open question whether this good regulation can persist in New Orleans, or if it can be consistently scaled to other cities.

Of course, strong regulation is not the only way to shift enrollment to higher-performing schools.

A city could also simply let family choose amongst all schools and wait for lower-performing schools to fold under enrollment pressures. This process will be parent driven and likely slower.

Every city will need to figure out its own path when it comes to balancing top down accountability and bottoms up family choice.

Personally, I favor a combination of both. Let government have the ability to selectively transform the lowest performing schools in a city, and let families choose from a wide array of schools.

Bloom, New Orleans, and Effect Sizes

Six months ago Matthew Kraft published an excellent article on effect sizes.

I worked in education for five years before I had any understanding of research design and reporting. I wish Matt’s piece was around a decade ago.

His article is a bit dense if you’re trying to just wrap your head around the issue, so consider this post a lay person’s intro to Matt’s piece and the subject itself.

If you catch any mistakes, please do let me know. I’m still learning.

Why are effect sizes useful?

Consider currencies. Currencies are useful because they allow you to easily compare prices across various goods. Instead of having to constantly refer to one set of goods in relation to another set (ie, three apples are worth the same as four oranges which is worth the same as three paperclips), we can use the same unit (dollars) to compare a bunch of different goods.

Effect sizes serve the same function. They help us easily compare the magnitude of the impact of a bunch different interventions. We can do research on graduation rates, test scores, suspension rates, or whatever we want, and then we can convert our results into an effect size to help us compare how big of an impact we had.

Effect sizes are the unit of currency for measuring impact.

What is an effect size?

Many effect size calculations in education research are expressed in standard deviations.

A common formula to determine the effect size is:

(mean of experimental group – mean of control group) / standard deviation

Let’s say we trying to find the effect size of a new math curriculum on test scores. We might give half the population the new curriculum, half the old curriculum, and then see what the difference is.

Let’s say the difference is +5 pts out of a 100 for the students using the new curriculum. The curriculum “worked.”

But what does that mean?

We now want to know if +5 pts is a big deal. This is where the standard deviation comes in.

A low standard deviation means there is very little difference in the population (everyone is scoring about the same score). A large standard deviation means there is a wide spread in scores.

Because the standard deviation is the denominator in the formula, the smaller it is, the large the effect will be for any given difference between two groups.

In other words, if everyone is scoring between 62 and 65 out of a hundred, and you jump five points, you could go from the bottom 1% of test takers to the top 1% of test takers.

Because the standard deviation is low (small spread), a modest jump leads to a big effect.

What is a large effect?

This is where Matt’s paper is particularly useful.

Much of the previous literature on effect sizes made many mistakes:

  1. Sample sizes were ignored.
  2. Duration of treatment were ignored.
  3. Time elapsed until measurement was ignored.
  4. Cost was ignored.

Taken together, scalability of interventions was ignored. This had the unintended consequence of setting the bar too high for what should be considered a large effect size.

Bloom’s 2 standard deviation effect 

You may have heard of Bloom’s 2 sigma tutoring intervention. This result is taken to show that 1-1 tutoring can have a 2 standard deviation (very large!) effect.

But Bloom’s study design was the following: take dozens of 4th, 5th, and 8th graders; give them 1-1 tutoring in discrete subjects like cartography or probability; and then test them on what they learned after 3-4 weeks!

It’s much easier to squeeze out a big effect under these conditions.

These types of small sample studies led to a research norm where an effect size had to be .8 standard deviations for it to be considered large.

New Orleans’ .4 standard deviation effect 

Contrast Bloom’s study to Doug Harris’ study on the New Orleans education reforms.

The New Orleans study covered tens of thousands of students. Students received the treatment across all major subjects, including math, reading, science, and social studies. The treatment lasted multiple years. And students were tested once every year in each subject.

It’s a lot harder to make large gains under these conditions, especially when the intervention costs under 20% of 1-1 tutoring.

Doug’s study found .4 standard deviation effects for New Orleans students over a five year period.

In his paper he wrote that he was “not aware of any other districts that have made such large improvements in such a short time.”

To summarize:

  1. The standard bar for a large effect was .8 standard deviations. This was irregardless of sample size, length of treatment, measurement proximity, or cost. The bar was poorly constructed.
  2. New Orleans achieved a +.4 standard deviation effect on test scores.
  3. Researchers had never seen a citywide effect this large before.

There are two ways to interpret this.

  1. The previous .8 standard deviation bar was way too high for large samples.
  2. The New Orleans effect, despite being relatively large for district improvement, is still so absolutely small that we should not be too impressed.

Was the New Orleans effect too small?

The +.4 standard deviation effect equates to the average New Orleans student moving from the 22nd to 37th percentile in performance.

For any individual, this might or not be life changing. But in the aggregate this means the average New Orleans student roughly went from a borderline high school dropout (bottom 20% of performance) to a student who has a real chance to enter a two year or four year college (modestly below average performance).

Across a large population, this is a pretty big deal.

We should pay attention to a city level +.4 standard deviation increase in test score. If this effect (or even one somewhat lower) can be scaled, kids across the country will have a better chance at leading a good life.

Of course, academics and test scores are just one piece of the puzzle of economic mobility, but they are an important piece. Schools with negative effects on test scores tend not to deliver great long-term life outcomes for kids.

Matt Kraft’s proposed effect size scale

When it comes to large interventions, Matt argues we should get rid of the .8 standard deviation benchmark.

I agree.

Matt proposes the following rough scale:

Small effect: less than .05

Medium effect: .05 to .2

Large effect: .2 or larger

Matt reviews a bunch of educational studies to help come up with this table. While I don’t love that it averages a bunch of very different studies, at the very least it sets conservative estimates on effects and cost (given that averages include studies that don’t meet the highest bar for sample / duration / etc.).

Screen Shot 2019-05-29 at 5.10.56 PM.png

Take a look at where .4 standard deviations shows up. The New Orleans reforms are in the 90th percentile of magnitude but the 60th percentile of costs. New Orleans increased it’s pup-pupil by $1,400 in the years following Katrina, though it’s not clear to me that the money is what really drove the effect. But even if you assume it did, the results pass a ROI test.

Again, the New Orleans impacts are pretty remarkable.

In considering impact, cost, and scale, Matt also provides the following matrix:

Screen Shot 2019-05-29 at 9.49.25 PM.png

New Orleans does well.

In Sum: Toyotas > Ferraris 

When it comes to effect sizes, be very careful to review sample sizes, treatment duration, measurement proximity, and cost.

Holding out for .8 standard deviation effects is foolish. These effects will rarely occur and when they do they tend to be very hard to scale.

When it comes to large scale interventions across medium term time frames, effects above .2 standard deviation warrant our attention.

The most realistic path for broad academic gains is to look for meaningful jumps in student performance that are caused by an intervention that has a real chance of scaling over time. And then testing and scaling and testing and scaling.

In other words: Toyotas > Ferraris.

Not just a great teacher. Not only a few good schools. An entire city.

Last year, Arnold Ventures commissioned CREDO (out of Stanford University) to study the effects of charter and innovation schools in select cities across the country.

Most of the cities included in the study were cities where Arnold Ventures (and now The City Fund) have partnered with local leaders to expand high-quality schools.

The results just came in for New Orleans.

Worries of Stagnation in New Orleans

Many observers, including myself, have been worried about the stagnating proficiency scores in New Orleans.

Screen Shot 2019-05-08 at 12.45.06 PM

One of the major problems of looking at proficiency cut-offs is they are a somewhat arbitrary cut-off line. For example, if a student started off extremely below proficient, but then improved to one test score question away from being proficient, the student would still show up as not proficient.

This is why Arnold Ventures funded the CREDO analysis. CREDO’s methodology allows us to examine achievement growth across all of the performance spectrum.

CREDO Analysis of New Orleans 

Fortunately for kids in New Orleans, CREDO found that New Orleans schools are continuing to outperform the state on academic growth.

Here is the most important slide from their analysis:

Screen Shot 2019-05-08 at 12.49.21 PM.png

The statistically positive achievements effects indicate that students in New Orleans are catching-up with the state somewhere in the range of .05 to .1 standard deviations a year.

To put these gains in some context, the black white achievement gap in the United States is .8 standard deviations.

These are strong, citywide annual gains. And it’s very important to emphasize that these are city level effects.

In most cities, the longer poor children stay in the system the further behind they get. In New Orleans, the opposite is true: the longer you enroll in New Orleans public schools, the closer you get to your peers across the state. 

This is not the case of a few charter schools doing well. This is about the whole city making gains.

The goal of city based education reform should be for all kids to benefit, not just a few.

And that’s happening in New Orleans.

Hopefully, these gains will continue to compound and, eventually, further raise citywide proficiency, graduation, and post-secondary success rates.

Sometimes it doesn’t matter if it worked

Rand just release the initial evaluation of de Blasio’s Renewal School program.

The program cost $773 million.

The researchers found that the program did not improve student achievement.

The Renewal program has only been around for three years, so it feels a bit early to tell if it worked or not.

But I think “did it work?” is an important but secondary question to ask with a program like this.

The most important question is: “will it last?”

Even if the program had it gotten results, I’m skeptical that the program would have lasted.

The Renewal program had two major things going against it’s sustainability.

First, the program was deeply tied to de Blasio and the previous superintendent, Carmen Farina.

Second, there were no governance or legal protections for the program.

Taken together, this meant that the next mayor / superintendent would likely replace this initiative with their pet initiative.

This already had started to happen a year ago. When Farina stepped, the new superintendent, Richard Carranza, said the idea behind the Renewal program was “fuzzy.”

I viewed this as code for “it’s not my signature project.”

Admittedly, even I’m a bit surprised at how quickly the Renewal program collapsed. I thought it would at least live through the de Blasio administration.

We do live in a democracy (thankfully), so no government funded program, even if it works, is guaranteed to last forever.

But I would not have spent $773 million dollars on program that could be so easily undone.

I am such a deep believer in non-profit governance because it greatly increases the chance that something great can last.

In New Orleans, since the outset of the reforms, we’ve had three mayors and five superintendents.

And the work continues.

 

 

 

Revisiting Diane Ravitch’s “A Challenge to KIPP”

7 years ago, Diane Ravitch wrote a blog post called “A Challenge to KIPP.”

In the piece, Diane accuses KIPP of cherry picking students and challenges them to serve an entire district.

When I gave my lecture, I chastised KIPP for encouraging the public perception that all charter schools are better than all public schools and for failing to denounce the growing numbers of incompetent, corrupt, and inept charter schools. I talked about the oft-heard complaint that KIPP cherry picks its students and has high attrition, which KIPP denies. I challenged KIPP to take over an entire inner city school district that was willing and show what it could do when no one was excluded.

She then makes the point in a more forceful way:

KIPP should find an impoverished district that is so desperate that it is willing to put all its students into KIPP’s care. Take them all: the children with disabilities, the children who don’t speak English, the children who are homeless, the children just released from the juvenile justice system,  the children who are angry and apathetic, and everyone else. No dumping. No selection. No cherry picking. Show us what you can do. Take them all.

This is part of what made New Orleans so important. KIPP didn’t grow to serve every kid in the city (which I don’t think would have been good, different kids thrive in different environments).

But charters did grow to serve every student in the city. And the last 20% of kids reached were harder to educate than the first 20% of kids reached.

But now, to use Diane’s words, charter schools in New Orleans “take them all.”

And they do it better than ever before.

Public charter schools in New Orleans have evolved to offer amazing programs for students with severe special needs. They also serve students in tough life circumstances, like teen mothers and those with extreme mental health and behavior diseases. None of these programs are perfect, but they are so much better than what New Orleans offered these students before, and some of these programs are on the path to be national exemplars.

And what about the results?

As readers of this blog know, New Orleans achieved some of the most impressive student achievement gains in our country’s recent history.

So is Diane Ravitch now saying: “New Orleans answered my challenge to KIPP. Charters enrolled all students and increased educational opportunity. I’m curious to learn more about how they did that. And I wonder if it could work in other cities?”

Of course not.

Unfortunately, with so many charter critics, the goalposts just continue to move.

But for the rest of us, we can keep the goalposts in place.

We can learn from successes like New Orleans, and we can try to figure out what’s scalable and what is not.

Back to the future: reflections on returning to New Orleans

I recently spent a week in New Orleans.

I lived in New Orleans for most of 1998-2015, save for a few years away at law school. Simply landing at the airport brings back a lot of emotions.

My friends in New Orleans were mostly my education colleagues. So catching-up with friends also means catching-up with amazing education and civic leaders.

______

My week in New Orleans was a shot in the arm.

In many cities it is becoming difficult to empower educators. In Los Angeles the school board just voted to ask the state to make it illegal to open new non-profit schools for the remainder of the year. Many hope to extend this to five years, if not forever.

In other cities advocates have thwarted creating easy to use online enrollment systems that can help families find great schools for their children.

These efforts are couched in the language of social justice. But their impact will not be just: educators will not be able to create, and parents will not be able to access, better public schools.

New Orleans stands apart from these cities. By next year, 100% of schools in the city will be governed by non-profit organizations. Each year, families can access the city’s online enrollment system to find a good fit for their children.

Writing in The New York Times, David Leonhardt  covered research showing large academic gains across the city, as well as positive trends in high school graduation, postsecondary enrollment, and postsecondary completion.

New Orleans citizens also have regained control of their schools. An elected board oversees the public education system. And the education leadership of the city is beginning to better reflect the students the schools serve. The school district is again black led. Patrick Dobard, the CEO of New Schools for New Orleans, is putting forth a great vision for the city.

______

As I work across the country, a lot of people are quick to tell me that the the education transformation in New Orleans could never happen in their city, nor would they want it to.

I’m beginning to wonder if I’m too conciliatory in my response. I always say that every city is different; that each city needs to blaze its own path. This is of course true and it is why I say it.

But I think from now on instead of my first response being “every city is different,” I’m going start with “why not?”

I want to be more direct because I think a part of the reason people dismiss New Orleans is that it relieves them of the burden of exploring if the best solution might be the solution they don’t want to deal with.

Yes, the New Orleans path might not be right for every city. But New Orleans is the only majority African-American student city in the nation to rank in the top ten for most academic growth between 2009 and 2015

If you’re a city leader trying to make public education better for low-income families, you should study New Orleans, not write it off.

And I would definitely not hold New Orleans up as a future to be avoided at all costs. Those advocating against non-profit schools claim that the private operation of public schools will ruin public education.

In New Orleans the public schools are operated by private non-profit organizations.

But public education has not been ruined. It has been revitalized and depoliticized.

Instead of worrying about the next top down mandate, educators can build enduring institutions that will serve students well for decades.

Instead of fretting about whether the next school board election will lead to strategic chaos, district leaders can thoughtfully evolve their oversight of public schools.

Instead of fearing that the only way to get into a good public school is to be able to afford an expensive house, families have much more equitable access to public schools in the city.

If this is the hell of privitization, what word should we use to describe the chaos of public education systems in Houston, Nashville, and so many other cities across the country?

_______

I don’t know if my returning to New Orleans was going back to our country’s future. The odds are surely against it.

And of course New Orleans has thousands of more problems to solve in its journey to create an amazing public school system, both in terms of improving its K12 system and expanding postsecondary options.

But what city is better equipped to solve its next wave of challenges? I can’t think of any.

If you’re feeling beat down about your city or country, I suggest a trip down to NOLA.

In so many ways, it will lift your spirits.

 

Early signs that the New Orleans reforms did more than raise test scores

For the past ten years, I’ve been worried that the test score gains we were seeing in New Orleans wouldn’t lead to longer-term benefits for New Orleans students.

I worried about this both because of the research showing that increases in test scores are not always correlated to better life outcomes, as well as the fact that many colleges in Louisiana are pretty mediocre and could still fail to educate students even if they came in better prepared.

Thankfully, just published research by Doug Harris and Matt Larson provides early indication that New Orleans students have both achieved an increase test score performance *and* better post-secondary outcomes.

While it’s wonderful to see this data, we’ll continue to learn more as additional cohorts student graduate from the new public system. So far, only a few cohorts of New Orleans students attended all of their high school and post-secondary education post-Katrina.

My expectation is that the results will improve overtime, as the major high school reforms took place later in the reform effort. Good non-profit charter school operators now run many more of the high schools than they did in the few years after the storm.

Lastly, it’s worth noting that the New Orleans reforms were not a randomized controlled trial. A tragedy occurred and educators and families did their best in the aftermath. I’m not expert enough to judge the author’s methodological choices, but am eager to see other researchers weigh in on whether or not this is the best way to estimate the impacts of the reforms.

Test Scores

Across all subjects, the researchers found +.4-.6 standard deviations effects.

A rough rule of thumb is that .25 standard deviation increase equals an additional year of learning.

By this estimate, New Orleans students achieved an additional two years of learning relative to the education they would have received before the storm.

Screen Shot 2018-07-22 at 8.30.07 AM

High School Graduation

The researchers found an estimated six percentile point increase for high school graduation. While positive, this is below the gains of twenty points often touted by reformers. The researchers explain that because high school graduation was going up across the state, some of the twenty point gains would have likely occurred without the reforms.

Screen Shot 2018-07-22 at 8.32.47 AM.png

College Enrollment

The researchers found a 10-15 percentage point jump in college entry. The effect was particularly high for on-time college enrollment (enrolling right after high school), where the rate jumped from 22% to to 37%.

Screen Shot 2018-07-22 at 8.39.13 AM.png

College Graduation 

Of course, we don’t just want to see students enter college. We hope that they graduate as well.

The researchers found +4 percentage point gain for on-time college graduation (graduation within five years of enrollment), from about 10 percent to 14 percent.

Screen Shot 2018-07-22 at 8.40.27 AM

The results for staying in college for two years were higher: about +8 percentage points. So it seems likely that many students who entered college because of the reforms made it through at least two years.

But, again, these are very preliminary results. The students in the data set for college graduation experienced very little of the reform efforts. This rate should go up over time. I really hope that 14% on time graduation rate will rise significantly in the coming years.

Cost / Benefit of the Reforms

The researchers end their paper by noting that the ROI of the New Orleans’ reform efforts is higher than class size reforms and is within range of the famous Perry pre-school study (which is probably on the very high end for pre-k results).

Screen Shot 2018-07-22 at 8.55.14 AM

My guess is that the New Orleans effort is likely near the top of what we can expect for these types of city level reforms. The low baseline achievement, coupled with the speed of the post-Katrina effort, meant that the reform efforts had the potential to show large increases quickly.

But even if other cities see a ROI of 50% less, that would still make the reforms worth scaling.

But, for now, it’s worth celebrating a bit for the students of New Orleans: more of them than ever in recent history appear to be on track for choice filled and meaningful lives.

The New Orleans reforms have been both impactful and popular but the gains are plateauing

The New Orleans reform efforts are nearly thirteen years old.

As a reminder, the efforts led to some of the most significant achievement gains in our country’s recent history.

ednext_XV_4_harris_fig01-small

In a recent Sean Reardon study that appeared in the New York Times, New Orleans was the only city that scored in the top ten for growth in the country and serves a majority of African-American students.

All of the other top performing districts, except for Chicago, barely serve any African-American students, as the chart of the top ten growth districts below shows:

nola AA.png

College enrollment rates have also skyrocketed, though we don’t yet have great data on college completion and post-secondary outcomes.

college enrollment NOLA

However, most of the test score gains were made in the first 7 years of the reforms, and the city is no longer increasing relative to overall state performance.

Screen Shot 2018-06-07 at 8.57.20 AM

The gains in New Orleans are very real. But so is the current plateauing of test score performance. It remains an open question whether or not the system will see another significant increase in student performance. My guess is that the increases will eventually pick up again, but at more modest rates.

A particularly difficult strategy question is what philanthropy can do to help New Orleans at this juncture.

Leaders in the city are both trying to double down on expanding the best operators, as well as help all schools increase their instructional rigor.

Over the long-haul, I believe that most gains will come from scaling the best school operators, selectively starting new operators, and replacing the worst. But I do think that supports to help all schools can lead to some improvement, though my expectations with these types of reforms are modest. Many smart people disagree on how to allocate funds across these two types of strategies. It will be interesting to see what we learn from New Orleans over the next few years on this issue.

____

But what do the people think?

The Cowen Institute just came out with its annual polling data on New Orleans education. The poll draws from registered voters in New Orleans.

When it comes to voter perception of the system, votes are split between “the system is getting better” and “the system is staying the same.” Very few feel it’s getting worse.

Screen Shot 2018-06-07 at 9.03.53 AM

The perception mirrors the data: the system has improved, but these improvements are slowing down, and it’s unclear that things are getting much better now.

The public also continues to support charter schools and the city’s unified enrollment system.

Screen Shot 2018-06-07 at 9.06.49 AM

Only 17% of voters disagreed that charters had improved education, putting charter favorability at a 3:1 ratio.

Even more interesting: 70% of public school parents believed charter schools had improved education, compared to only 50% of those without children.

Charters seem to be winning those they serve but not fully winning those they don’t.

Open enrollment also only had 16% negative rate, putting it a 3:1 favorability rating.

___

Overall, the New Orleans reforms have been both impactful and popular.

Right after Katrina, neither of these outcomes were inevitable. In the early years of the reforms, school performance was very uneven and the reforms were very controversial.

It is incredibly difficult to transform a whole city’s educational system in a way that increases opportunities for children and garners the support of the public.

I am hopeful that other cities, such as Camden (see here for a good New York Times profile on the city’s efforts), will also do great things for kids and gain the public’s trust.

 

 

Most students in NOLA whose school is closed end up in one of their top choices for the following year. Here’s how.

This post from Ed Navigator is worth reading. It covers schools closures in New Orleans.

Over ten years after Katrina, and under an elected school board, New Orleans continues to selectively close underperforming schools.

I view this as a good thing, given the growing body of research that shows that school closures help kids when the students end up in better schools.

New Orleans uses a unified enrollment system to help kids get into better schools.

The unified enrollment system gives preference to students whose schools were closed the year before. If your school was closed, the algorithm bumps you to the top of the list for any school you want to get into.

Ed Navigator works with families whose schools have been closed, so that they can help select great schools.

The result?

This year, 87% of students who attended a closing school and used the enrollment system received on of their top three choices for the next school year.

94% of the students will now attend a school that is rated higher by the state’s grading system.

The system is by no means perfect. My biggest critique is that the state’s grading system still relies too heavily on absolute test scores (rather than growth). I also understand the counterarguments that government should never close schools and should instead let enrollment patterns (driven by parental choice) determine which schools grow and which close.

But I would rather have the New Orleans enrollment and closure system than just about any other big city system in the country.  In too many cities, really bad schools stay open for too long. And if anything happens to them, kids often end up in schools that are just as bad.

This is not what happens in New Orleans.

It’s also great to see Parag Pathak (and his colleagues) work in action. Parag recently won the John Bates Clark award in part because of his contributions to working on unified enrollment systems.

It’s rare that an idea goes from the ivory tower to think tanks to actual implementation by a democratically elected body to  helping citizens.

This is really great to see. And really great for kids in New Orleans.

The Atlantic vs. Facts

I’m sympathetic to how hard it is for journalists to cover complicated policy areas.

Translating policy into readable articles that both inform and entertain is difficult. In this sense, writing a policy a blog is a luxury. For the most part, I just get to stick to the research and ideas.

But journalists do have a duty to accurately cover the basic facts of the issue they are covering.

In her recent article on Japan’s education system, Alana Semuels included a throw away line about New Orleans education reforms; she writes:

The equity in Iitate stands in stark contrast to a place like New Orleans, which was also hit by a disaster. While Japan’s national government tried to ensure that students in the affected area got more resources after the accident, officials in New Orleans disinvested in the public educational system in their city. Public-school teachers were put on leave and dismissed, many students disappeared from schools’ rolls, and the New Orleans system now consists almost entirely of charter schools.

If you click on the links, which presumably justify her claims, you don’t get taken to supporting research; instead, you get taken to an op-ed.

As it happens, rigorous research has been done on the New Orleans reforms.

In terms of student achievement, after Katrina New Orleans saw some of the biggest gains ever recorded in an urban school district. You can read about the research here.

In the article, the authors of the study note:

For New Orleans, the news on average student outcomes is quite positive by just about any measure. The reforms seem to have moved the average student up by 0.2 to 0.4 standard deviations and boosted rates of high school graduation and college entry. We are not aware of any other districts that have made such large improvements in such a short time.

Rigorous research has also been done on how public spending in education changed in New Orleans after Katrina. Contrary to Semuels claim, spending went up.

The authors of the study write:

New Orleans’ publicly funded schools spent 13% ($1,358 per student) more per pupil on operating expenditures than the comparison group after the reforms, even though the comparison group had nearly identical spending before the reforms.

A quick google search could have turned up both of these studies.

I wish Semuels had taken the time to review these studies.

Because she did not, thousands of readers have been misinformed.