New Orleans, the New Yorker, and the perils of flawed comparisons

Screen Shot 2018-08-10 at 5.49.26 PM

Gary Sernovitz just wrote an interesting piece in the New Yorker on the New Orleans public school system.  He argues that the New Orleans public school system is designed around free market principles that are, at times, being poorly applied to the public sector. Gary draws lessons from his time serving on the board of a charter school that eventually lost its charter for financial reasons.

Before considering his arguments, I just want to give a thank you to Gary. He joined a charter school board and devoted a lot time trying to make public education better. It’s great to see people with his commitment and intelligence serving on charter boards. I hope more people follow his lead.

__

Gary argues that the New Orleans public education system is designed around “the engines of the free market – autonomy, competition, and customer choice” but that these design principles are currently inadequate to meet the aims of public education in New Orleans.

Gary points to three main problems with the New Orleans system: rewards, incentives, and start-up capital.

Rewards

Gary argues that New Orleans’ schools demand crazy work hours but offer mediocre compensation. Unlike for-profit founders, there is no dream of a financial exit for charter founders or their teams.

I don’t disagree with these facts. If you want to get rich, starting a non-profit charter school in New Orleans is not the way to go.

But this is the problem with Gary’s premise: the New Orleans public school system was never designed to mimic all parts of the free market. The goal of working in a charter school is not to get rich; it’s to do good while earning enough to live a middle or upper middle class lifestyle.

The reward model provides a different set of rewards for different kinds of educators.

Many teachers teach in charter schools for 3-5 years, work long hours, and are rewarded with the meaning that comes with knowing you helped others. They then go onto other things.

A minority of teachers find that teaching is their lifelong calling. Their hours tend to go down overtime as their mastery of teaching goes up. The most skilled teachers in New Orleans can achieve in 50 hours a week what it takes a novice to achieve in 70 hours a week. Their rewards come from the joy of doing good work, building meaningful relationships with children, and earning a stable middle class income.

Another set of teachers move into administrative roles. They tend to spend another 5-10 years working in leadership positions. Their rewards come from the challenges of leadership, seeing impact at a larger scale, and earning an upper middle class income.

Yes, current model does rely on younger teachers, who work more hours, and leave the classroom more frequently than their traditional peers. But this model is delivering better results for children than the old talent model. And it has been doing so for over a decade, which leads me to believe that the talent model is sustainable.

That being said, I’m open to the idea the current model is not optimal. I can think of two potential improvements: raising taxes to increase educator salaries, or simply encouraging charters to be for-profits so there can actually be financially exits and equity based compensation. But New Orleans is a poor city in a poor state, so I’m skeptical that New Orleans will be able to raise salaries by large amounts. As for for-profit charters, while a reasonable idea in theory, their results to date have been underwhelming, so I’m not holding my breath here either.

Rather, I think New Orleans has organically evolved to the best talent model under very imperfect conditions.

Incentives 

Gary’s main criticism of the New Orleans public school system is that it does not fully fund the costs for students with special needs. In market terms, it gets the price wrong.

Gary sat on the board of Cypress Academy, which intentionally enrolled a lot of students with special needs. These students cost more money to serve well.

The New Orleans per-pupil revenue system is designed with this reality in mind: I believe New Orleans has the most weighted per-pupil system in the country. Schools receive up to 3x of the regular per-pupil to serves students with severe special needs.

Because of this model, numerous schools in the city are able to serve a lot of students with special needs. Many networks have even developed specialized programs for high needs students.

I’m open to the idea that the weights need to be further increased. But the Cypress financial model should have been built around the existing financial regulatory regime. It is well known to all charter operators in the city, and Cypress should not have opened if they did not have a viable financial model to serve the students they wanted to serve.

If Cypress thought the per-pupil funding system was wrong, it should have advocated for policy change before opening its doors. Instead, it opened with an unsustainable model. This was a mistake.

Start-up Capital 

Gary argues that there is not enough start-up money to help a new charter school get to scale.

I don’t think this is true.

When I worked at New Schools for New Orleans, we helped 10+ new charter school start-ups open schools, and none failed for financial reasons. Rather, all of them received enough funds (usually $500K to $1m in philanthropy) to cover their operations until they reached scale.

My understanding is that Cypress Academy received start-up grants in the range of other successful start-up charter schools in New Orleans, such as Bricolage Academy.

And, again, none of the financial realities should have been a surprise to the founders of Cypress. If they knew they were going to run a deficit over the first few years, they should not have opened unless they were fairly certain they could raise the necessary philanthropy to cover this gap.

___

Dozens of new charter schools have opened successfully over the past decade. These schools operate sustainable talent models, serve students with special needs, and scaled with the support of philanthropy.

Cypress Academy failed for reasons that seem to be mostly predictable. The balance of the fault appears to be with the school, not with system.

Lastly, it’s worth emphasizing that the New Orleans public school system is not designed to be a free market. It’s a publicly funded system operated by non-profit organizations.

Yes, it has more market mechanisms than a traditional government operated system, but it’s so far from being a free market that most comparisons to free markets obscure more than they illuminate.

The City Fund

While there are amazing public schools across the country, few cities have been able to increase educational opportunity for all children.

Over the past fifteen years, this has begun to change. Denver, Washington D.C., and New Orleans have made their entire public education systems better. Other cities, like Indianapolis and Camden, have taken these breakthroughs, tailored them to their local contexts, and seen promising early results. Because of this work, hundreds of thousands of children have benefited from a better public education. These students are more prepared than ever to further their education, get good jobs, and lead lives filled with opportunity. We are now creating a new non-profit organization, The City Fund, to expand on this work.

Previously, several of us operated within a dual structure supporting the education giving for both the Laura and John Arnold Foundation and the Hastings Fund. We are  in the process of changing this structure and have added new team members to create The City Fund. The Laura and John Arnold Foundation and the Hastings Fund will continue their support of the work as anchor funders.

The new organization includes leaders from Education Cities, as well as expert practitioners from the state, district, charter and non-profit sectors. This new team will help us provide better support to local leaders. We are just getting started and will have a website up in short order, but now that our team is hired, we wanted to share the news.

All of us are united by a common perspective. First, right now, too many students do not have access to a great public school. Second, we believe this can change.

We share this belief because we have seen better schools improve the lives of families in all of our respective work. We’ve seen awesome teachers inspire children to build the knowledge, skills, and values needed to make the world a better place. We’ve seen world class public schools provide rich educational experiences to all their students, regardless of race or economic circumstances.

___

Our nation’s education system is a complicated patchwork of thousands of local school districts. Improvements will not happen overnight, nor should they.

Rather, cities across the country are constantly innovating, and when a few cities do something that seems to be working, philanthropy can help shine a light on these local successes. If other cities are interested, philanthropy can help test these breakthroughs at a little larger scale.

The cities we have supported have made things better in their own way, but several commonalities stand out: each city increased the number of public schools that are governed by non-profit organizations; each city created an easy to use enrollment system that helps families find a great public school for their children; and each city provides families with transparent information about public school quality. We believe these strategies hold promise.

While we are optimistic that the work we’re supporting will succeed at the next level of scale, much more work, innovation, and research is needed. Over the coming years, we’ll continue to support a small set of local education leaders. We’ll also work with university researchers to study these local efforts. If cities show progress, we hope other cities will follow. If they don’t, we hope other promising innovations are able to scale, so that all students can have access to amazing public schools.

Chris Barbic

Gary Borden

Ken Bubp

Beverly Francis-Pryce

Ethan Gray

David Harris

Kevin Huffman

Noor Iqbal

Neerav Kingsland

Jessica Pena

Liset Rivera

Kevin Shafer

Kameelah Shaheed-Diallo

Gabrielle Wyatt

Early signs that the New Orleans reforms did more than raise test scores

For the past ten years, I’ve been worried that the test score gains we were seeing in New Orleans wouldn’t lead to longer-term benefits for New Orleans students.

I worried about this both because of the research showing that increases in test scores are not always correlated to better life outcomes, as well as the fact that many colleges in Louisiana are pretty mediocre and could still fail to educate students even if they came in better prepared.

Thankfully, just published research by Doug Harris and Matt Larson provides early indication that New Orleans students have both achieved an increase test score performance *and* better post-secondary outcomes.

While it’s wonderful to see this data, we’ll continue to learn more as additional cohorts student graduate from the new public system. So far, only a few cohorts of New Orleans students attended all of their high school and post-secondary education post-Katrina.

My expectation is that the results will improve overtime, as the major high school reforms took place later in the reform effort. Good non-profit charter school operators now run many more of the high schools than they did in the few years after the storm.

Lastly, it’s worth noting that the New Orleans reforms were not a randomized controlled trial. A tragedy occurred and educators and families did their best in the aftermath. I’m not expert enough to judge the author’s methodological choices, but am eager to see other researchers weigh in on whether or not this is the best way to estimate the impacts of the reforms.

Test Scores

Across all subjects, the researchers found +.4-.6 standard deviations effects.

A rough rule of thumb is that .25 standard deviation increase equals an additional year of learning.

By this estimate, New Orleans students achieved an additional two years of learning relative to the education they would have received before the storm.

Screen Shot 2018-07-22 at 8.30.07 AM

High School Graduation

The researchers found an estimated six percentile point increase for high school graduation. While positive, this is below the gains of twenty points often touted by reformers. The researchers explain that because high school graduation was going up across the state, some of the twenty point gains would have likely occurred without the reforms.

Screen Shot 2018-07-22 at 8.32.47 AM.png

College Enrollment

The researchers found a 10-15 percentage point jump in college entry. The effect was particularly high for on-time college enrollment (enrolling right after high school), where the rate jumped from 22% to to 37%.

Screen Shot 2018-07-22 at 8.39.13 AM.png

College Graduation 

Of course, we don’t just want to see students enter college. We hope that they graduate as well.

The researchers found +4 percentage point gain for on-time college graduation (graduation within five years of enrollment), from about 10 percent to 14 percent.

Screen Shot 2018-07-22 at 8.40.27 AM

The results for staying in college for two years were higher: about +8 percentage points. So it seems likely that many students who entered college because of the reforms made it through at least two years.

But, again, these are very preliminary results. The students in the data set for college graduation experienced very little of the reform efforts. This rate should go up over time. I really hope that 14% on time graduation rate will rise significantly in the coming years.

Cost / Benefit of the Reforms

The researchers end their paper by noting that the ROI of the New Orleans’ reform efforts is higher than class size reforms and is within range of the famous Perry pre-school study (which is probably on the very high end for pre-k results).

Screen Shot 2018-07-22 at 8.55.14 AM

My guess is that the New Orleans effort is likely near the top of what we can expect for these types of city level reforms. The low baseline achievement, coupled with the speed of the post-Katrina effort, meant that the reform efforts had the potential to show large increases quickly.

But even if other cities see a ROI of 50% less, that would still make the reforms worth scaling.

But, for now, it’s worth celebrating a bit for the students of New Orleans: more of them than ever in recent history appear to be on track for choice filled and meaningful lives.

Families in Washington D.C. are conflicted between what works and what they want to work

How do you figure out what parents want? 

There are two primary ways to figure out what families want from schools:

  1. Ask them what they want;
  2. Or observe what decisions they make.

The Washington D.C. Auditor’s office just released a public opinion report using the former method (asking families). Washington D.C. has a unified enrollment system where you can track what decisions parents are actually making. I wish this report would have used this data as well (like this report did in New Orleans). But despite this flaw, the opinion research is telling.

Families in D.C. are conflicted between what works and what they want to work 

D.C. families had the following preferences:

  1. School preference: Families say they value educator and academic quality much more than a school being close to home.
  2. School type preference: Families prefer charter schools over their local neighborhood school.
  3. Policy preference: Families want to invest more in neighborhood schools rather than giving more parents chance to opt into a charter or out of boundary traditional school.

Putting this together: families want public schools with great educators and challenging academics; they think charters schools are more likely to provide this; but they want more money put into neighborhood schools.

Of course, this on average. No doubt there is immense diversity in opinion across families.

Why do many families want to fix what’s not working rather than expand what is working? 

I can think of a couple reasons:

Empathy and bad strategy: Families feel bad for the kids who are stuck in the worse schools, and the most intuitive answer to “what should we do with more resources?” is “fix what’s not working.” In the private sector, fortunes have been lost on this fallacy. In the public sector, fortunes are spent on efforts that have failed for decades.

Virtue signaling: Families want to express that they are good people and not selfish, and in our society saying you want to invest more in neighborhood schools is a way to signal that you’re a good person.

They want their neighborhood schools to improve, they just don’t want their kids to suffer in the meantime: Perhaps families would rather send their own kids to the local neighborhood school, so they do want these schools to get better, but they’re just not willing to send their children there while they wait for this to happen.

I imagine all these factors are at play.

How do you respond?

I think this issue will continue to play out in high choice cities that are providing families with a lot of great public school options.

When it comes to their own children, families will, on average, send their children to the best public schools.

But when it comes to public opinion, families will say we should invest more in the schools that aren’t working.

This puts public charter school supporters in a very tricky position.

How can we try to create more great public schools under this dynamic?

I think charter supporters need to be very cautious in engaging in major, public citywide or statewide fights.

We can win an individual families’ hearts and minds when it comes to their own child, but the policy battles are tougher.

In other words: just keep on opening amazing new public charter schools. The system will become the best version of itself through educators opening up great public schools and families finding the best fit for their own children.

But it is also worth considering pacing: too much growth too quickly can shift an individual school opening debate into a citywide policy debate – a type of debate that is easily lost, as it was in Massachusetts.

Lastly, gradually opening up new great public schools will ultimately give families both what they want for their own children and what they want for the city: if every public school is great, than by default every school in every neighborhood will be great too.

Data from the report

Screen Shot 2018-07-14 at 9.15.30 AMScreen Shot 2018-07-14 at 9.15.03 AMScreen Shot 2018-07-14 at 9.14.55 AMScreen Shot 2018-07-14 at 9.13.13 AM

The problem with “risk / mitigation” sections in business plans

I see a lot of business plans and our team writes a lot of strategy memos.

I’ve always been a little frustrated with “risk / mitigation” sections of these documents, including some I’ve written myself.

“Risk / mitigation” sections feel like an exercise in confirmation bias. They almost always affirm the correctness of the business plan’s thesis.

The logic path of a risk / mitigation section is: “here’s our idea, there are some risks, but here’s why we’re going to do our idea anyway.

This is a modestly useful exercise. It’s worth understanding the risks of a plan and preparing for them. But this type of thinking does not force you to analyze the “crux” of your plan – i.e, what are the major trade-offs you’re making, and under what future conditions could you be proven wrong.

I prefer to see both of these lines of thinking in a business plan: first, a clear identification of the trade-offs in any major decision, and, second, a statement of what might make you change your mind about your plan’s thesis.

Any very hard decision will include difficult trade-offs. And any hard decision might also result in you being wrong. Identifying these ahead of time will help you be mentally prepared to make difficult shifts down the road.

It will also decrease the probability that you drink your own kool aid.  There is a humbling effect in writing down potential future conditions that will indicate that you are near failing or have already failed.

I hope that one day every business plan includes “trade-off” and “we will know we were wrong if” sections.

The New Orleans reforms have been both impactful and popular but the gains are plateauing

The New Orleans reform efforts are nearly thirteen years old.

As a reminder, the efforts led to some of the most significant achievement gains in our country’s recent history.

ednext_XV_4_harris_fig01-small

In a recent Sean Reardon study that appeared in the New York Times, New Orleans was the only city that scored in the top ten for growth in the country and serves a majority of African-American students.

All of the other top performing districts, except for Chicago, barely serve any African-American students, as the chart of the top ten growth districts below shows:

nola AA.png

College enrollment rates have also skyrocketed, though we don’t yet have great data on college completion and post-secondary outcomes.

college enrollment NOLA

However, most of the test score gains were made in the first 7 years of the reforms, and the city is no longer increasing relative to overall state performance.

Screen Shot 2018-06-07 at 8.57.20 AM

The gains in New Orleans are very real. But so is the current plateauing of test score performance. It remains an open question whether or not the system will see another significant increase in student performance. My guess is that the increases will eventually pick up again, but at more modest rates.

A particularly difficult strategy question is what philanthropy can do to help New Orleans at this juncture.

Leaders in the city are both trying to double down on expanding the best operators, as well as help all schools increase their instructional rigor.

Over the long-haul, I believe that most gains will come from scaling the best school operators, selectively starting new operators, and replacing the worst. But I do think that supports to help all schools can lead to some improvement, though my expectations with these types of reforms are modest. Many smart people disagree on how to allocate funds across these two types of strategies. It will be interesting to see what we learn from New Orleans over the next few years on this issue.

____

But what do the people think?

The Cowen Institute just came out with its annual polling data on New Orleans education. The poll draws from registered voters in New Orleans.

When it comes to voter perception of the system, votes are split between “the system is getting better” and “the system is staying the same.” Very few feel it’s getting worse.

Screen Shot 2018-06-07 at 9.03.53 AM

The perception mirrors the data: the system has improved, but these improvements are slowing down, and it’s unclear that things are getting much better now.

The public also continues to support charter schools and the city’s unified enrollment system.

Screen Shot 2018-06-07 at 9.06.49 AM

Only 17% of voters disagreed that charters had improved education, putting charter favorability at a 3:1 ratio.

Even more interesting: 70% of public school parents believed charter schools had improved education, compared to only 50% of those without children.

Charters seem to be winning those they serve but not fully winning those they don’t.

Open enrollment also only had 16% negative rate, putting it a 3:1 favorability rating.

___

Overall, the New Orleans reforms have been both impactful and popular.

Right after Katrina, neither of these outcomes were inevitable. In the early years of the reforms, school performance was very uneven and the reforms were very controversial.

It is incredibly difficult to transform a whole city’s educational system in a way that increases opportunities for children and garners the support of the public.

I am hopeful that other cities, such as Camden (see here for a good New York Times profile on the city’s efforts), will also do great things for kids and gain the public’s trust.

 

 

Some good music (nothing to do with education!)

I’m on vacation this week. This has involved a good amount of hiking, reading murder mysteries, reading nonfiction (Elephant in the Brain), good meals, and most importantly a lot of great time with my wife.

I haven’t posted about music in awhile, so see below for some music that I really like, and you might like too.

Angues and Julia Stone Chateau

Currently in the lead for my song personal song of the summer. Young love, wonderlust, and alternating male / female vocals – what more do you want for a summertime jam?

 

Loyal Light Up For You 

Wonderful summertime beats and a gentle refrain.

 

Arizona –  Cross My Mind

If you’re noticing a theme here you’re correct: I think I’ve somehow gotten into what the kids call “tropical house” or something like that. Who would have thought that at age 38  my tastes would take a turn toward low key, electric, pop music?

 

Valerie June – Astral Plane

Like the best of Dylan, Valerie June’s most amazing songs inspire complex emotions through seemingly “simple” music.

 

Tom Odell – Another Love

Great piano, great voice, and good melodrama. I’d also recommend this mix version. If you like that I’d also check out this version of Magnetized at BBC Radio1 show.

 

Mondo Cozmo – Thunder 

Sometimes songs just keep on walking up to the line of being too cheesy but then keep stepping back from the line in time to be saved – this is one of those songs.

 

Transviolet – Girls Your Age

Coming of age song of sorts – haunting voice and beat.

 

Alex the Astronaut Not Worth Hiding

Young Australian singer writes about coming out. I found it moving.

 

Mt. Joy – Silver Lining

This is the kind of music I loved in my mid-20s. Still have much love for it!

 

LP – Lost on You

A little rock and roll with alternating meanings of the song title woven throughout the song? Yes!

 

Ella Vos – White Noise

This version is good but might be slow for some, so check out the faster remix version too.