Sometimes it doesn’t matter if it worked

Rand just release the initial evaluation of de Blasio’s Renewal School program.

The program cost $773 million.

The researchers found that the program did not improve student achievement.

The Renewal program has only been around for three years, so it feels a bit early to tell if it worked or not.

But I think “did it work?” is an important but secondary question to ask with a program like this.

The most important question is: “will it last?”

Even if the program had it gotten results, I’m skeptical that the program would have lasted.

The Renewal program had two major things going against it’s sustainability.

First, the program was deeply tied to de Blasio and the previous superintendent, Carmen Farina.

Second, there were no governance or legal protections for the program.

Taken together, this meant that the next mayor / superintendent would likely replace this initiative with their pet initiative.

This already had started to happen a year ago. When Farina stepped, the new superintendent, Richard Carranza, said the idea behind the Renewal program was “fuzzy.”

I viewed this as code for “it’s not my signature project.”

Admittedly, even I’m a bit surprised at how quickly the Renewal program collapsed. I thought it would at least live through the de Blasio administration.

We do live in a democracy (thankfully), so no government funded program, even if it works, is guaranteed to last forever.

But I would not have spent $773 million dollars on program that could be so easily undone.

I am such a deep believer in non-profit governance because it greatly increases the chance that something great can last.

In New Orleans, since the outset of the reforms, we’ve had three mayors and five superintendents.

And the work continues.

 

 

 

Minor feedback and reflections on Ray Dalio’s capitalism manifesto

Ray Dalio just published a two part manifesto on how capitalism needs to be reformed.

Dalio’s Diagnosis 

Dalio’s main point is that “capitalists don’t know how to divide the pie well and socialists don’t know how to growth the pie well.”

This is a problem for capitalist countries because rising inequality has both negative affects on individuals and countries: high inequality reduces economic mobility and weakens political governance.

I am not an expert in inequality, economic mobility, or governance, so it’s hard for me evaluate his specific claims.

But my instinct is too put a little less emphasis on capitalism.

Rising inequality seems to be the trend of almost all political systems.

Walter Schiedel’s book The Great Leveler does a good job of showing how inequality tends to rise in most societies (any system can be gamed) and that it’s often only war, revolutions, state collapse, and plagues that bring inequality back down.

Dalio thinks there’s something specifically inherent about capitalism that leads to inequality. I think there’s something generally inherent about governments and power.

Once humans left hunter gatherer societies, rising inequality has been the norm in most societies.

Capitalism is just the latest manifestation of the trend.

Stability vs. Growth 

Another argument which is somewhat implicit in Dalio’s argument but I wish would have been more explicit: given the incredible compounding of even small GDP increases, we should care much more about societal stability than maximizing growth.

Over the long-run, humans will be very rich by today’s standards so long as we can survive a couple more centuries.

In most cases, growth and stability go hand-in-hand, but when they don’t (perhaps in the case of inequality), we should err on the side of stability.

Can We Do Anything About It?

Schiedel’s book presents a bleak picture about a society’s ability to reduce inequality.

But humans haven’t been around for that long. We’ve never been this wealthy. And never before have so many humans lived in democratic societies.

It took us from the Industrial Revolution to the end of World War II to find a governmental system that fit our last major round technological and economic advancement.

Most advanced countries are now some form of capitalist democracies with large welfare states (at least by historical standards).

I hope our next governance search requires less bloodshed.

If it’s simply a matter of adjusting our current system, this might be possible.

If it’s a matter of finding whole new systems of governance, then that likely means wars and revolutions, which we should all fear given the weapons at our disposal.

Revisiting Diane Ravitch’s “A Challenge to KIPP”

7 years ago, Diane Ravitch wrote a blog post called “A Challenge to KIPP.”

In the piece, Diane accuses KIPP of cherry picking students and challenges them to serve an entire district.

When I gave my lecture, I chastised KIPP for encouraging the public perception that all charter schools are better than all public schools and for failing to denounce the growing numbers of incompetent, corrupt, and inept charter schools. I talked about the oft-heard complaint that KIPP cherry picks its students and has high attrition, which KIPP denies. I challenged KIPP to take over an entire inner city school district that was willing and show what it could do when no one was excluded.

She then makes the point in a more forceful way:

KIPP should find an impoverished district that is so desperate that it is willing to put all its students into KIPP’s care. Take them all: the children with disabilities, the children who don’t speak English, the children who are homeless, the children just released from the juvenile justice system,  the children who are angry and apathetic, and everyone else. No dumping. No selection. No cherry picking. Show us what you can do. Take them all.

This is part of what made New Orleans so important. KIPP didn’t grow to serve every kid in the city (which I don’t think would have been good, different kids thrive in different environments).

But charters did grow to serve every student in the city. And the last 20% of kids reached were harder to educate than the first 20% of kids reached.

But now, to use Diane’s words, charter schools in New Orleans “take them all.”

And they do it better than ever before.

Public charter schools in New Orleans have evolved to offer amazing programs for students with severe special needs. They also serve students in tough life circumstances, like teen mothers and those with extreme mental health and behavior diseases. None of these programs are perfect, but they are so much better than what New Orleans offered these students before, and some of these programs are on the path to be national exemplars.

And what about the results?

As readers of this blog know, New Orleans achieved some of the most impressive student achievement gains in our country’s recent history.

So is Diane Ravitch now saying: “New Orleans answered my challenge to KIPP. Charters enrolled all students and increased educational opportunity. I’m curious to learn more about how they did that. And I wonder if it could work in other cities?”

Of course not.

Unfortunately, with so many charter critics, the goalposts just continue to move.

But for the rest of us, we can keep the goalposts in place.

We can learn from successes like New Orleans, and we can try to figure out what’s scalable and what is not.

Beware of under powered studies

Mathematica just released a study on how charter middle schools impact college enrollment and completion. Before I dig into the study, some quick context.

The research base on the connection between test scores and life outcomes

An important research finding over the past decade has been that schools with neutral test score impacts sometimes achieve positive lifetime outcomes for their students.

Equally important, schools with positive test score impacts often don’t achieve positive life outcomes for their students.

Another important finding (a finding that is often glossed over by my libertarian leaning friends) is that schools with negative test score impacts are much less likely to achieve positive lifetime outcomes for their students.

Taken together, these findings have narrowed my belief in test based accountability. Previously, I put a lot of weight on the difference between positive, neutral, and negative test score schools.

Now I focus more attention on selectively transforming schools with negative test score impacts.

I also care much more about parent demand than I used to.

The Mathematica Study Limitations 

The Mathematica study has two very major limitations.

First, it covers charter schools from a previous era (2005-2007). In the subsequent decade, many charter schools (especially those we work closely with) have increased their focus on getting kids to and through college and into meaningful careers.

Second, the study is very small. The sample only included 10 urban charter schools. Additionally, only 3 schools in the entire study had poverty rates over 75%!

The small sample means the study is fairly under powered. With a sample this small, only large effects will be picked up. A modest but positive correlation between achievement and college graduation, for example, would not be statistically significant in this study. 

Other similar studies have looked at entire states, and included many more high poverty charter schools, so I place much less weight on this study.

Findings

The study did not find a statistically significant relationship between test score and college completion results. The chart below details the major findings, with each dot representing a school.

College Completion 

mathematica 2

When you eyeball the graph, some trends emerge, especially the low number of schools in the bottom right quadrant (high test effects, low college effects). 

When I emailed the authors about this trend, they said they were not statistically significant and we should be cautious to draw conclusions by looking at patterns the scatter plot. 

In Sum

The small sample size and the decade old study window period mean that the study itself won’t move my opinion on the issue that much.

Only 3 schools in the study were high poverty schools. Nearly all of the schools that we work with are high poverty schools.

In partnership with the Arnold Foundation, we’re supporting a bunch of more research on the issue, and I’m excited to learn more. 

I will change my mind if the findings change.  

 

 

Who benefits from neighborhood schools in Washington D.C.?

Screen Shot 2019-03-25 at 5.41.54 PM.png

Thomas Toch (a City Fund grantee) has a great piece on Washington D.C.’s public education system in the Washington Post Magazine.

The article tracks a few families through their process of finding public schools for their children.

A lot of progress has been made in D.C., and it was wonderful to read about this progress through lens of families trying to find a great public school. Much of our work is strategically abstract, but there’s nothing abstract about a family trying to find a school.

I strongly encourage you to read the whole thing. It will likely be one of the best pieces you read this year about what public education could be.

I’ll just highlight one important point from the article:

A so-called in-boundary preference makes it particularly tough to get into the city’s best traditional neighborhood schools. Carving the city into attendance zones and guaranteeing students a spot in the traditional public school in their zone is a way of reconciling school choice with the right of every student to a free public education.

But not surprisingly, families living within the boundaries of top schools exercise their in-boundary advantage more often than families with weak neighborhood options. And because many of the highest-performing neighborhood schools are in predominantly white and more-affluent sections of the city, the My School DC preferences weaken the system’s ability to reduce long-standing racial and economic segregation in Washington’s public schools, despite citywide school choice. For the same reason, undoing the preferences would be politically impossible.

Overall, only 27% of families choose to attend their neighborhood school. Those that do tend to be white and living in wealthy neighborhoods.

Additionally, approximately 50% of DC’s white families send their children to private schools.

So wealthy, white families use exclusive neighborhood schools and private schools to educate their children, but somehow it’s charter schools that serve low-income families that are the problem?

This makes no sense. At some point, people will realize this. I just worry about how long it might take.

 

Be curious about the strengths of flawed people

When I was younger I would focus on the weaknesses of successful people.

I would think: “this person is very successful, and it’s clear to me they are doing X wrong, so if I do X right I’ll be even more successful than they are.”

This is a wrongheaded way to think about things and it’s taken me years to adjust my mental model.

Now when I meet a successful person I think: “this person is very successful, and it’s clear to me they are doing X wrong, so how are they still so successful?”

This is a small shift in a mental model that has had big impact in how I think.

A few takeaways from having made this shift:

I find it very pleasant to spend more time thinking about why people are amazing rather than why people are flawed. This is not a reason in and of itself to think this way, but it’s a nice bonus.

It’s clear that being very very good at a few things that are aligned with one’s role can lead to tremendous output. Specialization rules the day.

It’s clear that being very very good at a few ways of thinking and behaviors can be lead to tremendous output even if coupled with major flaws in ways of thinking and behavior.

A common flaw I see in successful people is that they are just as stubborn about the things they know a lot about as they are about the things they know a little about. This is a major flaw that has little repercussions so long as they stay specialized. But it can blow up in their faces if they branch out. This is a major risk in philanthropy.

Anyways, I suggest you give this way of thinking a try. Anytime you meet someone who’s done something amazing, try to be curious about why they were able to accomplish amazing things despite all their flaws.

The low ROI of focusing narrowly on teacher quality

Eric A. Hanushek, Marc Piopiunik, and Simon Wiederhold just published a study on international teacher quality.

They looked at an international sample of teacher cognitive ability (as measured by OECD’s Program for the International Assessment of Adult Competencies) and student achievement (as measured by PISA scores).

They found a lot of variation of teacher cognitive quality across countries.

Teachers in Finland and Japan score very well. Teachers in Italy, Russia, and Israel score poorly.

After running regressions and controls, the authors estimate that a one standard deviation increase in teacher quality leads to a .15 standard deviation in student achievement.

To put a one standard deviation increase in context, a one standard deviation bump from the mean in IQ scores is about 15 points.

I can’t think of a realistic intervention that, at scale, will lead to that kind of increase in teacher cognitive ability across the entire United States.

For me, this is just one more study that validates the idea that institutional change is the best hope we have for increasing educational opportunity.

High-performing urban charter school sectors across the United States often achieve a ~.1 standard deviation effect over the course of three to four years.

I believe we can thoughtfully grow these non-profit public schools in a manner that maintains quality at scale. I also think we can empower amazing district leaders by letting them adopt non-profit governance for their schools.

I really can’t think of a way that we increase teacher cognitive ability by one standard deviation at scale.

In every city we work in, we do invest in recruiting and supporting great teachers. Teachers should be recruited hard, paid more, and supported better.

But I’m skeptical that focusing most of our energy on improving our teacher pool is the best way to help low-income kids.

Instead, I think we should focus much of our energy on creating and expanding great public institutions where teachers can work.

Another way to say it: I don’t blame our teachers for the dysfunction that plagues many schools in cities across the United States.

It’s their employers that should be the focus of our attention.

 

 

 

Back to the future: reflections on returning to New Orleans

I recently spent a week in New Orleans.

I lived in New Orleans for most of 1998-2015, save for a few years away at law school. Simply landing at the airport brings back a lot of emotions.

My friends in New Orleans were mostly my education colleagues. So catching-up with friends also means catching-up with amazing education and civic leaders.

______

My week in New Orleans was a shot in the arm.

In many cities it is becoming difficult to empower educators. In Los Angeles the school board just voted to ask the state to make it illegal to open new non-profit schools for the remainder of the year. Many hope to extend this to five years, if not forever.

In other cities advocates have thwarted creating easy to use online enrollment systems that can help families find great schools for their children.

These efforts are couched in the language of social justice. But their impact will not be just: educators will not be able to create, and parents will not be able to access, better public schools.

New Orleans stands apart from these cities. By next year, 100% of schools in the city will be governed by non-profit organizations. Each year, families can access the city’s online enrollment system to find a good fit for their children.

Writing in The New York Times, David Leonhardt  covered research showing large academic gains across the city, as well as positive trends in high school graduation, postsecondary enrollment, and postsecondary completion.

New Orleans citizens also have regained control of their schools. An elected board oversees the public education system. And the education leadership of the city is beginning to better reflect the students the schools serve. The school district is again black led. Patrick Dobard, the CEO of New Schools for New Orleans, is putting forth a great vision for the city.

______

As I work across the country, a lot of people are quick to tell me that the the education transformation in New Orleans could never happen in their city, nor would they want it to.

I’m beginning to wonder if I’m too conciliatory in my response. I always say that every city is different; that each city needs to blaze its own path. This is of course true and it is why I say it.

But I think from now on instead of my first response being “every city is different,” I’m going start with “why not?”

I want to be more direct because I think a part of the reason people dismiss New Orleans is that it relieves them of the burden of exploring if the best solution might be the solution they don’t want to deal with.

Yes, the New Orleans path might not be right for every city. But New Orleans is the only majority African-American student city in the nation to rank in the top ten for most academic growth between 2009 and 2015

If you’re a city leader trying to make public education better for low-income families, you should study New Orleans, not write it off.

And I would definitely not hold New Orleans up as a future to be avoided at all costs. Those advocating against non-profit schools claim that the private operation of public schools will ruin public education.

In New Orleans the public schools are operated by private non-profit organizations.

But public education has not been ruined. It has been revitalized and depoliticized.

Instead of worrying about the next top down mandate, educators can build enduring institutions that will serve students well for decades.

Instead of fretting about whether the next school board election will lead to strategic chaos, district leaders can thoughtfully evolve their oversight of public schools.

Instead of fearing that the only way to get into a good public school is to be able to afford an expensive house, families have much more equitable access to public schools in the city.

If this is the hell of privitization, what word should we use to describe the chaos of public education systems in Houston, Nashville, and so many other cities across the country?

_______

I don’t know if my returning to New Orleans was going back to our country’s future. The odds are surely against it.

And of course New Orleans has thousands of more problems to solve in its journey to create an amazing public school system, both in terms of improving its K12 system and expanding postsecondary options.

But what city is better equipped to solve its next wave of challenges? I can’t think of any.

If you’re feeling beat down about your city or country, I suggest a trip down to NOLA.

In so many ways, it will lift your spirits.

 

Julia Galef interviews me on the Rationally Speaking podcast

I just did a podcast with Julia Galef of Rationally Speaking.

You can listen here.

We cover: New Orleans, the evidence base behind urban charter schools, common critiques of charter schools, how to balance evidence and lived experience, what we do know and don’t know about what happens when charters expand, what we do know and don’t about what parents want…. and more.

Here is a teaser:

Juli: Right. Has anyone yet proposed a nudge system, where we try nudge parents to pick the schools that actually improve scores more?

Neerav: There are experiments being run right now about how you show information to parents in these systems, and figuring out what that means. Now, again, you want a balance. Do you know what the right thing to show is? Because if you’re nudging, that means you have an opinion on how parents should be making decisions.

Julia: Right.

Neerav: Versus, do you want to be a little more humble and say, “We’re just gonna show the information as neutrally as we can and let you know”? Those are pretty complicated decisions for policymakers, on how to design choice systems.

Julia is an excellent interviewer.

If you have 30 min at the gym, on a walk, or during your commute – give it a listen.

Being less racist and sexist may account for 25% of increased economic output in the United States over past 50 years

Two of the most important goals of a liberal society are to treat people fairly and to increase economic productivity.

Treating people fairly increases the chance that individuals and communities can  flourish.

Increasing economic productivity usually leads to better overall health, wealth, and happiness.

It’s possible for these two goals to be at odds.

When it comes to racism and sexism in America, the goals don’t seem at odds.

This recent paper found that the United States saw major economic gains by being less racist towards African-Americans and less sexist toward woman.

It used to be the case that most leading professionals were white men. This is less true today.

The authors of the paper explore what we can draw from this change:

 In 1960, 94 percent of doctors and lawyers were white men. By 2010, the fraction was just 62 percent. Similar changes in other highly-skilled occupations have occurred throughout the U.S. economy during the last fifty years. Given the innate talent for these professions has unlikely changed differentially over time across groups, the change in the occupational distribution since 1960 suggests that a substantial pool of innately talented blacks and women in 1960 were not pursuing their comparative advantage. We examine the effect on aggregate productivity of the remarkable convergence in the occupational distribution between 1960 and 2010 through the prism of a Roy model. About one-quarter of growth in aggregate output per person over this period can be explained by the improved allocation of talent.

The paper details some powerful anecdotes to make their case.

For example, Supreme Court Justice Sandra Day O’Connor graduated from Stanford Law School in 1952; despite being ranked third in her class, the only private sector job she could get after graduating was as a legal secretary.

In teasing out what specific changes made things better, the authors consider three possibilities:

Changes in preferences; ie, there wasn’t much discrimination; it’s just that African-Americans and woman increasingly wanted more professional careers post 1960.

Reduction in discrimination in preparing and entering a field: ie, there were barriers that prevented talented people from going to school and entering specific fields.

Reduction in discrimination once in the field; ie, there were a bunch of glass ceilings that prevented advancement.

The authors find that declining barriers to entry into a field explain 24 percent of growth in U.S. GDP per person between 1960 and 2010.

Declining labor market discrimination once in the field explains 6 percent of growth.

Changing  preferences across groups explain little of U.S. growth during this time period.

So it wasn’t that after 1960 a bunch of African-American and women suddenly wanted to become doctors.

Rather, things were bad because a lot of white men prevented African-American and women from becoming doctors.

All the usual caveats apply: it’s one paper on a topic that can probably never be fully understood through pure academic research.

But it’s a useful reminder that being less racist and sexist has both individual rights and economic benefits.

I think increasing educational opportunity, in this sense, is akin to reducing discrimination.

If educational opportunity is further increased in our country, we’re likely to see major gains on both moral and economic fronts.