Category Archives: Uncategorized

Did a federal grant to turnaround failing schools in New Orleans and Tennessee work?

Back when I worked at New Schools for New Orleans, we applied for a $30m federal grant to turnaround failing schools in New Orleans and scale the model to Tennessee.

CREDO just came out with a research study on our efforts. Their findings, and my analysis, are below.

The New Schools Were Much Better than the Ones They Replaced 

Here’s what CREDO found when they compared the schools we created to the schools we replaced:

In New Orleans, we replaced schools (“closing schools”) that were at 26th percentile in the state with new schools (“CRM schools”) that performed at the ~33rd percentile in the state at the end of the study.

In Tennessee, schools went from the ~17th percentile to the ~23rd percentile by the end of the study.

To quote the CREDO report: “the CRM schools in both New Orleans and Tennessee showed significantly higher academic growth compared to the Closing schools they replaced.”

Translated into days of learning, these are large effects: “Closing school students experience 63 fewer days of learning in reading and 86 fewer days of learning in math when compared to students in non-CRM schools… students in CRM schools make comparable academic growth to non-CRM students.”

The New Schools Performed About the Same as Other Schools in the City

When CREDO compared the new schools to other existing schools (rather than the failing schools they replaced), they found no statistically significant effects:

Screen Shot 2018-02-15 at 12.07.57 PM.png

In other words, the new schools that replaced the failing schools performed no better or worse than other existing schools in the city.

On one hand, this is disappointing. Our most ambitious targets included having the new schools be amongst the highest performing schools in the city.

On the other hand, this is still a major improvement: the new schools replaced failing schools and ended up achieving at the same level of most other schools in the city.

Building a System that Keeps Getting Better 

Replacing failing schools with new schools is a process, not a one-time intervention.

Ideally, a subset of the schools you created will do really well, and then, overtime, these schools will continue to grow. The ones that don’t do well will not be supported to do additional turnarounds.

Over the long-haul, gradually increasing the number and scale of high-quality school operators is more important than the average effect of the first wave of replacements.

Here’s what CREDO found across the new schools when they compared them to existing schools:

Screen Shot 2018-02-15 at 12.11.36 PM.pngIn New Orleans, 50% of the new schools had positive effects in both Math and Reading. This is really positive: half of our turnaround schools in New Orleans achieved significantly better results than existing schools across the city.

In Tennessee, only one school had positive effects in both Math and Reading, though a few other schools had positive effects in only reading.

This makes me optimistic that the school operator base in New Orleans will continue to have the capacity to replace more failing schools over time.

The early results in Tennessee are a bit more worrying on the operator quality front, and the next few years will be extremely important in ensuring that a healthy operator base emerges.

Lastly: CREDO found that replacing failing schools with fresh start schools (that opened one grade at a time) had a higher success rate than whole school turnarounds. My takeaway here is that you need a mature operator base to do a lot of whole school turnarounds, and no city had enough capacity to really do whole school at scale. In hindsight, we should have done more fresh starts and less whole school turnarounds.

Was the Effort a Success?

At the outset of the project, I remember debating with our research partners at CREDO about how to set-up the evaluation.

I argued that we should ultimately be judged on whether or not the new schools we created were better than the failing schools we replaced.

I didn’t think we should be primarily judged on whether or not the new schools were better than other existing schools that weren’t failing.

Yes, we did include language in the grant application that had goals of schools performing much better than existing schools. And as we executed the project we tried to pick school operators that we thought could deliver top tier results. Our highest aspirations weren’t met. This is disappointing, but it does not mean the project was a failure.

Rather, I consider the project to be a positive step forward in improving public education in these cities.

Making Things Better

The result of the project strikes at the heart of what’s so difficult about education reform: our aspirations for our most at-risk children are incredibly high, but making progress in creating better educational opportunities is very difficult.

In roughly a five year period, we replaced failing schools with new schools that were on average 7 percentile points higher in state performance, which translates to an extra 60-90 days of learning per year.

If the process of opening and replacement continues, what is a modest success right now may eventually become a great success.

I hope that this occurs and that New Orleans continues on its impressive track record of increasing student achievement. As a reminder, the federal grant was just one piece of an overall effort that has radically reduced failing schools in the city:

Screen Shot 2018-02-20 at 5.47.30 PM

SaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSave

To renew or to relinquish? That is the question for New York City

DB

Sometimes the research gods shine down upon us. In this case, they gave us two NYC studies, each on a different strategy, within a few months of each other.

The studies tell us much about what’s happening in New York City. They also illuminate the rational for why I believe in relinquishment.

What to do with struggling schools?

Some people believe you should give struggling schools more support. Others believe you  should either replace them with charter schools or close them.

Bill de Blasio is spending $582 million on 78 Renewal Schools, a model built upon the support strategy.

Charter school supporters tend to support the replacement strategy.

Early evidence on Renewal Schools

Marcus Winters of the Manhattan Institute authored a recent study that found that Renewal Schools have seen around a .1 standard deviation increase in math and ELA scores. This is a healthy bump and it’s great to see modestly positive effects.

Winters also notes, however, that Mike Bloomberg’s closure strategy achieved about the same results at little to no cost.

Recent CREDO study on NYC charter schools

CREDO’s research found that charters in NYC achieved about .1 standard deviation effects as well.

And while CREDO did not put a price tag on the charter reforms, most estimates I’ve seen have charters coming in at a lower per school cost than the ~$8 million per Renewal School.

Is it worth it?

It can be painful to close schools and grow charter schools. This process undoubtedly causes some disruption to families and educators.

If I believed the Renewal Schools would continue to improve, I’d be open to the idea that they were worth it. Perhaps spending a few hundred million more dollars is worth the cost to avoid significant disruption to communities.

The Renewal Schools will not keep improving

Sometimes these blog posts write themselves.

Elizabeth Harris ended her NY Times piece on Renewal Schools with the following quotation:

“The thing we’re nervous about is losing any of these resources,” said Mr. Bradley of Renaissance School of the Arts. “Renewal is the bomb. I want to be renewed forever.”

I’m sure receiving $5 to $10 million to improve your school is the bomb.

But of course these resources will not last.

Additionally, because these schools are subject to the whims of changes in district policies, the strategies they implemented will likely fall out of the favor of a future administration.

Charter schools, on the other hand, receive most of the philanthropic support for start-up costs, not continuing costs.

Additionally, charter schools are governed by non-profit boards which can maintain strategic consistency through self-perpetuating boards.

To renew or to relinquish? In the long-run, relinquishment will deliver sustained gains for students. Renewal will not.

Philanthropy is a low accountability sector (and what to do about it)

In terms of accountability to stakeholders for continued survival, my ranking of sectors is as follows (from most accountable to least accountable):

  1. Politics: the people you serve vote on your fitness every 2-4 years.
  2. For-profit: the people you serve must give you their money.
  3. Non-profit: the people who care about your issue must give you their money.
  4. Philanthropy: people must be willing to accept your money.

Depending on the specific office / company / non-profit, these rankings might shift a bit, but at the median I think they hold true.

If you disagree, think about this question: which sector is the most ruthless? This is likely where this is the greatest accountability.

___

At the structural level, philanthropy is an extremely unaccountable sector. All you have to do to stay in business is have someone be willing to accept your money!

This is not the fault of anyone involved, it’s just the nature of the beast.

I think a few things follow from this structural condition:

Culture matters: Philanthropists, like all humans, care about what people (especially their peers), think of them. Given that direct accountability is not as strong of a lever in philanthropy, culture pressures will play a bigger role. I think the rise in status of evidence based giving has been a positive development on this front.

Governance matters: Certain organizational structures, such as foundations whose governance outlive their founders by 100+ years, should probably limited, as this further decreases accountability in an already low accountability sector.

Rigorous exits matter: In most cases, philanthropy can’t cover the permanent costs of its subsidies, and over time government will generally pick up the tab of continued social programs. As such, it’s government that controls when philanthropy can exit, and the more rigorous government is in its spending, the more accountable philanthropy will be for producing results.

___

Again, I don’t think philanthropy is to blame for operating in it a low accountability sector.

It is what it is.

But citizens and governments can increase accountability, and they should do so when feasible.

 

 

Personalized learning is a transformative idea without a transformative technology

Screen Shot 2017-07-11 at 4.59.53 PM.png

I just got back from vacation, which was a great time to read the The Three Body Problem science fiction trilogy, a wonderful series that revolves around the protoganist using first principles thinking to negotiate with an alien species.

Upon return, I read this Rand report on personalized learning, which was funded by the Gates foundation. The report covers a small set of schools in the early years of implementation, so best not to draw too firm of conclusions.

The report found:

  • Charters that adopted personalized learning strategies saw a +.1 effect in math and no statistically significant in reading.
  • District schools (very small N) saw no achievement gains.
  • Charter schools implemented personalized learning strategies with more operational fidelity.

Perhaps most interestingly, the authors noted:

In this theoretical conception, schools that are high implementers of PL [personalized learning]  approaches would look very different from more traditional schools. In practice, although there were some differences between the NGLC schools and the national sample, we found that schools in our study were implementing PL approaches to a varying degree, with none of the schools looking as radically different from traditional schools as theory might predict.

So in this sample, charters outperform traditional schools (thought by a lesser margin than urban charters as a whole outperform traditional schools); charters execute better; and the schools themselves don’t look radically different than traditional schools.

Hence the title of this post: personalized learning is a transformative idea without a transformative technology.

Without a technological breakthrough, the current personalized learning efforts will, at best, lead to modest improvements on the execution of common place ideas (using data to drive instruction, executing leveled small group instruction, investing children in goals, etc.). School will look the same and be a little more effective and pleasant for all involved.

This is fine and the world is in many ways built on modest improvements.

But for personalized learning to live up to its hype (as well as to its philanthropic investment), it will need a technological breakthrough.

Instructional platforms might be the first breakthrough, but even here I think the primary effects will be more around scaling great school models and content rather than deep personalization.

The crux of the issue is this: computers are simply not as good as humans in coaching students through instructional problems.

Your average person off the street remains a more effective grade school tutor than the most powerful computer in the world.

Until this changes, personalized learning will never realize its promise. The problem is one of technology, not practice.

How to work less hours and outperform your IQ

I. If you’re going to praise something, praise people who outperform their IQs

Everyone has an IQ. Just like everyone has a personality, a height, and an eye color.

IQ has definitional and measurement problems that make it more like personality than height, but, as with personality, research indicates that IQ is a predictive trait. People with higher IQs tent to have better life outcomes.

In our culture, we both fetishize high IQs and stigmatize low IQs. I wish this were not the case. Nobody selects their IQ from the IQ tree. It’s handed to them and then shaped by their environment (and nobody selects their environment from the environment tree).

To the extent you believe in free will, however, it is possible to proactively make decisions that can help you outperform your IQ.

If we are going to obsess over anything about IQs, we should obsess about people who outperform their IQs.

II. There are people out there with higher IQs than you 

Unless your IQ is extremely, extremely high, you will at some point be competing against, or working for, people who have higher IQs than you do.

This happens to me all the time. My guess is that both my current employers, as well as my last employer, have higher IQs than I do.

And yet I think I can perform my job better than they could.

Why?

Specialization.

III. Specialization 

I have a pretty specialized skill set. I know how to help cities transition from traditional public school systems to systems that rely more heavily on the non-profit operation of schools.

While the high level strategy of these types of transitions is not rocket science, the details are (at least it feels like it to me!).

It’s very difficult to build plans, marshal a coalition, and execute programmatic shifts in school development, talent, government, and advocacy – as well as develop a policy regime that fits within a state’s constitutional parameters.

I spent eight years doing this in New Orleans; one year doing this a consultant; and about two years doing this in philanthropy.

Now, when I come across a problem in a city, I’m bringing a decade of specialized knowledge and pattern recognition to the issue.

Even if you have a higher IQ than me, I’ll probably get the answer right more often (and quicker) than you will.

IV. You can work less too 

The more specialized your knowledge is, the less likely other people are to also possess this knowledge. This means you have less competitors. If you so desire, this should allow you to work less and still be valuable to your employer.

If you can get the right answer in less time than most other people, you’ll have some spare hours to play with. You could fill these hours with more work (and accomplish more great outcomes), or you could spend more time on other things, such as family and friends.

The key is that the choice is yours, not theirs.

Specialization is a marvelous thing!

Is Tyler Cowen right to be stubbornly attached to economic growth?

Screen Shot 2017-05-02 at 3.07.56 PM.png

Tyler Cowen just published Stubborn Attachments in e-book form.

With the book, you get deep exposure to Tyler’s mental model of the world.

As someone who attempts to collect mental models for subsequent application, I came away smarter for having read it.

Below I’ll try and restate Tyler’s main practical arguments (skipping over some of the philosophy) and then end with some questions and concerns.

Tyler’s Primary Thesis: Sustainable Economic Growth Should Guide Our Policy Making

Tyler argues that sustainable economic growth should guide much of our policy making because, over centuries, the gains from economic growth dwarf the gains of just about all other considerations.

He defines sustainable economic growth as gains in “wealth plus,” which “includes traditional measures of economic value, as would be found in gdp statistics, but also measures of leisure time, household production, and environmental amenities, as summed up in a relevant measure of wealth.”

Over let’s say a 5,000 years, consider two possibilities:

No economic growth: We stay the same. Poverty is all around us. People are starving. Billions of people barely get by. Life is hard for so many people.

Modest economic growth: We grow at 2% a year. In 5,000 years, future humans (assuming no population explosion) are crazy wealthy, have access to amazing technology, and even the poorest humans view the richest people of today as paupers. No one is losing sleep trying to make ends meet. Every person has every material need taken care of, plus the wealth to pursue their passions and interests.

In other words, the difference between poverty as we know it and each of us feeling like Bill Gates was materially poor is determined by one thing: sustainable economic growth.

So when we’re making policy decisions we should keep our eye on the prize: sustainable economic growth is what will allow us to flourish.

Of course, in order for you to accept this argument you have to be ok with two assumptions: future lives matter a lot (so we should focus on pro-growth policies that benefit future people), and, increased wealth leads to increased well-being (Tyler runs through the research).

While both of these assumptions are contentious, I believe both to be correct.

Other Considerations: Individual Rights, the Environment, and Social Stability 

Tyler’s provides guardrails and depth to the concept of sustainable economic growth.

First, Tyler argues that we shouldn’t harm individuals to achieve economic growth; i.e., no mass murder even if it helps us squeak out an additional point of growth.

Second, he argues that we shouldn’t destroy the planet in order to achieve economic growth; i.e., what’s the point of focusing on the future benefits of economic growth if there’s no place to live.

Third, he argues that societal stability is an important part of sustainable growth. So policies that might not seem purely connected to growth (such as welfare state) should be considered as part of an effort to maintain the continuity of our civilization.

All of these seem correct to me, though as I’ll argue at the end, I wonder if this last point (societal continuity) is actually what we should be most focused on.

Major Shifts in How We Think and Feel: Redistribute for Growth Only 

Perhaps the biggest practical implication of Tyler’s thesis is that, according to him, “we should redistribute only up to the point which maximizes the rate of sustainable economic growth.”

In other words, we should only give a starving person food if this can be tied to maximizing the rate of sustainable economic growth, not because we feel that the starving person deserves something to eat.

While this rationale logically follows from his assumptions, this line of thinking departs greatly from current values. Most of us justify the welfare state in terms of our care for living and breathing humans, not for future humans.

Tyler’s argument for redistribution is a monumental shift from our current moral calculus.

That being said, it’s possible the our current lack of opportunity is reducing sustainable economic growth, so for now Tyler’s thesis may actually call for an increase in certain types of charity and transfers.

But I predict that most people will find his logic emotionally unappealing.

Anti-Fragility > Sustainable Economic Growth 

As much as I appreciated Tyler’s argument, I think I disagree, though I’m not sure both because I’m still grappling with the text.

Ultimately I believe that “anti-fragility,” rather than “sustainable economic growth” should be the language we use to guide our policy decisions, and I think, though I’m not sure, that this puts me at odds with Tyler.

Consider this hypothetical: would you rather have rather have a million years of infinitesimal  economic decline (.0000000000000000000000000000000000000000000001% a year) and then human extinction or a thousand years of 5% economic growth and then human extinction.

I think almost everyone (including Tyler?) would choose the former.

While economic growth will likely make our society less fragile, this is not inevitable, and we need to incorporate this understanding in choosing guiding principles.

This thought experiment points to the idea that it’s actually the sustainability of a healthy society, and not the sustainability of economic growth, that we desire.

Most of us would accept small amounts of negative economic growth for hundreds of thousands of years of additional existence.

Similarly, I don’t think most of us would take high economic growth and a population explosion that had most people living at substance levels, even if this led to a sustainable society with high amounts of leisure.

So I would argue:

  • Sustainable economic growth is a sub-goal of anti-fragility; it should not be the primary goal in and of itself.
  • Economic growth should be considered in both societal and per-capita terms.
  • I would maintain Tyler’s support for individual rights.
  • I would make environmental sustainability, like economic growth, a sub-goal of anti-fragility.

In sum: anti-fragility and individual rights do all the work. Everything else follows from these two principles.

Thanks + Do the Work Yourself

All that being said, economic growth is clearly one of the primary ways to make our civilization less fragile, so in this sense I agree with many of the practical implications of Tyler’s worldview.

And Tyler’s mental model will be present in my head next time I’m trying to unpack an efficiency vs. equity argument on a specific policy.

Lastly, I do think that everyone should go through the process Tyler went through; ultimately, each of us should understand the philosophical underpinnings of our policy preferences.

I thank Tyler for his contribution.

What will be the stable charter school and teacher union equilibrium?

It appears Kentucky may pass a charter school law.

News recently broke that Noble charter schools may become unionized.

Where is this all heading?

The Forces at Work: Charter Market Share will Continue to Rise 

Charter market share will continue to rise because (a) 40+ states allow charters and this number is increasing and (b) once a charter school is open it is difficult to close.

Yes, policies like charter school caps and moratoriums may slow charter growth down, but it will be incredibly difficult for union leaders to fully stop charters from growing in a world where charters are legal in 90% of states. Rolling back laws in this many states is unlikely.

Charter market share will continue to grow.

The Forces at Work: Unions Organize Where the Teachers Are 

A simple consequence of rising charter market share is a rise in the number of teachers who work at charter schools.

Unions received dues (and power) from having as many teachers as possible enrolled as members.

The more charter schools there are, the more it will be be worthwhile for unions to attempt to organize these teachers, for both financial and political purposes.

The Future of Union Incentives 

So we’re basically going to be in multi-round unionization game between unions and charter schools.

Most importantly, this new game (unionizing a lot of charters) will have very different incentives than the old game (unionizing a local monopoly).

Under the old game, the unions paid a relatively small price for being obstructionist: with only one school operator in town (the district), they didn’t have to worry too much about how their actions affected the performance or reputation of that school operator.

In the new game, the union has a new incentive structure: if they are overly obstructionist, they will reduce their ability  to unionize more charters in the future; however, if they are not obstructionist enough, unionizing a charter won’t slow down the growth of that operator (which is in the unions interest, as they want the district to last as long as possible, as it’s the easiest entity to unionize).

The unions thus want to add value to unionized teachers and at the same time hobble charter school growth.

The ideal play for unions is to: (a) unionize charter schools  (b) demonstrate value to their unionized charter teachers so they can unionize future schools (c) slow down the enrollment growth of unionized charter schools  (d) avoid having unionized charter schools go down in flames so they can unionize future schools.

My Guess

The conditions for charter school unionization are favorable compared to other sectors of the economy: you have a long history of unionization, strong and well-financed existing unions, an inability to outsource to other states or countries, and weak accountability for performance (compared to the financial market).

All of this bodes well for unionization.

But unionizing charters will require unions to moderate their behavior and become less antagonist with management, as they will be working in a market of providers rather than a district monopoly. This will require significant change in their leadership and culture. This is hard to do.

So where are we heading?

One interesting comparison group is nurses, which generally operate in a non-profit, physically anchored, and heavily regulated environment.

Overall, about 20% of nurses belong to a union.

And while most industries have seen declining union membership, nursing union membership has risen over the past 15 years.

nurse-union-fever

I think we will see the same thing happen in the charter sector.

That being said, given that unions have to independently organize each charter organization (which is very expensive), and that their success will be predicated on cultural change, I don’t think we’ll quickly move into a world of 100% charter unionization.

But will we see 20% of charter school teachers (compared to ~7% right now) organized in unions over the next decade or so?

I think so.