Category Archives: Economics

New Orleans, the New Yorker, and the perils of flawed comparisons

1 Reply

Screen Shot 2018-08-10 at 5.49.26 PM

Gary Sernovitz just wrote an interesting piece in the New Yorker on the New Orleans public school system. He argues that the New Orleans public school system is designed around free market principles that are, at times, being poorly applied to the public sector. Gary draws lessons from his time serving on the board of a charter school that eventually lost its charter for financial reasons.

Before considering his arguments, I just want to give a thank you to Gary. He joined a charter school board and devoted a lot time trying to make public education better. It’s great to see people with his commitment and intelligence serving on charter boards. I hope more people follow his lead.

Gary argues that the New Orleans public education system is designed around “the engines of the free market – autonomy, competition, and customer choice” but that these design principles are currently inadequate to meet the aims of public education in New Orleans.

Gary points to three main problems with the New Orleans system: rewards, incentives, and start-up capital.

Rewards

Gary argues that New Orleans’ schools demand crazy work hours but offer mediocre compensation. Unlike for-profit founders, there is no dream of a financial exit for charter founders or their teams.

I don’t disagree with these facts. If you want to get rich, starting a non-profit charter school in New Orleans is not the way to go.

But this is the problem with Gary’s premise: the New Orleans public school system was never designed to mimic all parts of the free market. The goal of working in a charter school is not to get rich; it’s to do good while earning enough to live a middle or upper middle class lifestyle.

The reward model provides a different set of rewards for different kinds of educators.

Many teachers teach in charter schools for 3-5 years, work long hours, and are rewarded with the meaning that comes with knowing you helped others. They then go onto other things.

A minority of teachers find that teaching is their lifelong calling. Their hours tend to go down overtime as their mastery of teaching goes up. The most skilled teachers in New Orleans can achieve in 50 hours a week what it takes a novice to achieve in 70 hours a week. Their rewards come from the joy of doing good work, building meaningful relationships with children, and earning a stable middle class income.

Another set of teachers move into administrative roles. They tend to spend another 5-10 years working in leadership positions. Their rewards come from the challenges of leadership, seeing impact at a larger scale, and earning an upper middle class income.

Yes, current model does rely on younger teachers, who work more hours, and leave the classroom more frequently than their traditional peers. But this model is delivering better results for children than the old talent model. And it has been doing so for over a decade, which leads me to believe that the talent model is sustainable.

That being said, I’m open to the idea the current model is not optimal. I can think of two potential improvements: raising taxes to increase educator salaries, or simply encouraging charters to be for-profits so there can actually be financially exits and equity based compensation. But New Orleans is a poor city in a poor state, so I’m skeptical that New Orleans will be able to raise salaries by large amounts. As for for-profit charters, while a reasonable idea in theory, their results to date have been underwhelming, so I’m not holding my breath here either.

Rather, I think New Orleans has organically evolved to the best talent model under very imperfect conditions.

Incentives

Gary’s main criticism of the New Orleans public school system is that it does not fully fund the costs for students with special needs. In market terms, it gets the price wrong.

Gary sat on the board of Cypress Academy, which intentionally enrolled a lot of students with special needs. These students cost more money to serve well.

The New Orleans per-pupil revenue system is designed with this reality in mind: I believe New Orleans has the most weighted per-pupil system in the country. Schools receive up to 3x of the regular per-pupil to serves students with severe special needs.

Because of this model, numerous schools in the city are able to serve a lot of students with special needs. Many networks have even developed specialized programs for high needs students.

I’m open to the idea that the weights need to be further increased. But the Cypress financial model should have been built around the existing financial regulatory regime. It is well known to all charter operators in the city, and Cypress should not have opened if they did not have a viable financial model to serve the students they wanted to serve.

If Cypress thought the per-pupil funding system was wrong, it should have advocated for policy change before opening its doors. Instead, it opened with an unsustainable model. This was a mistake.

Start-up Capital

Gary argues that there is not enough start-up money to help a new charter school get to scale.

I don’t think this is true.

When I worked at New Schools for New Orleans, we helped 10+ new charter school start-ups open schools, and none failed for financial reasons. Rather, all of them received enough funds (usually $500K to $1m in philanthropy) to cover their operations until they reached scale.

My understanding is that Cypress Academy received start-up grants in the range of other successful start-up charter schools in New Orleans, such as Bricolage Academy.

And, again, none of the financial realities should have been a surprise to the founders of Cypress. If they knew they were going to run a deficit over the first few years, they should not have opened unless they were fairly certain they could raise the necessary philanthropy to cover this gap.

___

Dozens of new charter schools have opened successfully over the past decade. These schools operate sustainable talent models, serve students with special needs, and scaled with the support of philanthropy.

Cypress Academy failed for reasons that seem to be mostly predictable. The balance of the fault appears to be with the school, not with system.

Lastly, it’s worth emphasizing that the New Orleans public school system is not designed to be a free market. It’s a publicly funded system operated by non-profit organizations.

Yes, it has more market mechanisms than a traditional government operated system, but it’s so far from being a free market that most comparisons to free markets obscure more than they illuminate.

Education philanthropists should not take advice from Larry Summers

4 Replies

Screen Shot 2017-09-20 at 7.28.42 PM.png

Tyler Cowen just Larry Summers.

In a blog post about the interview, Tyler wrote: “if you think you know someone who is very smart, Larry is almost certainly smarter.”

This may be the case when it comes to economics. While I’m in no position to evaluate his economic policy claims, I found Summers to be reflective, curious, and thoughtful. He seems like the kind of person I would enjoy working with.

But Summers also discussed education philanthropy, and I came away with a strong belief that I almost certainly smarter than Summers on this subject.

I don’t say this because I for sure know that I’m right and Summers is wrong; rather, I say this because I have a firmer grasp of the research, more hands on experience, and a clearer strategic vision for scalable and sustainable change.

Given that Summers likely has a good 20-30 IQ points on me, and that he has risen to the top of an extremely competitive field, the fact that I’m likely smarter than Summers in this area is a testament to the powers of specialization and the domain specific nature of knowledge.

How should you spend a $100 million?

In the interview, Tyler asked Summers how he would advise a philanthropist in St. Louis who wanted to give away a $100 million to help her city. After admitting the he knew little about St. Louis, Summers answered the question more generally, and said that he would focus on public education.

As it happens, my job is to advise philanthropists who want to improve public education. Currently, our team manages the philanthropic giving for Reed Hastings and the Laura and John Arnold Foundation.

Given my day job, I was curious to hear how Summers would respond.

Moreover, while he may not remember, Summers was once asked about relinquishment during an interview on education reform (I was in attendance and vividly remember him saying “Ah, relinquishment…”), and I was curious to see if my ideas had impacted him at all.

They have not!

Summer’s advice: avoid charter schools and work outside the system

In the interview, Summers gave two pieces of advice to education philanthropists:

Avoid charter schools: Too many philanthropists set-up charter schools that cream students, pay teachers salaries that are not sustainable on the public dollar, and then ultimately cannibalize the traditional system of good students, good teachers, and public funds.
Avoid the K12 system: Instead of trying to tackle the core K-12 system, it’s better to fund efforts that work around the system, such as after school or summer school.

I think both of these points are wrong.

Summers ignores a large evidence base on charter schools

I recently summarized the evidence on charters schools on this blog. Summers ignores most of this research:

Achievement: Urban charter schools outperform traditional public schools, posting annual effects of .05-.1 standard deviations. This holds true with both quasi-experimental designs (where researchers try to control for student selection) and experimental designs (where student selection is randomized). Charters are not achieving their impacts because of student creaming.

Funding: Charter schools, on average, receive much less funding than traditional schools. As I previously wrote about, in numerous cities where charter schools receive less money, they still outperform the traditional system.

Teacher Pay: Nationally, traditional school teachers have higher average salaries than charter school teachers. And while some of this is due to the effect that charters hire younger teachers, I have seen no research that indicates that, at scale, charters are picking off the best teachers by offering them unsustainable salaries.

Impact on Traditional Schools: Lastly, most research shows that charter schools have positive or neutral effects on traditional school achievement. Moreover, cities that have improved their educational systems over the past decade have often seen rising charter school enrollment during the same period. Washington D.C. and Denver stand out as primary examples of cities where all schools got better as charter schools expanded.

All boats rising – and not cannibalism – is the norm.

It appears that Summers is reasoning from anecdote rather research.

I am sure there are some charter organizations that cream students and spend way above the public dollar (I can think of a few!), but these are outliers.

At scale, urban charter schools achieve more and spend less than traditional public schools.

Working outside the system is low impact and not leveraged with existing public funds

Summer’s second piece of advice – work outside the system rather than fix the system itself – is also flawed.

Yes, fixing the system is hard. But kids spend a lot of time in the system. It will be very difficult to improve public education if you ignore what happens to students from 8 AM to 3 PM for 13 years.

Moreover, to the extent that a philanthropist funds an outside the system intervention that works, the only way to scale the intervention is with more philanthropy or increased public revenues. There is no leverage with existing public dollars.

While I am not against raising additional public revenue for things that work, I think we should spend most of our energy improving the effectiveness of the dollars we already spend, especially given that systems level K12 interventions (like urban charter schools), are achieving success at scale.

If there was no evidence that the system could be fixed, I would tend to agree with Summers. But as more and more cities breakthrough and achieve citywide gains, the logic of working mostly outside the system is increasingly flawed. The one exception I’d make to this claim is pre-school, which has a reasonably strong evidence base and is increasingly funded with public dollars.

If you are a philanthropist who wants to improve public education in your city, please contact me

In the event that Tyler’s question was not hypothetical in nature, and that there is a philanthropist in St. Louis who wants to donate a $100 million, I do hope she contacts me (neeravkingsland at gmail) rather than takes Summers’ advice.

I am a firm believer that philanthropy well spent can forever positively alter the trajectory of a city’s public educational system.

And while those of us advocating for systems level change still have much to prove, we now have numerous examples of cities achieving citywide improvements for their most at-risk students. Philanthropists should double down on their successes, evolve the model based on local conditions, and continue to fund further research so we can keep on learning.

Forever Unequal, Immobile, and Politically Divided? Facing Brutal 500 Year Trends

1 Reply

Much of modern philanthropy focuses on reducing inequality, increasing economic mobility, and increasing the efficacy of government.

Three recent books, each in their own way, make the case that philanthropy will likely fail.

Forever Unequal: Inequality Persists Save for Massive Wars, Plagues, State Collapse

In , Walter Scheidel argues that inequality generally increases over time unless something very awful happens: massively mobilized warfare, societal upending revolutions, plagues, or state collapses.

In short: since the advent of farming, rising inequality has been the default state of humanity across almost all cultures and economic systems.

See below for a history of European inequality. Inequality has always risen save for the Fall of the Roman Empire, the plague, the Black Death, and WWI/WWII.

screen-shot-2017-02-10-at-11-10-21-am

Scheidel marshalls data sets that support this argument in societies across the world.

His final take: while it’s possible that we can inequality through policy and social programs, it’s unlikely.

Yes, individual countries can tweak inequality at the margins, but since the invention of farming, policy has never been able trump long-term immutable trends of increased inequality.

Forever Immobile: The Persistence of Family Status

In , Gregory Clarke utilizes a novel technique – tracking the status of last names over time – to solve many previous problems of economic mobility research, which usually only tracked economic shift of 1-2 generations.

Clarke’s method allows him to avoid the noise of only looking at short time horizons.

If a rich person’s son becomes a poet, it might appear that the family was downwardly mobile. However, if the poet’s daughter then becomes a CEO, the downwardly mobile trend is erased – and so on.

Clarke’s main argument is that, over multiple generations, there’s much less mobility than we thought.

Clarke’s results are stunning: the previous literature estimated intergenerational earnings elasticity to be around ~.3; Clarke’s data raises this estimate to ~.8.

screen-shot-2017-02-10-at-11-15-45-am

Under Clarke’s estimate, family advantages don’t disappear over two or three generations, but ten to fifteen generations.

Forever Divided: The Long Hold of Original Immigration Patterns

In , David Fischer argues that you can trace many of our country’s current conditions to long-ago immigration patterns from Europe (note: I have not read the book yet, and am largely relying on Scott Alexander’s review).

Fischer tracks the migrations of the Puritans, Cavaliers, Quakers, and Scotch-Irish – and shows how current inequities and culture can in many ways be tied to these 400-500 year old immigrations patterns.

In summarizing the book, Scott Alexander makes a few observations:

If this is true, I think it paints a very pessimistic world-view. The “iceberg model” of culture argues that apart from the surface cultural features we all recognize like language, clothing, and food, there are deeper levels of culture that determine the features and institutions of a people: whether they are progressive or traditional, peaceful or warlike, mercantile or self-contained.

And:

If America is best explained as a Puritan-Quaker culture locked in a death-match with a Cavalier-Borderer culture, with all of the appeals to freedom and equality and order and justice being just so many epiphenomena – well, I’m not sure what to do with that information. Push it under the rug? Say “Well, my culture is better, so I intend to do as good a job dominating yours as possible?” Agree that We Are Very Different Yet In The End All The Same And So Must Seek Common Ground? Start researching genetic engineering? Maybe secede? I’m not a Trump fan much more than I’m an Osama bin Laden fan; if somehow Osama ended up being elected President, should I start thinking “Maybe that time we made a country that was 49% people like me and 51% members of the Taliban – maybe that was a bad idea“.

Many have argued that the post-colonial country formation process led to unworkable patchworks of different cultures be thrown into single countries.

Perhaps this is true of the United States as well.

Will This Time Be Different?

On one hand, all of the above makes me incredibly gloomy about our prospects of evolving our society into a more equal, mobile, and better governed nation.

On the other hand, the sample size is small: humans have only had post hunter and gather economies for relatively small time frame, and our current institutions and technologies are very different than those of a few hundred years ago.

Moreover, there’s one place we have improved things: we’re incredibly more productive and wealth than we used to be.

So perhaps what we need is the equivalent of the industrial revolution but for inequality, mobility, and political culture.

But, at the very least, baseline predictions should keep us sober: it will take a radical departure from historical trends to change the trajectory of our nation.

What helps poor children more: increasing the EITC or increasing educational funding?

1 Reply

screen-shot-2017-01-08-at-9-19-55-pm

*Note: I’m still working through all this research. If I’ve made a mistake, let me know!

In a previous post, I argued that after a certain expenditure level, call it $15K a child, my guess is that families would be better off with additional funds as cash transfers. So a district that spends $20K a student (D.C, Newark, New York, etc.) would best serve families by spending $15K per student and sending $10K in cash to a two child home.

Here’s a similar policy question: if taxpayers are willing to spend extra money to help poor children, should they increase the earned income tax credit (giving working low-income parents increased money) or raise educational funding?

Last month, Kevin Carey and Elizabeth Harris wrote a NYT column summarizing the most recent research on increasing education funding. Their conclusion: money probably does increase test scores.

Unfortunately, they did not the review the research on what happens when you give similar amounts of money to families via other transfers, such as the EITC.

The Effects are Within the Same Range

While none of the research is an exact science, research on the EITC (see a summary here) finds that children under the age of 12 see increases of ~.06-.1 SD per $1,000 increase in the EITC (cumulative full schooling impact).

Rothstein finds that low income districts increase their performance by ~.1 SD per $1K increase in funding (cumulative 10 year impact).

And while these rough estimates find a slightly higher impact for education spending, remember that education spending costs twice as much for a two child family.

The EITC estimates are based on $1K per family, while the public spending estimates are based on $1K per child.

Also: families don’t use the EITC every year, so while increasing spending via education expenditures is a constant expense, the EITC is a variable expense that is only used when families are in poverty.

Because of these factors, my hunch is that the ETIC effects are actually higher per $ spent, but for sake of argument, let’s call it a wash.

The EITC is Well Targeted, Education Spending is Not

Assuming equal effects, the reason the EITC is more efficient in that is better targeted: only poor families get the increases.

Most state funding formulas, on the other hand, give increased funding to districts, not individual students.

This means that giving $1K per student in additional funding to low-income districts is spread across all students in the district, not simply low-income students.

Most importantly, it means that low-incomes students in non-low-income districts don’t receive the benefit.

Rothstein notes as much in his study, writing:

Courts and legislatures can evidently force improvements in school quality for students in low-income districts. But there is an important caveat to this conclusion. As we discuss in Section VI, the average low-income student does not live in a particularly low-income district, so is not well targeted by a transfer of resources to the latter. Thus, we find that finance reforms reduced achievement gaps between high- and low-income school districts but did not have detectable effects on resource or achievement gaps between high- and low-income (or white and black) students. Attacking these gaps via school finance policies would require changing the allocation of resources within school districts, something that was not attempted by the reforms that we study.

Unless States Change Their Funding Formulas, Transfers > Increased Spending

In summary: transfers are targeted at all poor families in a jurisdiction, while education funding increases are generally only targeted at poor families living in low-income districts.

Assuming the research holds on both transfers and education spending – and we continue to see similar effects – then transfers seem to be the much better option, as they reach low-income families in all jurisdictions.

More Research Needed

I view the question of wage subsidies vs. universal basic income vs. increased public services to be one of the most important policy topics out there.

Hopefully we can learn more about the cost / benefits with further research.

Book Review: The Wealth of Humans

2 Replies

51bhoo6jtl-_sx324_bo1204203200_

I just finished Ryan Avent’s .

Summary: Economic Disruptions Require New Social Contracts, which can be a Bloody Process

Ryan’s primary argument is as follows:

1. Periods of rapid technological innovation usually lead to increased prosperity, but the transition can be very disruptive to the existing social and economic order.

2. During these periods of disruption, workers, the economic elite, and those in governmental power have to create the social contract will be for the new order. This is a very difficult process that involves a lot of trial and error.

3. The last time this happened was after the industrial revolution, where numerous wars and revolutions eventually led to a few dominant orders: capitalism and the welfare state (in the West, South and Central America, and parts of the East), socialist dictatorship (in China), and resource based dictatorships (primarily in the Middle East). Of these different variations, capitalism + the welfare state have proven most successful.

4. The digital revolution, which is being driven by continuing gains in computing power, will requite a new social order, especially if this revolution leads to massive surpluses of labor.

5. Creating a new social contact for this age could be just as bloody – or bloodier – than the last go around (WWI, WWII, Mao, the Cold War, etc.).

Reflection #1: Time Between Disruptions is Decreasing, Power of Weapons is Increasing

I generally agree with Ryan’s argument. One additional issue to consider is that the time between economic singularities is decreasing. It took us a very, very longtime to get from hunter gathers to farmers, and a very longtime to get from farming to the industrial revolution.

It’s barely taken us a 150 year to get from the industrial revolution to the computing revolution.

And it’s likely that the computing revolution will seed another revolution (perhaps general artificial intelligence) in another 50-100 years – and who knows what next economic singularity will spring from superior artificial intelligence…

Additionally, technological advancement increases the power and scope of our weapons. We will likely continue to build new weapons that can wipe out humanity, such as synthetic viruses.

In short, the time between the rolls of the dice will decrease, while our odds of losing any given die roll may increase.

One way to reduce the odds of losing is to disperse ourselves and / or our decendents amongst the cosmos in order to decrease the fragility of single planet living.

Reflection #2: A Minor Guess of How to Ease Into the Next Social Order

The more I puzzle over the accelerating impacts of the digital revolution, the more I come back to wage subsidies as the best tool we have for stumbling our way into the next social order.

While universal basic incomes might at some time be warranted, this will be incredibly expensive (given current productivity) and we don’t yet know how to structure a modern society where many people simply don’t work.

Wage subsidies, on the other hand: (1) maintain the connection between work and income (2) lead to less economic distortion, especially compared to minimum wage raises (3) can be raised over time to maintain a sense of economic progress, and (4) help avoid an economy where purchasing power (and presumably social power) consolidates with the top 10%.

Reflection #3: What is Inflationary? What is Deflationary?

Over the past few decades, goods have faced deflationary pressures (most things you buy for day-to-day uses are cheaper now).

Education and healthcare, on the other hand, have been subject to inflationary pressures (they cost more than they used to).

From a pure material progress standpoint, a deflationary future means that wage subsidies might not be necessary to keep improving welfare.

However, if healthcare, housing, and education continue to eat up budgets, people will need higher wages to keep up, especially those that don’t receive government subsidies in these areas.

Lastly, it’s possible that even if purchasing power increases, if income inequality is still increasing, social unrest could still be a major issue.

All this is to say: it’s worth looking at both income and expense.

Reflection #4: Consider Yourself, Consider the Monkey, Consider the Dog

To the extent humans survive the new social order that comes after an artificial intelligence singularity, it’s worth considering what this existence might be like.

Dogs, for example, have done quite well during the era of human dominance. Specifically, they were bred to be happier.

Dogs have also been provided a universal basic income in the form of shelter, food, and treats.

I often struggle with the gap between what I believe to be the best version of myself and the actual reality of the current version of myself. I sometimes get depressed by the lack of progress I’m making.

The fact is that it’s incredibly difficult to become an even better person once you’ve eaten up the low-hanging fruit of adopting classical liberal beliefs and not murdering your fellow humans.

So it’s worth noting that humans (perhaps?) have created the best version of dogs.

Perhaps our descendants will do the same for us, especially if we are able to bring value to whatever is they are seeking in life. Interestingly enough, more intelligent primates have not faired as well as dogs and cats. So don’t assume being #2 on the intelligence pecking order means you’ll be ok.

This may all sound crazy, but it seems extremely unlikely that humans are the endpoint of evolution. So it’s worth considering – what comes next?

An Alternative Interpretation of the Fryer / Dobbie Texas Charter School Study

7 Replies

Screen Shot 2016-08-18 at 10.26.46 AM

Roland Fryer and Will Dobbie just published an excellent study on the Texas charter school sector.

But it’s unclear to me that they captured a very important implication of their research.

I. Study Overview

The study found that charter schools in Texas, on average, have no impact on test scores and a slightly negative impact on earnings.

More interestingly, the study found that No Excuses charter schools increase test scores but only have a small and statistically insignificant impact on earnings.

Their paper ends with this cautionary statement:

Screen Shot 2016-08-18 at 10.34.38 AM

II. Walking Through Low Effect Size and High Effect Size Schools

The famous Anna Karena quote goes something like this: “Happy families are all alike; every unhappy family is unhappy in its own way.”

I think the opposite is true of schools.

When I visit low effect size schools, I am often saddened by the level of dysfunction. Students walk the halls aimlessly, teachers seem woefully unprepared for working in a low-income environment, and the principal generally spends her day putting out fires.

When I visit high effect size schools, I’m often struck by how different they are. While most hit the basics of a calm culture and thoughtful instruction, they vary greatly in atmosphere, curriculum, and staffing models – as well as the overall student experience. A Summit school is very different than a Collegiate Academies school, despite both achieving high effects. Even No Excuses schools can feel fairly different from each other, though they do tend to gravitate around some core practices (that Fryer has helped illuminate).

I also think I would struggle mightily in a blind walk through of .1 and .2 effect size schools; it is highly unlikely I would be able to tell you which school has which effect.

So while it’s easy to identify schools that are a total mess, it’s a little difficult to tease out what’s going well in non-dysfuctional schools, as well as to distinguish between high-performing and very-high-performing schools.

III. Bad Schools Have Bad Effects on Earnings, Good Schools Have Neutral Effects on Earnings

I found this to be the most interesting chart in the study:

Screen Shot 2016-08-18 at 10.33.43 AM

What you see here is that going from (-.2) to (0) effect really matters for earnings. This is indicated by the rising slope in the bottom left quadrant.

Interestingly enough, once you hit (0) effect, going to (.2) effect has little effect on earnings. This is indicated by the relatively flat slope in the the bottom right quadrant.

In short, getting rid of bad schools could have a major effect on the earnings of graduates in an education system (assuming our economy is not a zero sum signaling game).

In a sense, this fits my experiences in spending time in schools. It’s very easy to see how a totally dysfunctional environment could negatively impact students, whereas it’s a little more difficult to tease out the additional impact on students once the basics are in place.

IV. Portfolio Management: What Happens When Charter Schools Grow?

In a world where states and districts are managing their portfolio of schools, the growth of functional schools will be accompanied by the phasing out of dysfunctional schools.

In the best possible world, the growth of new effective charter schools will be accompanied with a reduction in under-performing traditional and charter schools.

Overtime, a system can potentially rid itself of failing schools.

This is what happened in New Orleans.

Screen Shot 2016-08-18 at 12.37.37 PM

While the above analysis is more weighted toward absolute scores (rather than effect sizes), my hunch is that the story would stand with effect sizes as well (I have not run this data yet).

I think much of New Orleans’ gains were driven by the phasing out of failing schools.

It is much less clear to me that schools in New Orleans, to date, have figured out to crack the code of creating schools that are radically superior to your average functioning traditional school.

Hopefully they will.

V. No Excuses Charter Schools May Allow Us to Eliminate Failing Schools and Raise the Aggregate Earnings of Low-Income Students in the United States

So another way to interpret this study is that the growth of No Excuses charter schools could be the key to eliminating failing schools and raising wages of low-income students who would have otherwise have attended failing schools.

Two things would have to hold true for this to be the case: (1) government action or family choice lead to the phasing out of failing schools and (2) No Excuses schools can maintain their neutral effects on earnings even if they enroll the most challenging students from the phased out failing schools.

In other words, for now, the importance of charter school growth might be much more directly tied to eliminating failing schools rather than vastly outperforming functional district schools.

If this is right, No Excuses charter schools might still very well be the most important education reform of the past quarter century.

Is Philanthropic Capital Scarce?

1 Reply

Over at this blog, Albert Wenger has been arguing that private capital is no longer scarce.

He writes:

This means that global investable capital exceeds by 2x the capital required to operate the economy. In fact working capital needs have been declining substantially due to just in time manufacturing, faster electronic payments and better working capital management (eg. through C2FO). If you can reduce the working capital needs of firms by 25% you would move investable capital to close to 3x of required operating capital for the economy.

That means we have massive amounts of capital available to invest in new endeavors. It explains why interest rates are low and there is fairly little that central banks can do about it unless they figure out a way to dramatically reduce investable capital – they can certainly shorten their balance sheets but even that impact is likely to relatively small in the overall scheme of things (eg US Fed about $3 Trillion).

Another way to think it about it is this: we have an oversupply of money and an undersupply of good ideas to invest in.

I’ve been in philanthropy for a year now, and Albert’s thesis led me to reflect on philanthropy.

Broadly speaking, philanthropy can be used to either (1) directly alleviate suffering or (2) help solve complex problems.

For the foreseeable future, there will not be an oversupply of capital to directly alleviate suffering.

If a philanthropist wants to save lives and reduce suffering, there is plenty to invest in; and there is always the option of simply giving cash to people living in poverty.

Many philanthropists, however, also desire to support efforts to solve complex social problems; i.e, to try and create better education, health, and criminal justice systems – or to invest directly in technological solutions in areas such as energy.

The goal here is to reduce future suffering rather than simply alleviate current suffering.

It is not easy to solve such problems. In my work, my days are not chalk full of meetings with people pitching tested, operationally scalable, and financially sustainable interventions that will lead to major improvements in our country’s educational system.

Working in areas such as education, criminal justice, and health is extremely difficult, and scalable solutions are hard to find.

So perhaps Albert’s thesis, in some form, is beginning to hold true for philanthropy.

For this second part of philanthropy’s mission – working to solve complex social problems – it is unclear to me that capital is scarce.

If this is true, it has numerous implications for philanthropists, non-profits, and government.

If I’m able to wrap my head around these implications and organize them in a thoughtful manner, I’ll write a follow-up post.

How to Increase Funding for Public Schooling by ~10 Billion a Year

Is Roland Fryer Right? Or has the RCT Fallacy Reared its Ugly Head?

5 Replies

Screen Shot 2016-04-11 at 9.34.05 AM

Roland Fryer just published a compilation guide to 196 RCTs in education. HT to my colleague Stuart Buck for passing it along.

The compilation is a good review of a bunch of interesting studies. Roland’s contributions always make me think. He also won the John Bates Clark Medal, which is basically the Nobel prize for economics for people under 40.

Yet, while this RCT compilation is informative, I’d be very, very, very hesitant to pass a bunch of laws and regulations based on this type of meta-research.

___

Increasingly, policy makers and pundits are using RCT evidence to make policy. This is generally a step in the right direction, and it’s great to see evidence playing a bigger role in policy making.

Yet, sometimes RCTs are more about Rigorously Contorted Tales than Randomized Controlled Trials.

Call it the RCT Fallacy.

In statistical terms, the RCT Fallacy is pretty close to the concept of external validity, but I think the RCT Fallacy has a little more psychology to it.

So here goes:

The RCT Fallacy occurs when thought leaders propose adoption of policies based on the results of RCTs so as to avoid the messiness of politics, ideology, history, psychology, and evolution.

Fryer is more balanced than most, but, in this case, I think he still succumbs to the fallacy.

___

The RCT Fallacy is grounded in the following:

There is an inverse correlation between the external validity of a RCT and the operational complexity of an industry.
If you have a RCT on your side, it’s much easier to defend yourself against being unreasonable, even if the RCT has very questionable external validity.
If you don’t have a RCT on your side, you can be called an ideologue even if you’re making a very well thought out case.
This leads to the perverse incentive of thought leaders being in a safer place trumpeting policies with modest RCT support rather than proposing solutions that are grounded in a deep understanding of systems, organizations, and humans – but which are difficult to measure with RCTs.
RCTs overvalue what can be measured quantitatively.
RCTs overvalue the worth of understanding existing best practices and testing pilots over the creation of entire systems that accelerate new best practices.
In complex systems with complex organizations, evolution is a better change mechanism than running RCTs and implementing best practice adoption, especially in policy areas where some type of accountability (user choice, output measurement, etc.) can “kill off” bad ideas.
Quasi-experimental studies are often a better way to capture the effects of the impact of complex systems, as it is very difficult to conduct large scale RCTs on system level policy adoption.

___

In other words, RCTs will never tell us:

Whether democracies are better than dictatorships.
How to invent an iPhone.
Whether capitalism is better than Communism.
Whether single payer health systems are better than market based health systems.
Whether or not a start-up will be successful.

Yes, well designed RCTs can inform our decisions on the above issues, but RCTs will not provide definitive evidence on these issues.

___

Fryer’s paper ends with his summary of the RCT evidence in education.

He argues that RCTs have demonstrated that four interventions work: pre-k, high dosage tutoring, managed teacher PD, and charter schools.

The paper ends with the following rally cry:

Screen Shot 2016-04-07 at 8.00.11 PM

I’m not sure courage is what we need:

Pre-K: There is pretty mixed evidence on our ability to scale effective pre-k. Fryer himself notes: “of the 64 treatment effects recorded in these randomized studies [on pre-k], 21 were statistically positive; zero were statistically negative and 43 were statistically indistinguishable from zero.”

Again, I’m not sure “courage” is the term I’d use to describe scaling an intervention that shows zero effect 67% of the time.

Tutoring: Fryer covers some high-dosage tutoring studies that show strong effects. However, the costs of these programs are sometimes upwards of 20% of total per-student spending. Moreover, there would likely be severe human capital limitations if we tried to give high dosage tutoring to all the students who needed it.

Managed Teacher PD: Fryer covers studies that show success for Success For All and Reading Recovery programs. The data seems robust and schools should surely consider adopting these programs. But here’s the thing: nothing is preventing districts from adopting these programs right now!

Perhaps either districts know something that these RCTs aren’t picking up, or perhaps districts are so poorly run that it takes a dramatic intervention to get them to adopt effective programs that have been around for 10+ years.

Charter Schools: While I clearly support charter expansion, charter RCTs often run into the issue of using lottery data which limits trials to schools that are oversubscribed (and thus creates positive bias); as such, I generally view CREDO’s far reaching urban quasi-expermintal studies to be of more use.

___

Again, I don’t mean to pick on Fryer. I’ve learned a ton from reading his research and children would be better off universities were filled with thinkers like him. His work on “looking under the hood” of high-performing charters greatly influenced my thinking on schools, as has his research on tutoring.

Moreover, it’s much better to try and build a policy regime from RCTs than from the weak theory that comes out of many education departments.

But, ultimately, I don’t think that (a) the RCTs covered in his study make a strong case for the scaling of his preferred interventions or (b) that RCTs can ever really tell us how to best design our public education systems.

I do think we should utilize RCTs to help schools make choices about which practices to adopt, but, ultimately, we should utilize theory and quasi-expermential evidence to handle the major public policy questions concerning education, which in mind have more to do with system structure than educational practice.

“We” (researchers, thought leaders, policy makers, etc.) shouldn’t be operationally scaling much; rather, we should be running experiments that give empowered educators and families more information to make great choices.

How Hard is Life? Here’s What the Numbers Say