Category Archives: Policy

What helps poor children more: increasing the EITC or increasing educational funding?

screen-shot-2017-01-08-at-9-19-55-pm

*Note: I’m still working through all this research. If I’ve made a mistake, let me know!

_

In a previous post, I argued that after a certain expenditure level, call it $15K a child, my guess is that families would be better off with additional funds as cash transfers. So a district that spends $20K a student (D.C, Newark, New York, etc.) would best serve families by spending $15K per student and sending $10K in cash to a two child home.

Here’s  a similar policy question: if taxpayers are willing to spend extra money to help poor children, should they increase the earned income tax credit (giving working low-income parents increased money) or raise educational funding?

Last month, Kevin Carey and Elizabeth Harris wrote a NYT column summarizing the most recent research on increasing education funding. Their conclusion: money probably does increase test scores.

Unfortunately, they did not the review the research on what happens when you give similar amounts of money to families via other transfers, such as the EITC.

The Effects are Within the Same Range

While none of the research is an exact science, research on the EITC (see a summary here) finds that children under the age of 12 see increases of ~.06-.1 SD per $1,000 increase in the EITC (cumulative full schooling impact).

Rothstein finds that low income districts increase their performance by ~.1 SD per $1K increase in funding (cumulative 10 year impact).

And while these rough estimates find a slightly higher impact for education spending, remember that education spending costs twice as much for a two child family.

The EITC estimates are based on $1K per family, while the public spending estimates are based on $1K per child.

Also: families don’t use the EITC every year, so while increasing spending via education expenditures is a constant expense, the EITC is a variable expense that is only used when families are in poverty.

Because of these factors, my hunch is that the ETIC effects are actually higher per $ spent, but for sake of argument, let’s call it a wash.

The EITC is Well Targeted, Education Spending is Not

Assuming equal effects, the reason the EITC is more efficient in that is better targeted: only poor families get the increases.

Most state funding formulas, on the other hand, give increased funding to districts, not individual students.

This means that giving $1K per student in additional funding to low-income districts is spread across all students in the district, not simply low-income students.

Most importantly, it means that low-incomes students in non-low-income districts don’t receive the benefit.

Rothstein notes as much in his study, writing:

Courts and legislatures can evidently force improvements in school quality for students in low-income districts. But there is an important caveat to this conclusion. As we discuss in Section VI, the average low-income student does not live in a particularly low-income district, so is not well targeted by a transfer of resources to the latter. Thus, we find that finance reforms reduced achievement gaps between high- and low-income school districts but did not have detectable effects on resource or achievement gaps between high- and low-income (or white and black) students. Attacking these gaps via school finance policies would require changing the allocation of resources within school districts, something that was not attempted by the reforms that we study.

Unless States Change Their Funding Formulas, Transfers > Increased Spending

In summary: transfers are targeted at all poor families in a jurisdiction, while education funding increases are generally only targeted at poor families living in low-income districts.

Assuming the research holds on both transfers and education spending – and we continue to see similar effects – then transfers seem to be the much better option, as they reach low-income families in all jurisdictions.

More Research Needed

I view the question of wage subsidies vs. universal basic income vs. increased public services to be one of the most important policy topics out there.

Hopefully we can learn more about the cost / benefits with further research.

These major cities could afford a $10K per family basic income w/o raising taxes

There’s been a lot of talk recently about the pros and cons of a universal basic income.

There’s also been a lot of talk about how expensive this would be.

But, from my quick analysis, I actually think quite a few major cities could institute a child based basic income utilizing only existing tax revenues.

I. Numerous big cities spend over $20,000 per student

Getting accurate city per-pupil spending amounts can be a near impossible task, but nearly all sources I reviewed showed that Washington D.C., Newark, and New York City spend at least ~$18K a student. I think Boston spends around this as well.

And higher end estimates get closer to $25-30K per student.

For the sake of modeling out how to end child poverty, let’s assume ~$20K per student.

II. Giving $5K per student per year basic income back to families

Let’s say that starting next school year, each of these cities decided to reduce public education spending from $20K to $15K per student, and instead of giving this money back to taxpayers, provided a universal basic income of $5K per child back to families.

Assuming your average family has about two kids in the public school system, that’s $10K per family.

That won’t make any family rich, but it would probably get most families out of deep poverty.

III. A $5K per student spending reduction would likely not lead to major education losses

Dropping to $15K per student would still put these cities ahead of the national average of ~$10K per student. Even adjusting for cost of living differences, none of the cities would be that far off typical educational spending.

To get a taste for what cities are able to achieve with various students and spending, see below:

screen-shot-2016-12-05-at-9-04-47-pm

The above is by no means an accurate picture of school system effectiveness, as it’s based on absolute scores rather than growth; however, it provides decent evidence that schools systems that spend $10K-15K to student can still achieve relatively ok outcomes.

50% of the top ten adjusted scoring cities in the country are located in Texas and Florida, both of which spend very modestly.

Ultimately, the students in Washington D.C., Newark, and New York City are different than students in other cities, so we can’t make any claims with 100% confidence, but the experience of other cities suggests that spending $15K per student is enough to provide an education on par with other major cities across the country.

IV. What do you think parents would want?

Somebody should poll this question, but I expect families that have two children would rather have $10K in cash per year / $15K in education spending rather than $0K in cash / $20K in education spending.

For many of these families, $10K per year would be absolutely game changing.

It would be very interesting for an aspiring politician to run on this as a single issue platform. Or to take the issue to a popular referendum.

V. Trade off that no one explicitly made

Here’s the thing: every marginal $1K increase in education spending can be justified at the time. There’s always a compelling financial ask to be made when families in poverty are struggling to get a great education.

But, eventually, these marginal increases can lead to spending allocations that just might be out of line with what families want and what might be in the public interest.

My guess is that providing a $5K per student basic income to families – and reducing educational spending by the same amount – would increase the welfare of families in some cities.

 

Here’s a better framework for thinking about Trump

As I’m reading anti-Trump and pro-Trump commentary, I’m finding very few pieces that fully explore the different possibilities of a Trump presidency.

So I tried to create a graph to chart what I think are three dominant considerations we should be using to understand the president elect.

A Framework for Understanding the President Elect 

screen-shot-2016-11-14-at-9-55-18-am

This framework captures 3 primary spectra:

Social Liberalism: Does a leader have respect for people of all races, gender, sexuality, religion, and places of birth?

Economics: Does a leader lean more toward populist economics (which often involves trade protectionism and anti-immigration stances) or globalist economics (which generally leans towards free trade and more immigration)?

Rule of Law: Does a leader behave within the established norms of domestic democracy and international rule of law, or does she lead by greatly damaging democratic institutions and grossly violating international law?

To chart some historical examples, I spent a few minutes trying to plot the last few American presidents and Hitler. I was just aiming to be directionally correct but am in no way trying to argue that I plotted these perfectly.

Each Variable is Very Important, But I Think Rule of Law is Probably Most Important

You could make reasonable arguments for each variable being the most important consideration.

If I had to argue for social liberalism, I’d say that even someone who works within the rule of law can do terrible harm to minority populations.

If I had to argue for economics, I’d say that someone who wrecks the international economic system could unleash untold suffering on the poor of the world.

In arguing for rule of law, I’m mostly arguing from this recent historical fact that so many of the world’s major mass deaths have been caused by dictators, such as Hitler, Mao and Stalin.

This graph is illustrative:

dictators

I’d need to think harder before having stronger opinions on the relative importance of each variable.

The only thing I am confident in is that they’re all important.

When to Build Bridges, When to Join the Resistance 

I think both Trump and Clinton supporters have reasonable grievances about the world.

I don’t think that it’s in our country’s best long-term interest for each side to: (1) argue loudly about their legitimate grievances (2) not listen to the other side’s legitimate grievances and (3) not differentiate between policy differences and threats to the survival of the nation.

I think economics and immigration are policy differences.

I think respect for rule of law is an issue that gets at the survival of our nation.

And I think social liberalism sits between the two, in that it determines who receives the full benefit of the rule of law within our country, which in its most severe form can threaten the survival of our nation (slavery) but in other cases can be solved through the political process (gay marriage).

I think it’s worth trying to build bridges around policy and less severe forms of social illiberalism.

I think it’s worth considering more radical forms of resistance in cases of major threats to the rule of law and severe cases of social illiberalism.

In Sum

Our country is deeply divided about many issues.

It’s important to tease out the differences between these issues, both to understand ourselves and to understand the president elect.

I know that this is a rather unemotional way of trying to understand issues riven with deep emotions.

I’ve felt a lot over the past week – it’s been especially hard to hear stories of children in our schools who don’t feel safe – and I’ll continue to listen to these emotions.

But I also want to try and understand the way forward, and, for me, frameworks help.

Post election reflections

On this blog, I’m going to keep my post-election reflections focused on education, save for one thought: continued progress in the realization of the American dream is extremely important for our nation and the world as a whole, and I look forward to continuing to play a bit part in this vitally important endeavor. I hope you do too.

On to education.

#1: MA and GA Drive Home that Traditional Public School Support is Boosted by Populism 

In Massachusetts, a populist blue state, the charter school cap was not lifted because unions effectively portrayed this expansion as something that would harm traditional public schools.

In Georgia, a populist red state, a state takeover entity was rejected because opponents effectively portrayed the intervention as something that would hurt local public schools.

In populist politics, communications messages that focus on preserving local, traditional public schools appear to be very effective.

Interestingly enough, it’s not clear to me that these results demonstrate a significant antipathy toward charter schools; rather, they seem to indicate a deep protectionist instinct for traditional public schools.

For charters to be successful, we may need to communicate in a fashion that reduces fears that existing traditional schools will be harmed.

#2: In Cities Where Charters are Normalized, Elections are Producing Pro-Reform Results

In New Orleans, Indianapolis, and Oakland – all cities with 25%+ high-performing charter sectors – reforms either held or expanded majorities.

Generally, reformers prefer top-down quick wins like ballot initiatives; however, we have emerging evidence that, in cities with higher charter market share, elected school boards can tip into modestly sustainable pro-reform majorities.

This is more evidence that market share drives everything.

#3 What Will Ideological but Not Constituent Support Deliver? 

The federal government is now fully controlled by Republicans, a party that is highly ideologically aligned with choice and charters.

However, many Republicans represent states without large charter sectors. As such, there is not uniform constituent demand for more charters.

An open question to me is how much Republicans will use this moment to expand thoughtful, sustainable choice reforms.

#4 The Status of Within District Reform will Rise 

Given the populist rise of traditional school protection, reforms that disrupt within districts – such as technology – will likely see in increase in philanthropic support.

In Sum

There is much to be learned by listening to how people express themselves through voting.

It’s a noisy signal, but in a world of communication bubbles, it’s a signal nonetheless.

The Answer is 6.7 Miles. What is the Question?

Screen Shot 2016-06-15 at 8.01.52 PM

The question is: how far, on average, would a family send their child to attend a school that is in the highest category of the state accountability system compared to a school in the lowest category of the state accountability system?

This is from a recent report on the DC public school system. The analysis, while useful, isn’t perfect in that it only includes families who utilized the enrollment system, but it does add to the emerging literature on the revealed preferences of families that participate in transparent enrollment systems.

 

___

Here’s another answer: it increases racial integration.

The question is: does DC’s unified enrollment system increase or decrease racial segregation?

Screen Shot 2016-06-15 at 8.06.39 PM

Shockingly enough, assigning families to neighborhood schools that are zoned by property values is not a great way to decrease segregation.

 

___

Answer: Unclear.

Question: Do parents care about a school’s academic growth (as opposed to absolute test scores)?

Screen Shot 2016-06-15 at 8.20.46 PM

Interesting but not shocking. Parents probably care a lot about peers and status.

Also interesting, this seems more true of low-income families:

Screen Shot 2016-06-15 at 8.25.26 PM

This raises an interesting question for policy makers: given that growth more accurately measures a school’s impact, should they design grading systems that prioritize growth (as DC’s charter framework does) even though low-income parents might care more about absolute scores?

Or perhaps not – maybe low-income families aren’t considering the growth based performance framework because the government is hiding this information:

Screen Shot 2016-06-15 at 8.27.05 PM

___

One last answer: Families who aren’t assigned to a school in the lowest performance category, as well as the politicians and superintendents who seek their favor.

The question: who loves neighborhood schools?

It remains shocking to me that public leaders in cities such as Oakland are vehemently opposed to unified enrollment on the grounds that such systems will undermine public education.

The only thing a unified enrollment system undermines is the privilege of those who benefit from institutional racism and widespread income inequality.

 

 

5 Dominant Theories in Education Philanthropy

It is important for funders, entrepreneurs, and policy leaders to understand the dominant theories of change in education philanthropy.

Funders should be clear about what they believe, as well as understand why other funders hold different beliefs.

Entrepreneurs should seeks funds from aligned funders and be pushing foundations to align their theories to what’s actually happening on the ground.

Policy leaders should be evaluating, debating, and challenging funders on how their theories might be improved – and calling out when the theories are simply wrong.

I see five dominant theories in education philanthropy; they are detailed below, with some minor commentary on areas of agreement, admiration, and concern.

___

#1: Teachers! 

Theory: The most effective way to increase student achievement is to improve teacher recruitment, preparation, development, and evaluation.

How to Identify these Funders: These folks often start their sentences with “research shows that teachers are the most important in-school factor” and end their sentences with “as Finland and Singapore have shown.” I kid but it’s true.

What I Agree With: In my work in cities across the country, few high-quality charter schools are satisfied with teacher preparation at colleges of education. Moreover, colleges of education have done a poor job of developing a knowledge based around effective teaching. Improvements in these areas (if feasible) would be of great use.

Concerns: I think this theory’s greatest flaw is that teachers are in fact not the most important factor. As I recently wrote, I’m highly convinced that school operators are the most important factor. An over emphasis on teachers may come at the detriment of a focus on operators.

#2 We Need Better Products

Theory: Innovation in products (schools models, software, platforms, etc.) will radically improve the student learning experience.

How to Identify These Funders: You’re in Silicon Valley talking to a 29 year old billionaire who begin his sentence with “factory model” and ends his sentences with “disruption through exponential growth.” I kid but it’s true.

What I Agree With: I’m bullish on much of this theory of change. I’m extremely excited by many new school models (see Silicon Schools portfolio), software (Zearn, Dreambox, etc.), and platforms (Alt School, Summit). Education reform has a history of not being end user focused, and the consumer oriented discipline of this crew is welcome.

Concerns: I worry that these funders underestimate the power of regulatory change in creating the conditions for better products. The “we don’t need more charter schools we just need to scale Summit” ethos is dangerous, as Summit will inevitably not be the pinnacle of education delivery. If the product folks will shy away from necessary regulatory battles because these battles are not as fun as creating new products, we will have far fewer great education products.

#3 Turn the Battleship 10 Degrees 

Theory: If you don’t focus on the where the kids are at now, you’re going to lose a generation of kids while you build all these great products / charter schools / etc – minor improvements in big systems matter.

How to Identify These Funders: When you go to pitch them they begin by presenting you with a 90 slide ppt deck which begins with “district proof point” and ends with “teachers really, really do love VAM.” I kid but it’s true.

What I Agree With: Most students in this country do attend traditional public schools, and the regulatory policy framework governing these schools – especially in areas such as standards and assessments – is worth trying to get right.

Concerns: Outside of a few key areas (such as standards and assessments), I’m skeptical that over the long haul many of these reforms (such as teacher evaluations) will work or stick. And even if they do stick the political opportunity cost is so high that they will have to achieve major impacts to warrant the cost.

#4: Social Justice 

Theory: Radically increasing educationally opportunity will require significant improvements in racial justice, economic inequality, integration, criminal justice, and healthcare (including early childhood services).

How to Identify These Funders: The Bernie Sanders lapel pins. I kid but it’s true.

What I Agree With: It’s been exciting to watch a new wave of reformers who believe in this theory and also believe you need to push on in-school reforms as well; for too long, many of the most vocal leaders of this theory were paradoxically nihilistic about making schools better (teachers can’t improve student achievement! pay teachers more!). Given the obvious importance of these social issues, I’m eager to watch how these leaders make the reform movement better.

Concerns: When it comes to actual policy making, I sometimes find that these leaders have a somewhat naive belief on the ability to improve districts, as well as an under underappreciation for how hard it is to scale effective social services.

#5: Governance! 

Theory: The governance of public education is the root cause of most of our system’s ills.

How to Identify These Funders: They start their sentences with “there was this groundbreaking voucher study that gave 14 kids a $600 stipend” and end their sentences with “Freedom!” I kid but it’s true.

What I Agree With: Most of it. I believe that will we see increased educational opportunity and, yes – freedom – by allowing educators to start and run schools, as well as giving parents the right to freely choose amongst these schools. And the only way this will occur is if we overhaul how we govern schools.

Concerns: There’s a ton of internal debate within the governance community about how to best regulate these systems, and I have concerns about all the most popular models (vouchers, education savings accounts, charters, portfolio, etc.). The tension between innovation and equity is one I struggle with here, and I’m eager to watch more experiments unfold.

____

All right, this was just a quick run down. I’m sure I’m missing a dominant theory or two.

And I admit that I cheated by lumping in early childhood with social justice, but 5 theories seems tidier than 6.

Lastly, I think this is less of an issue of “one of these theories is true and the rest aren’t” and more of a case of resource allocation.

In a world of limited time, talent, and money – what should we focus on?

 

Is Roland Fryer Right? Or has the RCT Fallacy Reared its Ugly Head?

Screen Shot 2016-04-11 at 9.34.05 AM

Roland Fryer just published a compilation guide to 196 RCTs in education. HT to my colleague Stuart Buck for passing it along.

The compilation is a good review of a bunch of interesting studies. Roland’s contributions always make me think. He also won the John Bates Clark Medal, which is basically the Nobel prize for economics for people under 40.

Yet, while this RCT compilation is informative, I’d be very, very, very hesitant to pass a bunch of laws and regulations based on this type of meta-research.

___

Increasingly, policy makers and pundits are using RCT evidence to make policy. This is generally a step in the right direction, and it’s great to see evidence playing a bigger role in policy making.

Yet, sometimes RCTs are more about Rigorously Contorted Tales than Randomized Controlled Trials.

Call it the RCT Fallacy.

In statistical terms, the RCT Fallacy is pretty close to the concept of external validity, but I think the RCT Fallacy has a little more psychology to it.

So here goes:

The RCT Fallacy occurs when thought leaders propose adoption of policies based on the results of   RCTs so as to avoid the messiness of politics, ideology, history, psychology, and evolution.

Fryer is more balanced than most, but, in this case, I think he still succumbs to the fallacy.

___

The RCT Fallacy is grounded in the following:

  • There is an inverse correlation between the external validity of a RCT and the operational complexity of an industry.
  • If you have a RCT on your side, it’s much easier to defend yourself against being unreasonable, even if the RCT has very questionable external validity.
  • If you don’t have a RCT on your side, you can be called an ideologue even if you’re making a very well thought out case.
  • This leads to the perverse incentive of thought leaders being in a safer place trumpeting policies with modest RCT support rather than proposing solutions that are grounded in a deep understanding of systems, organizations, and humans – but which are difficult to measure with RCTs.
  • RCTs overvalue what can be measured quantitatively.
  • RCTs overvalue the worth of understanding existing best practices and testing pilots over the creation of entire systems that accelerate new best practices.
  • In complex systems with complex organizations, evolution is a  better change mechanism than running RCTs and implementing best practice adoption, especially in policy areas where some type of accountability (user choice, output measurement, etc.) can “kill off” bad ideas.
  • Quasi-experimental studies are often a better way to capture the effects of the impact of complex systems, as it is very difficult to conduct large scale RCTs on system level policy adoption.

___

In other words, RCTs will never tell us:

  • Whether democracies are better than dictatorships.
  • How to invent an iPhone.
  • Whether capitalism is better than Communism.
  • Whether single payer health systems are better than market based health systems.
  • Whether or not a start-up will be successful.

Yes, well designed RCTs can inform our decisions on the above issues, but RCTs will not provide definitive evidence on these issues.

___

Fryer’s paper ends with his summary of the RCT evidence in education.

He argues that RCTs have demonstrated that four interventions work:  pre-k, high dosage tutoring, managed teacher PD, and charter schools.

The paper ends with the following rally cry:

Screen Shot 2016-04-07 at 8.00.11 PM

I’m not sure courage is what we need:

Pre-K: There is pretty mixed evidence on our ability to scale effective pre-k. Fryer himself notes: “of the 64 treatment effects recorded in these randomized studies [on pre-k], 21 were statistically positive; zero were statistically negative and 43 were statistically indistinguishable from zero.”

Again, I’m not sure “courage” is the term I’d use to describe scaling an intervention that shows zero effect 67% of the time.

Tutoring: Fryer covers some high-dosage tutoring studies that show strong effects. However, the costs of these programs are sometimes upwards of 20% of total per-student spending. Moreover, there would likely be severe human capital limitations if we tried to give high dosage tutoring to all the students who needed it.

Managed Teacher PD: Fryer covers studies that show success for Success For All and Reading Recovery programs. The data seems robust and schools should surely consider adopting these programs. But here’s the thing: nothing is preventing districts from adopting these programs right now!

Perhaps either districts know something that these RCTs aren’t picking up, or perhaps districts are so poorly run that it takes a dramatic intervention to get them to adopt effective programs that have been around for 10+ years.

Charter Schools: While I clearly support charter expansion, charter RCTs often run into the issue of using lottery data which limits trials to schools that are oversubscribed (and thus creates positive bias); as such, I generally view CREDO’s far reaching urban quasi-expermintal studies to be of more use.

___

Again, I don’t mean to pick on Fryer. I’ve learned a ton from reading his research and children would be better off universities were filled with thinkers like him. His work on “looking under the hood” of high-performing charters greatly influenced my thinking on schools, as has his research on tutoring.

Moreover, it’s much better to try and build a policy regime from RCTs than from the weak theory that comes out of many education departments.

But, ultimately, I don’t think that (a) the RCTs covered in his study make a strong case for the scaling of his preferred interventions or (b) that RCTs can ever really tell us how to best design our public education systems.

I do think we should utilize RCTs to help schools make choices about which practices to adopt, but, ultimately, we should utilize theory and quasi-expermential evidence to handle the major public policy questions concerning education, which in mind have more to do with system structure than educational practice.

“We” (researchers, thought leaders, policy makers, etc.) shouldn’t be operationally scaling much; rather, we should be running experiments that give empowered educators and families more information to make great choices.