Category Archives: Regulation

What is California telling us about what parents want from public education?

I sit on the board of the California Charter School Association (CCSA), which is one of most effective charter associations in the nation.

Their data team put this slide together.

It’s a little complicated, but it’s very informative.

Screen Shot 2018-04-05 at 8.08.33 PM

To build the chart, CCSA looked at schools that are serving the same type of demographics and then compares their test scores.

A school that outperforms similar schools, gets a higher score (8-10 is really good); a school that underperforms similar schools, gets a lower school (1-3 is pretty bad).

This methodology – absolute test scores controlled for demographics – is imperfect, but it’s the best methodology you can use given California does not publicly report student growth scores.

Charter schools serving students in poverty outperform district schools on state tests 

43% of charter schools with higher concentrations of poverty outperform similar schools.

In California, if you are a low-income Hispanic or African-American child, you are more likely get a better education (as measured by test scores and parent demand) if you attend a charter school.

This is good news for the educators and families who are working together to create better educational outcomes in at-risk communities.

If parents are demanding schools with good test score impacts, the government’s response should be obvious: let more of these schools open.

Charter schools serving middle class students underperform district schools on state tests 

But not all charter schools are outperforming their peers on tests scores.

47% of charter schools serving middle class students perform worse than similar schools. And only 26% perform better.

So perhaps California should close some of these charter schools that serve middle class families? Research covered in this post shows that schools with negative test impacts tend not to have large positive life outcome impacts.

But here’s the odd thing: presumably, middle class families have a decent amount of information at hand when making school choices. It takes all of two minutes to scan Zillow or Great Schools to get a quick read on the absolute test score performance of any school in the state.

So why are all these relatively well resourced families sending their children to lower-performing charter schools as measured by state tests?

I’m not sure. It would be interesting to focus group and poll them to learn more.

And with regards to closure, while I surely disagree with middle class Californians on many policy issues, I’m not sure that I think I know enough about their children to close schools that have modest negative test impacts but high enrollment demand.

Charter schools don’t exist unless parents want their children to attend them 

One of the best features of charter schools is that they don’t exist unless parents choose them. No one is assigned to a charter school.

So what are we to make of this data where families in living in poverty are choosing schools with positive test scores impacts and middle class families are choosing schools with negative test score impacts?

I think the starting point should be to assume that families, on average, are in a better position to make an informed choice than government is.

Remember, government’s default assignment algorithm is to look at your family’s address and then assign your child to the nearest school. It’s not very nuanced!

My guess is that parent choice will outperform geographic assignment when it comes to finding great fits between kids and schools.

But I do think we should be open to the idea that parents, sometimes en masse, can make mistakes. And, at times, this can warrant government intervention.

Sometimes performance might be an indicator, such as when families keep sending their children to high schools with below 40% graduation rates and /or schools with extremely negative value-add scores. If less than half the kids are graduating, and those that do are barely literate, government should step in.

Sometimes lack of alignment with our nation’s professed values might be an indicator: certain public schools have at times been captured by groups, sometimes religious, that do not teach basic democratic values.

In these cases of significant performance or culture malfeasance, government should consider intervention, ideally by handing over management of the school to a non-profit organization that can achieve better results.

I don’t know enough about individual school performance to know if what’s happening in California with middle class families equates to education malfeasance, but I’m a bit skeptical.

My hunch is that once absolute test score levels surpass a certain floor (as they tend to in middle class schools), families just care a lot about other factors.

Even if I might make a different choice, I don’t know that the situation warrants government intervention.

When government should not intervene 

While it’s difficult to decide when government should intervene, it’s still pretty clear to me when government should not intervene.

When schools have both high demand and high test score impacts with students living in poverty, government should not prevent these schools from serving more students!

California should follow this common sense policy.

Unfortunately, too many school districts do not.

SaveSave

SaveSave

What do test scores tell us about schools?

Collin Hitt, Michael Q. McShane, and Patrick J. Wolf just published a report on the connection (or lack of connection) between test scores and long-term outcomes.

They looked at a bunch of school choices studies and tried to see if a school’s impact on student test scores was connected to its impact on student life outcomes.

Their conclusion: “at least for school choice programs, there is a weak relationship between impacts on test scores and later-life outcomes.”

Much of our K12 education policy is predicated on the idea that test scores are an important measure of school performance. If this is not true, behavior should change.

The Laura and John Arnold Foundation is currently funding more research on this question, and I’m eager to see what we find, especially with regards to income gains over time. This study was predicated on high school and college attainment being an indicator of long-term outcomes, but schooling isn’t always learning. So we should be careful judging school performance based on later school attainment, rather than income (or other measures).

But for now, here’s what’s on my mind after reading the study.

What do bad test score results tell us?

If I’m reading their report correctly (and I hope the authors correct me if I’m not), it seems rare that schools have a negative impact on test scores but a positive impact on long-term outcomes.

In the 126 study comparisons where test scores impacts were compared to high school or college outcomes, there were only 2 instances where a study found a significant negative impact on test scores and a significant positive impact on life outcomes.

It seems rare for a school to do really poorly on test scores but really great on life outcomes. I think the authors underemphasize this point in their paper.

What do mediocre test scores tell us?

It is much more common for schools to have a neutral (insignificant) impact on test scores but a significant positive impact on long-term outcomes. It looks like this occurred 32 times in the research review.

There may be a bunch of schools that don’t really impact test scores but are doing something that helps with long-term outcomes.

What do great test scores tell us?

There are no cases where a study found significantly positive test scores and significantly negative life outcomes.

So it seems rare for schools to jack up test scores but ruin kids lives. That’s good.

However, there were 17 instances of studies finding positive impact on test scores but neutral impacts on long-term outcomes.

So there seem to be some schools that are achieving good test results without translating these into great long-term outcomes.

How should this research affect regulation and philanthropy?

If these results hold, I think I will maintain my belief that we should replace schools with persistent very negative test scores. There appears to be little risk that these schools are really amazing schools. The negative test scores are a useful signal.

Yes, there might be other schools that are just as bad at life outcomes that are not closed because they achieve better test scores, but so long as we are closing schools that are not delivering great life outcomes, and opening schools that have a better chance of achieving great life outcomes, this seems like a worthwhile tradeoff.

But when it comes to expanding schools, if this research holds, I will rely less on positive test scores, and I think authorizers should do the same.

From an authorizer perspective, so long as a school does not have significantly negative test scores, perhaps the school should be able to expand so long as there is parent demand.

Philanthropy may also need to adjust by investing more heavily in school operators that show a positive impact on life outcomes (irregardless of test scores), and being willing to fund mediocre test score schools who either have high parent demand or who are using practices that are correlated with positive long-term outcomes (more research needed to determine what these might be).

I am very open to moving in this direction if research warrants it. The idea that it’s easier to tell a bad school than it is to identify a great school already matches my intuition, and deferring to parent judgment makes a lot of sense if we are not confident in our analysis of performance.

Charters growing in your city? You have 5 options.

Charters schools continue to scale in urban areas. In many cities, charters serve over 30% of students.

Screen Shot 2017-03-23 at 10.03.33 PM.png

In 44 cities charters serve over 20% of students.

These 44 cities, as well as many others in the future, will have to evolve their educational systems to govern a mixed portfolio of school types.

What options are available to these cities? Here’s five, some of which will be much better for children than others.

1. Implode (Detroit) 

In Detroit, the school district responded to charter growth by bankrupting itself. It lost enrollment, took on debt, and continued its academic and operational dysfunction.

In failing to respond productively to charter growth, the district hurt students and cost taxpayers nearly a billion dollars.

2. Compete (Washington D.C.)

In Washington D.C., charters now serve nearly 50% of the students. During the past decade of charter growth, the district responded by becoming perhaps the highest performing urban school district in the nation.

The district lost nearly half its students and radically increased its performance.

The district didn’t really partner with charters, it just stepped up its game.

3. Coordinate and Collaborate (Denver)

Denver Public Schools responded to charter growth by coordinating with the charter sector.

For much of the past decade, it gave charters facilities to grow in neighborhoods where more good schools were needed. The district also set-up a unified enrollment system that made it easy for families to choose easily between district and charter schools.

While there have been some bumps along the way, for the most part the district has supported the best charters to expand and has closed the worst charters.

4. Blur the Lines (Indianapolis, Camden) 

A few years ago, the Indiana legislature passed a law that allows for Innovation Schools, which are authorized by the district, have many of the autonomies of charters, are governed by non-profit boards, but still sit within the district’s enrollment and accountability reporting.

With its Renaissance Schools, Camden has done something similar: Renaissance Schools are more tightly managed by the district, but still retain most of the autonomies of charters schools.

In both Indianapolis and Camden, the district has co-opted the best of the charter model while still maintaining a tighter form of local oversight and control.

5. Govern (New Orleans)

In New Orleans, the district responded to increasing charter growth by relinquishing its operational duties and transforming into a regulator.

Rather than operate schools, the district sets performance targets, monitors for equity, and annually opens great schools and orchestrates the transformation of failing schools.

This has led to unprecedented student achievement gains.

Which Way to Go? 

While I think the last option (govern) is the best way to go, cities have also seen academic growth by competing, coordinating, and blurring the lines.

These cities are the reason I’m skeptical of people who argue that charter growth will hurt traditional public schools.

There’s an emerging group of cities who are proving this clearly doesn’t need to be the case.

I’m hopeful that their successes will be replicated much more often than not.

Why don’t we have a 10x better school?

10x

There’s a Silicon Valley mantra that your need new product needs to be 10x better than the incumbent in order for you to displace them and have a shot at a market monopoly.

Uber, for example, is a 10x product. It vastly better than taxis on so many dimensions (price, easy of use, consistency, service, etc.).

In education, it’s unclear to me that we’ve built a schools that are 10x better than the median traditional district school.

We have built schools that are 10x better than failing urban schools, and it’s no surprise that this is where the entrepreneurial sector has seen so much success.

Why haven’t we built a bunch of schools that are 10x better than an average school?

I’m not sure, but some reflections below.

1. Educators are trying to be 10x at the wrong thing 

Great tech companies usually initially succeed because their technology – not their operations – is 10x better than their competitors.

Often times, technology can be built by smallish group of highly talented people and then scaled at little marginal effort or cost.

So far, school operators have not been able to replicate this model of technological advancement and scale. This way of thinking is not in their DNA. They are still trying to squeeze 10x improvements out of areas such as program design, human resources, and operations.

It will be interesting to see if Summit, Alt School, Khan Academy and others can utilize a 100x tech backbone to scale an instructional program that, over time, evolves into a 10x better school.

2. There’s no profit motive

Perhaps. With companies like Bridge Academies, we are seeing interesting attempts at 10x breakthroughs in the for-profit international market.

On the other hand, there’s plenty of for-profit K12 and university operators in this country, and they aren’t launching 10x better schools that are displacing government and non-profit operators.

3. The education sector is over-regulated

Perhaps regulation is stifling innovation.

I’m sure that this is at least partially true, but the most in demand private schools are not very innovative. Rather, they tend to be highly selective, academically rigorous, extracurricular rich, and culturally strong.

And they also cost $30,000 a year.

So, to date, the private side of things is not exactly delivering a bunch of breakthrough innovations.

Maybe an expansion of education savings accounts will unleash some 10x products, but it’s hard to say this with great confidence.

4. The industry culture is risk averse 

Education may be attracting and retaining professionals who are generally not willing to take the risks needed to achieve 10x products.

In some sense, given that children are involved, this culture is to some extent warranted.

But maybe it needs to be loosened up a bit.

5. This is (mostly) as good as it gets

Not everything can be made better. The fork I ate my dinner with today is not that much better than a fork from the 1970s.

Perhaps this is about as good as schooling gets.

My guess? 

I don’t yet have opinions that are strong enough to warrant action beyond the work I’m already doing.

But I want to keep thinking about this.

Draft Text for a State Constitutional Amendment to End the Education Wars

dutch

If the United States could adopt the educational regime of any country in the world, I would not choose Finland or Singapore or South Korea.

I would choose the Netherlands.

___

In 1917, the Dutch had a national education battle about what types of schools deserve public funding.

This battle, as well as other policy battles, was settled with a constitutional amendment which was passed during what is known as the “Pacification of 1917.”

The constitutional amendment established a fundamental right to open a school and receive pubic funding.

What a remarkable way to end the education wars!

___

Since the Pacification of 1917, the Dutch government has built a set of regulations to manage the implementation of the constitutional amendment.

Depending on where you are on the freedom axis, you might find these regulations reasonable or tyrannic.

I find some of them to be reasonable (national academic objectives) and some not (negotiating teacher salaries at the national level).

The Dutch have blazed one trail on how to regulate the freedom to open publicly funded schools; surely, other experiments would teach us much.

___

So here is proposed text for a state constitutional amendment in the United States of America:

“The right to found a school or enroll in a school shall not be abridged by government or any entity receiving government funding. All schools that meet basic education standards shall receive public funding based on a per-pupil allotment that is weighted based on student need and uniform across schools.”

___

I don’t think every state in the United States of America should pass this amendment.

But I think it would be great if a few states did.

I imagine each state would blaze its own path in determining how to manage a system where citizens had a constitutional right to open schools and where families had a constitutional right to choose amongst these schools.

I also think this approach – passing a constitutional amendment – has much more moral and legal force than pushing for ad hoc funding programs, such as education savings accounts or limited vouchers.

A right is a fundamental, a program is not.

___

Oh, and for whatever it’s worth, the Dutch rank number 10 in the world in student achievement based on the 2012 PISA results (they’re actually #7 if you throw out Shanghai, Hong Kong, and Macau, which last time I checked aren’t countries).

Screen Shot 2016-06-03 at 6.36.41 PM

 

The Folly of Voucher Advocates?

voucher

A new study just came out showing that the Louisiana voucher program had negative effects on student achievement.

It’s one year of data on a new program, so I would caution against any grand proclamations on the usefulness of vouchers. There’s a much richer literature from which one can draw conclusions.

Perhaps more interesting is how voucher advocates reacted.

Jason Bedrick’s piece – The Folly of Overregulating Vouchers – criticized the Louisiana program for:

  • Not allowing tuition in excess of the vouchers.
  • Not allowing private schools to use selection criteria for admitting students.

I feel like I’m missing something.

The logical extension of Jason’s argument is that an all voucher education system would lead to a public education system where all schools would be allowed to reject students based on wealth, academic performance, and behavior.

Is this right?

Either voucher proponents have very different views of equity than most citizens, or they don’t really view vouchers as a replacement model for the current public education.

I’m curious – which is it?

Overall, I’m sympathetic to lowering barriers to entry (you have a crazy idea that parents will sign up for, go for it) and to reducing test based accountability (you and families think there’s a better way to measure school performance, go for it).

I understand the risks involved with this type of deregulation, but I think it’s worth trying and seeing what we learn. I don’t know if it would work, but it might, and the potential the upside seems high.

I also think there are things you can do to solve for equity (significantly weighting vouchers for at-risk students), that will lead to higher performing private schools enrolling hard to serve kids.

But, ultimately, I’m not ok with taking the public out of public education.

A system where every school can systematically discriminate based on wealth is not one that I want to be a part of.

Is this is where the voucher movement is heading, count me out.

If, on the other hand, the voucher movement is really about innovation, entrepreneurship, and family empowerment – then count me in.

Lastly, I have a ton of respect for people on all sides of this debate, so if I’m mischaracterizing anyone’s views, I’ll update the post.

But, admittedly, I found some of my voucher friends making arguments that, to me at least, were pretty unconvincing.

 

The Parable of the 3 Blacksmiths

Unknown

During a time long ago and in a kingdom far away, there once lived three blacksmiths.

All three blacksmiths worked under their lordship, Sir Tuda.

Sir Tuda was a benevolent lord. He wanted his blacksmiths to be as productive as possible, both so they and the kingdom could prosper.

Sir Tuda knew the old adage  – “a kingdom is never better than its blacksmiths” – and he took it to heart.

Unfortunately, Sir Tuda was not schooled in public policy, so instead of issuing a proclamation on his own, he wisely approached his top blacksmiths and asked them how to increase production.

“Tell me what you need from me, and I will make it so,” he told the blacksmiths.

The first blacksmith said: “I need autonomy! If you give me the freedom to run my shop the way I desire, I will increase production!”

Sir Tuda said: “Well then it is so.”

The second blacksmith, wanting to outdo the first blacksmith, said: “I need autonomy too! But I also need a board of directors to guide me!”

Sir Tuda said: “Well then it is so.”

The third blacksmith was actually not yet a full blacksmith. She was only an apprentice and she would not have even been invited to this meeting but for the fact that her boss was sick with the plague.

She like both of the requests that had already been mentioned, but what she really wanted was to start her own blacksmith shop.

So she said: “I want autonomy! And I want a board of directors to guide me! But I also want a charter to open up my own blacksmith shop!”

Sir Tuda said: “Well then it is so.”

Five years then passed and much changed, including the death of Sir Tuda, who was killed by the Kingdom of the Strategic Inconsistency, which had a long history of invading kingdoms, even those run by great lords.

So what happened to our blacksmiths?

The first blacksmith, who had asked for autonomy, did see a spike in production for two years, but when Sir Tuda died, the new Lord took away the autonomy and production decreased back to its previous levels.

The second blacksmith, who asked for autonomy and a board of directors, increased his production a modest but statistically significant amount; moreover, his board of directors protected him from the new Lord, so he was able to maintain his production increase.

The third blacksmith, who asked to be able to start a new blacksmith shop, gain autonomy, and be overseen by a board of directors, saw her production skyrocket. The young blacksmith had been experimenting with a new innovative method of blacksmithing, and it was only once she got her  own shop that he was able to implement his new method. And, like the second blacksmith, her board of directors protected her from the new Lord’s top-down blacksmith policies.

Lastly, and oddly enough, the new Lord kept on trying to prevent the third blacksmith from expanding even though her shop was so successful. The new Lord kept on muttering, “you’re stealing from me” even though all the blacksmith shops equally benefited the health of the kingdom.

But, by organizing and mobilizing her consumers, the third blacksmith was able eventually open up ten more highly effective blacksmith shops, which made the kingdom the number one in production in the world, even surpassing the Kingdom of Finland (I swear it’s true).

Wise readers will see the moral of this story.

Blacksmith autonomy is useful but fleeting.

Blacksmith autonomy coupled with non-profit governance will lead to modest but important improvements.

Blacksmith autonomy coupled with non-profit governance coupled with entrepreneurship can transform the kingdom.