Category Archives: School Choice

Families in Washington D.C. are conflicted between what works and what they want to work

How do you figure out what parents want? 

There are two primary ways to figure out what families want from schools:

  1. Ask them what they want;
  2. Or observe what decisions they make.

The Washington D.C. Auditor’s office just released a public opinion report using the former method (asking families). Washington D.C. has a unified enrollment system where you can track what decisions parents are actually making. I wish this report would have used this data as well (like this report did in New Orleans). But despite this flaw, the opinion research is telling.

Families in D.C. are conflicted between what works and what they want to work 

D.C. families had the following preferences:

  1. School preference: Families say they value educator and academic quality much more than a school being close to home.
  2. School type preference: Families prefer charter schools over their local neighborhood school.
  3. Policy preference: Families want to invest more in neighborhood schools rather than giving more parents chance to opt into a charter or out of boundary traditional school.

Putting this together: families want public schools with great educators and challenging academics; they think charters schools are more likely to provide this; but they want more money put into neighborhood schools.

Of course, this on average. No doubt there is immense diversity in opinion across families.

Why do many families want to fix what’s not working rather than expand what is working? 

I can think of a couple reasons:

Empathy and bad strategy: Families feel bad for the kids who are stuck in the worse schools, and the most intuitive answer to “what should we do with more resources?” is “fix what’s not working.” In the private sector, fortunes have been lost on this fallacy. In the public sector, fortunes are spent on efforts that have failed for decades.

Virtue signaling: Families want to express that they are good people and not selfish, and in our society saying you want to invest more in neighborhood schools is a way to signal that you’re a good person.

They want their neighborhood schools to improve, they just don’t want their kids to suffer in the meantime: Perhaps families would rather send their own kids to the local neighborhood school, so they do want these schools to get better, but they’re just not willing to send their children there while they wait for this to happen.

I imagine all these factors are at play.

How do you respond?

I think this issue will continue to play out in high choice cities that are providing families with a lot of great public school options.

When it comes to their own children, families will, on average, send their children to the best public schools.

But when it comes to public opinion, families will say we should invest more in the schools that aren’t working.

This puts public charter school supporters in a very tricky position.

How can we try to create more great public schools under this dynamic?

I think charter supporters need to be very cautious in engaging in major, public citywide or statewide fights.

We can win an individual families’ hearts and minds when it comes to their own child, but the policy battles are tougher.

In other words: just keep on opening amazing new public charter schools. The system will become the best version of itself through educators opening up great public schools and families finding the best fit for their own children.

But it is also worth considering pacing: too much growth too quickly can shift an individual school opening debate into a citywide policy debate – a type of debate that is easily lost, as it was in Massachusetts.

Lastly, gradually opening up new great public schools will ultimately give families both what they want for their own children and what they want for the city: if every public school is great, than by default every school in every neighborhood will be great too.

Data from the report

Screen Shot 2018-07-14 at 9.15.30 AMScreen Shot 2018-07-14 at 9.15.03 AMScreen Shot 2018-07-14 at 9.14.55 AMScreen Shot 2018-07-14 at 9.13.13 AM

Most students in NOLA whose school is closed end up in one of their top choices for the following year. Here’s how.

This post from Ed Navigator is worth reading. It covers schools closures in New Orleans.

Over ten years after Katrina, and under an elected school board, New Orleans continues to selectively close underperforming schools.

I view this as a good thing, given the growing body of research that shows that school closures help kids when the students end up in better schools.

New Orleans uses a unified enrollment system to help kids get into better schools.

The unified enrollment system gives preference to students whose schools were closed the year before. If your school was closed, the algorithm bumps you to the top of the list for any school you want to get into.

Ed Navigator works with families whose schools have been closed, so that they can help select great schools.

The result?

This year, 87% of students who attended a closing school and used the enrollment system received on of their top three choices for the next school year.

94% of the students will now attend a school that is rated higher by the state’s grading system.

The system is by no means perfect. My biggest critique is that the state’s grading system still relies too heavily on absolute test scores (rather than growth). I also understand the counterarguments that government should never close schools and should instead let enrollment patterns (driven by parental choice) determine which schools grow and which close.

But I would rather have the New Orleans enrollment and closure system than just about any other big city system in the country.  In too many cities, really bad schools stay open for too long. And if anything happens to them, kids often end up in schools that are just as bad.

This is not what happens in New Orleans.

It’s also great to see Parag Pathak (and his colleagues) work in action. Parag recently won the John Bates Clark award in part because of his contributions to working on unified enrollment systems.

It’s rare that an idea goes from the ivory tower to think tanks to actual implementation by a democratically elected body to  helping citizens.

This is really great to see. And really great for kids in New Orleans.

Quick feedback for the New York Times

Screen Shot 2017-05-05 at 8.44.11 AM.png

The NYT just published an article on NYC’s school choice system.

The article is worth reading for its qualitative insights into what it’s like to navigate the system. I have deep empathy for families that struggle to find great schools for their children. They deserve much better.

But the framing of the piece is flawed, and I hope other journalists don’t repeat this mistake in future articles.

The authors argue that school choice has not delivered on its promise because many students still don’t have access to great schools.

But school choice does not increase the supply of great schools; rather, it is a mechanism to allow families to choose from schools that already exist.

School choice is about access and fairness. You can assign families to schools based on their address, or you can try to create more just systems. I strongly believe we should do the latter.

But increasing equity of access will likely not lead to dramatic jumps in quality.

It is only be creating new schools, scaling the best schools, and improving existing schools that quality increases. This is not the job of a citywide enrollment system.

Moreover, if you increase access but restrict supply you well get frustration. And this is exactly what has happened in New York City. The city’s enrollment system persists, but its efforts to increase supply have faltered.

When NYC leaders have focused on increasing supply – both through the small schools movement and growing the charter sector – rigorous research found that school quality increased. The results of these efforts are detailed below.

In sum:

School choice is all about equity in access.

School supply is about creating better options.

We should not confuse the two, and we should not expect school choice to increase school performance in and of itself. It must be coupled with a deep focus on school supply.

Small schools results:

Screen Shot 2017-05-05 at 8.58.15 AM

Charter results:

Screen Shot 2017-05-05 at 8.53.01 AM.png

 

 

 

What if unified enrollment platforms were 10x better?

An emerging group of cities – including Washington D.C., Newark, Camden, New Orleans, and Denver – have adopted unified enrollment systems. With these systems, families can enroll in schools across the city via an online application system.

This is a huge step forward. For too long, parents have not had enough information or access to the public schools in their cities.

However, the new enrollment systems are still in their infancy. The best version of these systems could radically improve public education. Unfortunately, we’re very far from this endgame.

I. Early Wins: Access, Equity, and Ranking

Access: With the best open enrollment systems, families who can’t afford a house in a fancy neighborhood can now finally transparently apply to a school in a more wealthy neighborhood.

As a result of increase in access, a recent study in Washington D.C. found that the new enrollment regime would likely reduce segregation over time:

Screen Shot 2017-03-17 at 9.23.22 AM

Equity: In most cities, traditional and charter schools play a ton of games with enrollment. Traditional magnet schools use opaque entry requirements. Bad apple charters don’t take in kids with special needs. There is no equity.

With online enrollment platforms, these problems go away, as schools are no longer in control of their enrollment.

Quite simply: the algorithm is fairer than the enrollment clerk.

Ranking: These new enrollment systems also allow parents to rank their top schools. This is extremely important.

First, a family’s high desire to enroll their child in a school can now  be translated into an increased chance that they actually get into this school.

Previously, high desire meant little unless you were connected, wealthy, or dogged.

Second, ranking allows  parents to publicly signal to government which schools are most and least in demand (which will ideally affect opening, expansion, and closure decisions). It also signals to school operators what attributes make a school in high demand.

By analyzing ranking preferences, researchers in New Orleans were able to correlate school characteristics with parent preference:

Screen Shot 2017-03-17 at 9.44.39 AMRanking transforms family desire into actionable information.

II. Unified Enrollment Systems are Mediocre Platforms

In preparation for writing this blog, I spend an hour on unified enrollment system websites. It was not a great experience.

Here is the school finder homepage from Washington D.C. – I couldn’t even find a way to filter schools by academic performance!

Screen Shot 2017-03-17 at 9.50.14 AM

Here is the search function for Newark’s enrollment system – you have to download a pdf!

Screen Shot 2017-03-17 at 9.55.36 AM

By comparison, here’s the search page from Zillow:

Screen Shot 2017-03-17 at 9.53.10 AM.png

On Zillow, you can easily search for homes based on the most useful search criteria. Yes, Zillow is surely better capitalized than your average enrollment system, but even with modest funds a city should be able to do better than a downloadable pdf.

III. Moving From Equity and Ranking to Matching and Prediction

More sophisticated uniform enrollment could offer two extraordinary improvements: they could better match families with schools, and they could better predict how any given student would do at a school.

Matching: Right now families mostly use enrollment systems for ranking: they know the schools they want and they use enrollment systems to express this desire.

What is not really happening (as far as I can tell) is sophisticated algorithms actually helping families match with schools.

For example, on Camden’s enrollment site (where you can thankfully filter by academic performance!), I found three schools that all met the “on track” performance criteria, and pulled up the comparison page:

Screen Shot 2017-03-17 at 10.11.14 AM

This does very little to help me choose between these schools. My preference variables are limited to very broad categories such as “art classes” and “after school programs.”

After playing around on these websites, I get very little indication that that the platform knows me or the schools very well. Compare this to Netflix, Amazon, or dating websites (previous life) – platforms where I feel like the algorithms know me better than myself.

Unified enrollment systems need to more deeply understand children and schools in order to make better matches.

Prediction: Right now, government accountability systems are a basket case of poor design (generally don’t weight growth enough), brutal politics (what politician wants to tell a bunch of communities they only have “F” schools?), and awful transparency (good luck trying to navigate your average state department of education website).

Most importantly, government accountability systems evaluate schools rather than make predictions.

As a parent, it’s one thing to tell you that a school is a “C+” – it’s another thing to give you a prediction of what will happen to your child if she attends the school.

With current date, we could probably gather basic information on your child’s age, gender, current academic performance, personality type, etc.,  and make a reasonably accurate prediction that if she attends school X she will have a Y% chance of graduating from high schools and a Z% chance of earning a post-secondary degree.

Good enrollment systems, over time, should become better and better predictive agents, and, perhaps, can end up augmenting (displacing?) government accountability systems.

IV. Root Causes and Potential Solutions 

I don’t yet have strong beliefs about the root causes of why these enrollment products aren’t getting better faster. But here’s some guesses:

Non-profits > government operated: Most of the enrollment systems are run by governments, which are not good at running tech products and have bad incentives around giving parents accurate information about schools. Non-profits would likely be better operationally and have better incentives, and avoid the privacy concerns associated with for-profits.

Lack of scale: Matching and predication can better with bigger data sets, and if all these systems are structured as isolated city based data silos, the algorithms will be dumber than they should be.

Weak Customer Demand -> Bad Economics: SchoolMint, from what I understand, is the most successful player in the market. For reasons I don’t underhand, this company has not developed a better product. Perhaps it’s because their government customers don’t actually want it. Or perhaps the economics don’t work (which might suggest philanthropy is needed).

If the above is true, a national non-profit should be backed to scale to enough size to create smart algorithms, and it should be financially structured in a manner that gets it out of the perverse incentives of being beholden to government or individual schools rather than families.

A philanthropic foundation with a great tech backbone could be well situated to support this endeavor.

V. Expectations

Better matching and prediction would probably not make the average student’s education experience 10x better, just as dating websites don’t inevitably lead to great marriages.

But I do think better matching and predication could increase the probabilities that millions of families could find a better fit for their children.

At scale, that’s a better world.

When It Comes to Schools, Who Knows Best?

Screen Shot 2015-12-09 at 4.53.13 PM

Traditional public education systems assign students to schools based on postal addresses.

This can be viewed as either a feature or a bug of public schooling.

Postal address assignment can be viewed as a feature if one holds either of the following opinions: (1) school district officials will be better choosers than families; or (2) the costs of allowing families to choose individual schools are too high to the system as a whole.

I view postal address school assignment as a horrible bug.

___

I have no illusion that families will always choose a great school for their children. But this is not the right question. The right question is whether or not, on average, families will be better choosers than district officials.

Two pieces of evidence make it very clear to me that families will, on average, choose better than district officials.

First, to date, district officials (who do not give families choice) have not improved on their century old algorithm of assigning schools based on geographic proximity. If this is the best that they can come up with, then good riddance.

Second, district officials continue assigning poor black and Hispanic families to terrible schools. If they cannot see that their algorithm is terribly racist, then good riddance.

As for the second issue – that the cost of giving choice to families is too high – New Orleans, Denver, and Washington D.C. have already demonstrated that this concern can be overcome. Rather than choice negatively hurting these schools systems, choice has been expanded while these cities have seen significant gains in overall achievement.

____

One last issue: should we do anything if families choose to send their children to terribly failing schools?

Yes. I think that we should transform or close these schools.

Here, an analogy might be made to health inspections – even if a restaurant has a line out the door, if it is dishing out salmonella the government will close it.

While it’s a complicated issue, I do think there are instances where the government should override family choice.

___

Fordham just released its analysis of school choice in cities across the country (the Laura and John Arnold Foundation helped fund the work).

Here are the results.

I am glad they did not grade on a curve.

This would have masked the brutal reality that most district officials do not trust families to choose schools for their own children.

Screen Shot 2015-12-09 at 5.13.21 PM

 

 

 

A Debate Within the Family: To Regulate or Not?

Jay Greene had a series of posts on choice regulation over at his blog.

His overarching argument: regulating school choice does more harm than good.

Broadly, I think Jay makes a number of good points. I also think he overstates his case.

More specifically, I think his arguments are somewhat strong on performance and pretty weak on equity.

I also think that Jay could be more conservative on how he generalizes fairly narrow research findings, especially given how hard he is on others who misuse research!

Overall, Jay made me think harder about how philanthropists should allocate resources across choice interventions. He might be right that there is too much attention to charters. I think if voucher proponents were more serious about equity regulation they could help shift the focus. I’d be happy to work with Jay and others on this. Nevada and other pilots that attempt to achieve scale could be fruitful places to partner and learn more.

Below I tackle his main points.

Government funding does not require performance oversight.

Jay notes that cash transfer programs (such as Social Security) do not come with government performance contracts. This is true.

Jay also notes that food stamps don’t come up with performance contracts. In his words: “When the government provides food stamps it does not require recipients to submit BMI measurements or other indicators of adequate nutrition.”

This is true, but it’s not a perfect example, as the government does require the providers of food to meet performance input targets (food must have some nutritional value). The government does not trust the consumer to make his or her own decisions, so choice is restricted based on the nutritional performance of the food provider’s product.

While there is not accountability for outcomes, there is heavy regulation of inputs.

More broadly, we are seeing more and more performance accountability in government health programs (Obamacare looking at things such as readmission rates) and government post-secondary programs (requiring schools that receive Pell grants to achieve certain performance targets).

Jay overstates his case when he says: “Repeating that government funds require accountability to the government is just mindless sloganeering, not an accurate description of how government programs typically operate.”

Yes, education testing is probably on the more regulated end of pay for performance. But the issue is a complicated one, and most government programs (except for direct cash transfers) regulate inputs and many also regulate outputs.

There is a vast academic literature on these various approaches.

Whether or not to regulate for outputs is worthy of deep debate, not outright dismissal.

Personally, I’m probably closer to Jay on this than the average charter supporter. I think that, over time, not regulating for performance would likely work out. In my previous writings, I’ve often said that I’d make the “all choice for no testing” trade with the far left and the far right.

But I think the issue is complicated. And, as I note below, I’m fairly comfortable with the government putting in a performance floor and closing down the very worst performers.

Test Scores are Limited Tools; Attainment is a Better Proxy for Quality than Test Scores; Vouchers Do Better on Attainment

Jay rightfully points out that test don’t measure everything: things like grit and conscientiousness are likely very important to lifetime outcomes and it is very difficult to capture these in tests.

Matt Barnum did a good response on this. In short, numerous studies have tied increases in test scores to increases in long-term outcomes. None of these studies are a slam dunk, and causation can be tricky, but there’s enough here to make a case that achievement tests measure some of what we want schools to be doing.

For these reasons, I think there’s a reasonable argument to be made that performance accountability, if it is to be used, should be used as a floor rather than the end all be all of school grading. Perhaps “A-F” systems should just be “F” systems.

Jay then argues that because testing doesn’t measure everything, attainment is actually a better measure of school quality.

Jay might be right, but it’s complicated. Yes, achieving a degree (be it high school or post secondary) will likely increase a student’s life outcomes. But the more this degree attainment is divorced from knowledge attainment (and test scores), the weaker this effect might become over time (unless employers really just care about conscientiousness, which may very well be the case).

All told, the international evidence on “schooling ain’t learning” is robust.

That being said, in terms of policy, I don’t really have much to disagree with here. As I noted above, I have mixed feelings on performance accountability as it is, and an increased focus on attainment might be very healthy for the charter community.

Jay then makes the case that vouchers do better on achievement than charters. Jay cites three studies, two of which find higher degree attainment. The Milwaukee study found increased high school degree and college persistence with voucher students, while the DC study found voucher students had increased high school graduation. The NYC programs only increased college attendance (I didn’t see any evidence on graduation). Moreover, the NYC program was only a partial scholarship, which as far as I can tell required families to pay for a portion of schooling (which makes it hard to generalize to families that could not afford to pay). The DC voucher program was fairly small in scope as well.

This does not seem to be an evidence base from which one can make strong, generalized claims about what voucher effects would be on a systems level. Only the Milwaukee and DC programs come near providing a full voucher program, and only the Milwaukee program got to real scale.

As Jay points out, the most rigorous charter research finds positive effects on test scores rather than attainment. But the evidence on test score gains is massive: CREDO has studied dozens of cities and have found an overall effect of ~.1 for urban areas serving hundreds of thousands of students. This research includes three markets where charters serve roughly half of all students or more (NOLA, DC, Detroit). And the impact on New Orleans attainment has been very significant (high school graduation rates are up twenty points). Of course, systems level evidence is  not randomly controlled, but this doesn’t mean that it’s not useful; in many instances, it’s probably more predictive than small RCTs.

If I had to bet on which intervention is most likely to work at scale, I’d be inclined to bet on a massive data set that found positive effects on test scores rather than a very narrow data set of three studies where only two study found higher degree attainment.

Of course, I might be wrong. But Jay has surely not proven his case. The evidence he cites covers small scale studies that make it difficult to generalize.

We Should Not Regulate Choice Programs for Equity

Jay first notes that heavy regulations –  “such as mandating that schools accept voucher amounts as payment in full, prohibiting schools from applying their own admissions requirements” – decreases the number of participating private schools.  This is clearly true.

Undoubtedly, more private schools will enroll voucher students if they can screen out students with behavioral issues and very low income students (which is what admissions requirements and pay-sharing would do).

In arguing against regulation, Jay writes: “But real education reform requires using the power of choice and competition to provide incentives to create more good and to reduce bad.”

Yet Jay’s version of education reform would clearly incentivize schools to not enroll students with behavioral issues or children from very low incomes. Why enroll a student who is hard to serve or who can only pay partial tuition?

This is the biggest flaw in Jay’s argument.

Jay is probably right that the performance market would correct itself over time.

It is very unclear to me that this would be true for equity.

It was surprising to me that Jay did not even find this obvious rebuttal worth mentioning.

Voucher proponents such as Jay would do well to think hard about reasonable equity regulations that ensure that decentralized choice markets offer good educational opportunities to all students.

Charter markets are moving toward unified enrollment and expulsion processes as easy to regulate for equity. Perhaps these processes are too heavy handed.

But I think some equity mechanisms need to be put in place.

Of course, you can make the argument that it is ok to have unequal public educational access so long as this system lifts all boats in the aggregate. Given that I believe that equity in access is a principle in of itself, it would take fairly large overall achievement gains for me to be willing to compromise so hard on equity.

Concluding Thoughts

There does seem to be enough in theory and evidence to support larger scale pilots of voucher programs.

This is why many people, charter school supporters included, are eager to see what we learn from Nevada.

For whatever it’s worth, my two major critique of the Nevada voucher were that the voucher amount was not enough money and that equitable access safeguards do not seem to be in place.

But overall I think it’s a very important breakthrough for choice.

So if the goal is more experimentation at scale, sign me up.

But in this experimentation I’d just argue for some basic equity guardrails so that choice is available to everyone, including the most disadvantaged.

Lastly, thanks to Jay for putting forth a good argument. It’s well worth grappling with.

Families Choose Between Schools, Not Talking Points

students

Normandy High honors student Cameron Hensley (left), a senior, checks his phone as some students sleep away physics class on Thursday, April 30, 2015. Though the teacher was inside the classroom the entire class period, no instruction occurred. Instead, the teacher and many students spent the hour on their phones or practicing for the dance team’s upcoming parade.

I was recently talking to Mike Goldstein about the NYT Success Academies piece.

One could read the article, which painted the schools as fairly oppressive, and struggle to understand why 22,000 families applied for 2,700 open slots.

Mike raised the issue that it is odd that we continue to see two stories play out in the media: (1) detractors decry high-peforming charter schools for test prep, strict discipline, and privatization; and (2) parent demand for these schools is through the roof.

Mike and I think this is another appearance of the Nirvana Fallacy: detractors critiquing something for its imperfections while over idealizing alternative options.

Our primary lens of understanding high-performing charter schools should not be by comparing them to Nirvana; it should be by comparing them to other available options.

Mike noted that opinions might not be so polarized if detractors actually witnessed what happens at underperforming schools, both in the classroom, as well in social places – the lunchroom, recess, and, yes, the bathroom. The intellectual apathy and the physical and emotional bullying can be heartbreaking.

For a window into what parents might be choosing between, read this piece on Normandy High School:

“Third period is nothing to expect,” she said after she sat down. “She doesn’t take attendance. No work at all. No intent to do any work.”

The instructor, Ivy Word, sat in the front corner looking at her computer screen. She’s a substitute teacher who’s been in charge of the class since the start of the school year.

Some students sat at the back of the room, ear buds in their ears. Some slept. Carver and four other girls talked about prom. Carver took out a comb and began braiding a classmate’s hair. Another girl began gluing fake eyelashes.

In classes where teachers have given up, students have, too. They spend the hour texting friends, snapping photos and sending them by social media.

This is the reality that families living in poverty often face. They are not choosing between dueling narratives; they are choosing between schools that may set their children on the path to college or schools that may push their children into prison.

To their credit, the New York Times also did a piece sharing six parent perspectives.

But here’s what I would have loved to see: interviews with the 22,000 families who tried to get into Success Academies.

This might have given us a better insight into how families, unlike so many commentators, understand that the choice is not between high-performing charter schools and Nirvana.

Rather, the choice is between schools, that, for all their imperfections, do well by children – and schools that do not.