Category Archives: Accountability

Response to Matt Ladner and Jay Greene

Over at Jay’s blog, Jay and Matt wrote two critical posts of portfolio management and harbormasters (our team calls them quarterbacks).

Jay and Matt think differently than I do, have different political orientations, and are sharp. Their writing makes me smarter.

I also find their post titles, writing tone, and evidence analysis to be a bit over the top. They sometimes overstate their claims and under appreciate the other side of the argument.

In these two posts, Jay and Matt use NAEP charter sector gains in Arizona, Michigan, and Texas – as well as the mediocre NAEP scores seen in Louisiana’s charter sector – to argue that portfolio management and quarterbacks aren’t working.

I found their analysis to be overly narrow. Instead of taking some new evidence in and synthesizing this with the broad set of evidence available, they anchored on to one set of data points and made too strong of claims (especially in the titles of their posts).

Don’t Look at NAEP in Isolation 

Matt is right to point out that some states with fairly loose charter regulations saw a lot of charter gains in NAEP between 2009 and 2017.

I think this should modestly increase our belief that being loose on charter openings and closings can lead, over time, to a healthy charter sector.

But the story is not that clean.

This CREDO paper, which looked at charter school performance in Texas between 2011 and 2015, found a small positive effect in reading and no effect in math. Given that CREDO tracks individual students across time, and NAEP does not, the CREDO data should make us cautious in interpreting the NAEP gains as a huge victory.

Screen Shot 2018-04-24 at 9.57.03 PM

Another study found that it took Texas charters ten years to achieve the performance gains of their traditional school peers.

While it’s great to see the sector improving in Texas, perhaps with better regulation the sector wouldn’t have had to improve for ten years just to achieve neutral effects.

The story of the Texas charter sector is much more complicated than Matt’s piece indicated. The same is true for Arizona and Florida, where quasi-experimental research has found muted test score effects.

Given the different results of NAEP and CREDO, we should be trying to figure out the puzzle rather than claiming victory, as Matt tends to do.

Don’t Look at States in Isolation 

Looking at state level gains is not a great measure of whether portfolio and quarterbacks are working, as portfolio (to some extent) and quarterbacks (almost always) are city based endeavors.

For city based data, this CREDO study measured student learning effects in a bunch of cities across the country.

Washington D.C., Denver, New Orleans, and Newark all did very well. These cities are also some of the most mature portfolio cities.

Phoenix, Austin, El Paso, Forth Worth, Mesa, and San Antonio did not do well. These cities are all found in loose regulation states.

It hardly seems like a slam dunk to me that portfolio / quarterbacks are bad and loose regulation is good.

Of course, the CREDO analysis is not perfect: test scores aren’t everything and the virtual twin methodology may miss unobserved differences between students.

But looking at state based NAEP scores to make broad judgements on portfolio and quarterbacks is unwise, especially with so much other evidence available.

The portfolio / quarterback model seems to be doing some good in many cities.

Quarterbacks are a Step in the Right Direction  

Jay and Matt often criticize quarterbacks as vehicles for people who think they are smarter than everyone else (especially educators and families). I find this to be an overly simple critique.

Quarterback originated as a way to use expertise to aggregate and allocate philanthropy.

In many cities, philanthropists were funding low impact activities, often wasting it on the  pet projects of district leadership. A lot of money was spent for very little academic gain.

Quarterbacks have helped improve philanthropy: instead of just passively giving money to the district, philanthropists partner with expert management teams to try and launch and grow great non-profits.

I think this is a major improvement on the status quo. Of course, there are some drawbacks, and too much centralization of philanthropic capital poses risks. This is why I don’t think all of a city’s philanthropic capital should flow through one organization.

But quarterbacks are increasing, not decreasing, educator entrepreneurship and family choice. Yes, they do often use test score results selecting who to fund, but I suspect this will change if a better way to invest is developed over time.

In Sum

The NAEP data should not be ignored. It’s made me more open to the idea that looser regulations can lead to charter scale and quality, especially at the state level. And I found Matt’s data analysis to be quite helpful. I love it when smart people who think differently than me play with complex data sets and come to novel conclusions.

But I think there’s plenty of other state based evidence that should make us cautious, such as the CREDO Texas study.

I also think there’s a lot of evidence that the charter sectors in portfolio / quarterback cities are making a lot of gains. The NAEP data Matt and Jay site, which is state based and does not track individual students, is not convincing enough to make me deeply question our city based work.

All that being said, I look forward to reading more from Jay and Matt in the future.

I don’t take smart, critical friends for granted.

What is California telling us about what parents want from public education?

I sit on the board of the California Charter School Association (CCSA), which is one of most effective charter associations in the nation.

Their data team put this slide together.

It’s a little complicated, but it’s very informative.

Screen Shot 2018-04-05 at 8.08.33 PM

To build the chart, CCSA looked at schools that are serving the same type of demographics and then compares their test scores.

A school that outperforms similar schools, gets a higher score (8-10 is really good); a school that underperforms similar schools, gets a lower school (1-3 is pretty bad).

This methodology – absolute test scores controlled for demographics – is imperfect, but it’s the best methodology you can use given California does not publicly report student growth scores.

Charter schools serving students in poverty outperform district schools on state tests 

43% of charter schools with higher concentrations of poverty outperform similar schools.

In California, if you are a low-income Hispanic or African-American child, you are more likely get a better education (as measured by test scores and parent demand) if you attend a charter school.

This is good news for the educators and families who are working together to create better educational outcomes in at-risk communities.

If parents are demanding schools with good test score impacts, the government’s response should be obvious: let more of these schools open.

Charter schools serving middle class students underperform district schools on state tests 

But not all charter schools are outperforming their peers on tests scores.

47% of charter schools serving middle class students perform worse than similar schools. And only 26% perform better.

So perhaps California should close some of these charter schools that serve middle class families? Research covered in this post shows that schools with negative test impacts tend not to have large positive life outcome impacts.

But here’s the odd thing: presumably, middle class families have a decent amount of information at hand when making school choices. It takes all of two minutes to scan Zillow or Great Schools to get a quick read on the absolute test score performance of any school in the state.

So why are all these relatively well resourced families sending their children to lower-performing charter schools as measured by state tests?

I’m not sure. It would be interesting to focus group and poll them to learn more.

And with regards to closure, while I surely disagree with middle class Californians on many policy issues, I’m not sure that I think I know enough about their children to close schools that have modest negative test impacts but high enrollment demand.

Charter schools don’t exist unless parents want their children to attend them 

One of the best features of charter schools is that they don’t exist unless parents choose them. No one is assigned to a charter school.

So what are we to make of this data where families in living in poverty are choosing schools with positive test scores impacts and middle class families are choosing schools with negative test score impacts?

I think the starting point should be to assume that families, on average, are in a better position to make an informed choice than government is.

Remember, government’s default assignment algorithm is to look at your family’s address and then assign your child to the nearest school. It’s not very nuanced!

My guess is that parent choice will outperform geographic assignment when it comes to finding great fits between kids and schools.

But I do think we should be open to the idea that parents, sometimes en masse, can make mistakes. And, at times, this can warrant government intervention.

Sometimes performance might be an indicator, such as when families keep sending their children to high schools with below 40% graduation rates and /or schools with extremely negative value-add scores. If less than half the kids are graduating, and those that do are barely literate, government should step in.

Sometimes lack of alignment with our nation’s professed values might be an indicator: certain public schools have at times been captured by groups, sometimes religious, that do not teach basic democratic values.

In these cases of significant performance or culture malfeasance, government should consider intervention, ideally by handing over management of the school to a non-profit organization that can achieve better results.

I don’t know enough about individual school performance to know if what’s happening in California with middle class families equates to education malfeasance, but I’m a bit skeptical.

My hunch is that once absolute test score levels surpass a certain floor (as they tend to in middle class schools), families just care a lot about other factors.

Even if I might make a different choice, I don’t know that the situation warrants government intervention.

When government should not intervene 

While it’s difficult to decide when government should intervene, it’s still pretty clear to me when government should not intervene.

When schools have both high demand and high test score impacts with students living in poverty, government should not prevent these schools from serving more students!

California should follow this common sense policy.

Unfortunately, too many school districts do not.

SaveSave

SaveSave

Does anyone know why Chicago children are getting smarter?

If you just read the headlines, you might suspect that Chicago’s public schools are in a terrible tailspin. Part of this is the noise of big city politics. Part of this stems from cloud of violence that hangs over the city.

But Chicago has improved on academic test scores more than most other cities in the country. Rather than one of the worst, Chicago is one of the best.

A recent report by Sean Reardon and Rebecca Hinze-Pifer found that between 2009 and 2014:

“This [student achievement] growth rate [of Chicago] is higher than 96% of all districts in the US. Among the 100 largest districts in the country, the average growth rate from third to eighth grade is 0.95 grade equivalents per year; Chicago has the highest growth rate between third and eighth grade of any large district in the United States.”

The authors admit that they don’t know why this occurred.

I can’t prove why Chicago kids are getting smarter, but I have a hypothesis.

What’s Been Going on in Chicago Public Schools? 

One way to try and solve the mystery of why Chicago children are getting smarter is to look at the district’s previous major initiatives.

As this report details, between roughy 1990 and 2010 there were three overarching eras of reform in Chicago: the decentralization era, the the accountability era, and the do a lot of things era.

The authors are very careful to not attribute a causal relationship between reform eras and outcomes. The reforms were messy and not rolled out in an experimental manner – so fair enough.

But in this post I’ll try and make my best guess on what was causal and what was not.

The Decentralization Era

The decentralization era was best known for the creation of Local School Councils. This reform gave local councils real control over decisions about how schools were run. The councils were made up of school leadership, parents, and community members.

The councils always seemed like a terrible idea to me. It’s basically taking all we know about charter schools (good central offices, scalable instructional programs, governance matters) and doing the exact opposite!

Not surprisingly, research on the reforms found that the councils had some positive effects on advantage communities, but were least likely to improve schools in low-income communites. Communities with low social capital didn’t gain a lot from ad-hoc and poorly constructed local boards.

I’m very skeptical that the decentralization era and school councils were the root cause of later gains.

The Accountability Era

In 1995 Mayor Daly put in Paul Valls as the superintendent (I later worked with Paul when he was the superintendent of the RSD in Louisiana). Vallas, who did not have deep instructional expertise, used test driven accountability to try and make things better.

New tests, promotional standards, and interventions for failing schools were all put in place.

The reforms had better impacts for low-performing schools; the researchers noted:

“This was the only era to show large improvements in the lowest- achieving schools. However, the patterns in test scores in the lowest-performing schools suggest that some of the improvements resulted from instruction that was aligned specifically to the high stakes tests.”

This matches other research on accountability reforms: you tend to see gains in the lowest preforming schools, but the high stakes can cause narrowing of the curriculum.

 

The Do a Lot of Things Era

Arne Duncan came in after Vallas, and he instituted a lot of reforms.

Arne launched 100 new schools, implemented internal district instructional and curricular reforms, overhauled school leadership pipelines, and placed a deep focus on on-time high school progression.

Perhaps the biggest initiative of this era was the Renaissance 2010 project, which launched about a 100 new district, charter, and contract schools between 2005 and 2010.

Unfortunately, no one has conducted a full evaluation of the program. Someone should do this!

Two interim research reports came out around 2010. One study, which only included a few years of data from the early Renaissance cohorts, found that the new schools performed about the same as the existing district schools. The other study was inconclusive.

Not much help from the research community.

A lot of work was also done on school leadership. The Chicago Public Education Fund, in partnership with the district, invested heavily in school leader development, placing bets on both district based and non-profit providers.

The lastest research I could find on these programs found that “results indicate that one-year learning gains in elementary and high schools led by Fund-supported principals were not different than those in other similar schools.”

Another major reform, another mediocre result.

All told, researchers found that this era produced more gains in high school than elementary schools, but wrote: “while the effects of the dominant policies of Eras 1 and 2 are largely understood, much research remains to be done to understand both the positive and problematic effects of the policies in Era 3.”

Not super helpful, especially since this is the era that preceded the large gains in test scores that occurred after 2009.

What About the Charter Sector? 

CREDO published a report on Chicago charters that covered test scores from 2010 to 2012, which is right in the middle of the period where Chicago saw a lot of gains.

The study found +.01 effects in reading and +.03 effects in math. These effects amount to about a month or so of extra learning per year, maybe a bit less. Given Chicago’s relatively small charter market share, and the modest size of these positive effects, it’s unlikely that charters themselves accounted for the 2009-2014 gains.

A more recent study, which just looked at charter high school performance from 2010-2013, found much larger effects: +.2 effects on ACT related tests and much higher college enrollment rates.

These are large effects, but they are for high school only. The study lauding Chicago’s gains only covered grades 3-8.

So WTF Happened in Chicago to Make Kids Smarter?

To summarize: Chicago improved its test scores more than any other big city in the country, and researchers really don’t know why.

So why are Chicago kids getting smarter?

Here’s my guess: competition and accountability lifted all boats.

When you put accountability in place (the Vallas era) and then launch a 100 new schools (the Duncan era) you get a city where school leaders know there are consequences for failure and the best of the new schools begin to raise the bar for what’s possible.

This theory helps explain why the Renaissance schools and charter effects were a bit muted. In the studies on these reforms, researchers compared the new schools to existing schools. So if the existing schools were improving due to increased competition, you would not see large relative effects for the new schools.

I can’t prove that accountability and competition caused the results, but in many sectors accountability and competition make everyone better. It also fits stories we’ve seen elsewhere. In place like Denver and Washington D.C. increased competition led to all boats rising in the public school system.

If you have a better theory, let me know.

What Should Chicago Do Now? 

Here’s another tough question: if it was accountability and competition that caused Chicago’s gains, how should this impact Chicago’s future strategy?

Since 2002 (while the district was getting much better!) Chicago enrollment plummeted from 440,000 students to 370,000 students.

This means that there are lot of under-enrolled schools in the district and the city might have to go through another round of painful closures.

This also it means it’s harder to push the very reform (opening new schools) that might have driven Chicago’s previous gains in achievement.

So what should the city do?

Reasonable people can surely disagree, but I would continue to create new schools, albeit in a different fashion.

First, I’d open new schools in the areas where population is increasing. Chicago is made up of a lot of neighborhoods, and not all neighborhoods are losing children.

Second, I would do some replacement work. Instead of closing all the under-enrolled schools, I’d try and select some neighborhoods where there’s enough child density that you could imagine families coming back to the public schools if there were better options. I’d launch replacement schools in these neighborhoods.

There are clear drawbacks to this strategy. Politically, it’s hard to justify opening schools when you’re in the midst of closures. Programmatically, it’s hard sell to get the operators of new schools to open up in neighbors with shrinking enrollment.

But I think it’s the best thing for children.

Lastly, I might also try and launch some diverse by design schools.

In a city as diverse as Chicago, it’s sad that it’s schools are so segregated.

The Last Word

Chicago’s Chief Education Officer, Janice Jackson, recently gave her take on why things are better.

Her list: pre-k, better professional development, better curriculum, competition from private and charter schools, and clear accountability standards.

In her own words:

“I believe the level of transparency we have provided around what a quality school is has been transformational in this district.”

Rational compassion is a competitive advantage

bloom2

Paul Bloom recently wrote a book called Against Empathy.

The thesis of the book is: rational compassion > empathy.

In other words: empathy (caring how someone feels at the moment) is poor guide for moral decision making when compared to rational compassion (which is more utilitarian in nature).

The difference is easiest to see when it comes to parenting: an overly empathetic parent might respond to a child’s failure by giving the child a cookie (thereby immediately decreasing the child’s suffering), while a parent utilizing rational compassion might help the child process her emotions (thereby reducing the probability of future instances of suffering).

While the idea is rather intuitive, we’re so hardwired for empathy that practicing rational compassion, especially at work, is very difficult.

Because it’s so hard to practice, and because most people are not good at it, the consistent use of rational compassion can be a competitive advantage for doing good in both the for-profit and non-profit sector.

List of Areas Where Rational Compassion > Empathy at the Work Place

Executing strategies that cause short-term harm for long-term gain: Tough decisions (such as school closures) cause short-term pain to others but can provide significant long-term outcomes. Being guided by rational compassion can help you get through this pain.

Pivoting and cannibalizing: Similarly, at times an organization needs to destroy existing program lines and harm existing beneficiaries of their work in order to pivot to a more productive model which will eventually add move value to more people (think Netflix going from mailbox to streaming). Empathy for existing employees and customers can blind one from the rationally compassionate act of eventually serving more people better.

Performance feedback: Rational compassion will lead you to give very direct and practical feedback so a colleague can improve her performance and achieve her and the organization’s goals. Having empathy for underperformance will lead to the avoidance of direct conversations, which in the short term causes more pain.

Firing people: Too much empathy for an individual who needs to be let go can cause immense harm to the people you are trying to serve. Especially in philanthropic work, firing a relatively privileged person in order to better serve people in extreme need is the rationally compassionate thing to do.

Accepting flaws of ambitious people: Sometimes ambitious people have a lot of flaws, which can lead you to empathize with all the people they are negatively impacting. However, these flawed people can also change the world for the better. Analyzing their actions through a rational compassion lens will help you understand if it’s worth supporting or partnering with people who are flawed but who can help the world become amazingly better. It will also help you avoid working deeply with nice people who are not effective.

The Risk of Rational Compassion 

One of the hardest parts of rational compassion is that it often involves overriding the legitimate short-term needs of others.

In other words: you’re saying you know what’s better for someone than she does.

While this is less of a tension in managerial situations (it’s your job to make feedback, coaching, firing decisions) and for-profit work (the customer will ultimately hold you accountable), in philanthropy (where it’s your job to help others) this can be a deadly sin.

It’s a blurry line between rational compassion and technocratic hubris.

There’s no easy way around this, though research and accountability can help.

In education, test scores, attainment, and parent demand can provide medium term feedback loops to provide a check on incorrect rational compassionate assumptions.

But while there are risks with rational compassion, most of society is so tilted toward empathy (especially in the education sector!) that an increase in the practice of rational compassion would be a welcome turn.

 

What if unified enrollment platforms were 10x better?

An emerging group of cities – including Washington D.C., Newark, Camden, New Orleans, and Denver – have adopted unified enrollment systems. With these systems, families can enroll in schools across the city via an online application system.

This is a huge step forward. For too long, parents have not had enough information or access to the public schools in their cities.

However, the new enrollment systems are still in their infancy. The best version of these systems could radically improve public education. Unfortunately, we’re very far from this endgame.

I. Early Wins: Access, Equity, and Ranking

Access: With the best open enrollment systems, families who can’t afford a house in a fancy neighborhood can now finally transparently apply to a school in a more wealthy neighborhood.

As a result of increase in access, a recent study in Washington D.C. found that the new enrollment regime would likely reduce segregation over time:

Screen Shot 2017-03-17 at 9.23.22 AM

Equity: In most cities, traditional and charter schools play a ton of games with enrollment. Traditional magnet schools use opaque entry requirements. Bad apple charters don’t take in kids with special needs. There is no equity.

With online enrollment platforms, these problems go away, as schools are no longer in control of their enrollment.

Quite simply: the algorithm is fairer than the enrollment clerk.

Ranking: These new enrollment systems also allow parents to rank their top schools. This is extremely important.

First, a family’s high desire to enroll their child in a school can now  be translated into an increased chance that they actually get into this school.

Previously, high desire meant little unless you were connected, wealthy, or dogged.

Second, ranking allows  parents to publicly signal to government which schools are most and least in demand (which will ideally affect opening, expansion, and closure decisions). It also signals to school operators what attributes make a school in high demand.

By analyzing ranking preferences, researchers in New Orleans were able to correlate school characteristics with parent preference:

Screen Shot 2017-03-17 at 9.44.39 AMRanking transforms family desire into actionable information.

II. Unified Enrollment Systems are Mediocre Platforms

In preparation for writing this blog, I spend an hour on unified enrollment system websites. It was not a great experience.

Here is the school finder homepage from Washington D.C. – I couldn’t even find a way to filter schools by academic performance!

Screen Shot 2017-03-17 at 9.50.14 AM

Here is the search function for Newark’s enrollment system – you have to download a pdf!

Screen Shot 2017-03-17 at 9.55.36 AM

By comparison, here’s the search page from Zillow:

Screen Shot 2017-03-17 at 9.53.10 AM.png

On Zillow, you can easily search for homes based on the most useful search criteria. Yes, Zillow is surely better capitalized than your average enrollment system, but even with modest funds a city should be able to do better than a downloadable pdf.

III. Moving From Equity and Ranking to Matching and Prediction

More sophisticated uniform enrollment could offer two extraordinary improvements: they could better match families with schools, and they could better predict how any given student would do at a school.

Matching: Right now families mostly use enrollment systems for ranking: they know the schools they want and they use enrollment systems to express this desire.

What is not really happening (as far as I can tell) is sophisticated algorithms actually helping families match with schools.

For example, on Camden’s enrollment site (where you can thankfully filter by academic performance!), I found three schools that all met the “on track” performance criteria, and pulled up the comparison page:

Screen Shot 2017-03-17 at 10.11.14 AM

This does very little to help me choose between these schools. My preference variables are limited to very broad categories such as “art classes” and “after school programs.”

After playing around on these websites, I get very little indication that that the platform knows me or the schools very well. Compare this to Netflix, Amazon, or dating websites (previous life) – platforms where I feel like the algorithms know me better than myself.

Unified enrollment systems need to more deeply understand children and schools in order to make better matches.

Prediction: Right now, government accountability systems are a basket case of poor design (generally don’t weight growth enough), brutal politics (what politician wants to tell a bunch of communities they only have “F” schools?), and awful transparency (good luck trying to navigate your average state department of education website).

Most importantly, government accountability systems evaluate schools rather than make predictions.

As a parent, it’s one thing to tell you that a school is a “C+” – it’s another thing to give you a prediction of what will happen to your child if she attends the school.

With current date, we could probably gather basic information on your child’s age, gender, current academic performance, personality type, etc.,  and make a reasonably accurate prediction that if she attends school X she will have a Y% chance of graduating from high schools and a Z% chance of earning a post-secondary degree.

Good enrollment systems, over time, should become better and better predictive agents, and, perhaps, can end up augmenting (displacing?) government accountability systems.

IV. Root Causes and Potential Solutions 

I don’t yet have strong beliefs about the root causes of why these enrollment products aren’t getting better faster. But here’s some guesses:

Non-profits > government operated: Most of the enrollment systems are run by governments, which are not good at running tech products and have bad incentives around giving parents accurate information about schools. Non-profits would likely be better operationally and have better incentives, and avoid the privacy concerns associated with for-profits.

Lack of scale: Matching and predication can better with bigger data sets, and if all these systems are structured as isolated city based data silos, the algorithms will be dumber than they should be.

Weak Customer Demand -> Bad Economics: SchoolMint, from what I understand, is the most successful player in the market. For reasons I don’t underhand, this company has not developed a better product. Perhaps it’s because their government customers don’t actually want it. Or perhaps the economics don’t work (which might suggest philanthropy is needed).

If the above is true, a national non-profit should be backed to scale to enough size to create smart algorithms, and it should be financially structured in a manner that gets it out of the perverse incentives of being beholden to government or individual schools rather than families.

A philanthropic foundation with a great tech backbone could be well situated to support this endeavor.

V. Expectations

Better matching and prediction would probably not make the average student’s education experience 10x better, just as dating websites don’t inevitably lead to great marriages.

But I do think better matching and predication could increase the probabilities that millions of families could find a better fit for their children.

At scale, that’s a better world.

A weak SIGnal: flawed research means we don’t know if SIG worked

I have a piece in Education Next about the study that came out the $7 billion federal school turnaround program.

Everyone is saying that the study proves SIG didn’t work.

I disagree.

My main argument is here:

In detailing these results, the authors note:

“The smallest impacts our benchmark approach could detect ranged from 0.19 to 0.22 standard deviations for test score outcomes, from 0.15 to 0.26 standard deviations for high school graduation, and from 0.27 to 0.39 standard deviations for college enrollment.”

Now, look back up at urban charter effects and you’ll see the three year results in math are about at the floor of what the SIG study could detect, and the results in reading are much lower than what the SIG study could detect (the SIG study also tracked children for 3 years).

So even if SIG achieved the same effects as urban charter schools the study may not have been able to detect these effects. 

It seems pretty unfair for charter (or voucher) champions to call SIG a failure when SIG might have very well achieved near the same results as urban charter schools.

My conclusion:

Until I see results that show that SIG worked, I won’t change my prior belief that SIG funds would have been better spent on high-quality charter growth.

Moreover, neither the existing research base nor theory warranted a $7 billion spend on district turnarounds, so even had the intervention worked I still would consider it a lucky outcome on an ill-advised bet.

But I also won’t claim that SIG failed.

Due to poor research design, we simply don’t know if that’s true.

The study authors, reporters, and commentators should walk back their strong claims on SIG’s failures.

At the same time, we should all keep advocating for government investment amounts to be in line with the existing evidence base.

If we have no reason to believe something will work, we should not spend $7 billion.

Too often, moonshots garner more status then they deserve.

Read the the whole piece here.

What Happens When What Works for Children Doesn’t Feel Good?

Closing schools does not feel good: it’s painful for families, educators, and politicians.

But closing schools, and opening new better schools, can dramatically help low-income children.

Sometimes, the best thing that can happen to a child is for her school to close.

Closing Schools Led to a +.3 SD Gain for Elementary Students in NOLA 

In New Orleans, Tulane researchers found that closing schools and creating new better schools led to very significant achievement gains:

screen-shot-2016-10-19-at-4-33-16-pm

Elementary students who attended a failing school started .1 SD behind their matched peers – two years later, these same students were .2SD ahead of their matched peers.

This +.3 SD impact is higher than the impacts of most educational interventions, and it equates to closing about 33% of the black-white student achievement gap.

You Don’t Have to Harm Existing Children for the Sake of Future Children

What’s incredible about these results is that the students whose schools were closed increased their student achievement.

Before seeing this data, my guess would have been that closing schools slightly harms existing students but is much better for future students who get to attend a better school without going through the disruption of closure.

But the NOLA data indicates that it’s possible to help both existing and future students, which should increase your belief in the benefits of school closure.

You Should Not Close Failing Schools and Send Children to Other Failing Schools 

In Baton Rouge, school closure did not lead to positive effects. This seems to be because these students enrolled into other failing schools after their original school was closed.

 

Screen Shot 2016-10-19 at 4.45.08 PM.png

For school closure to work, a city needs to either have open spaces in existing higher-performing schools or be opening new high-quality schools.

There are Good and Bad Ways to Close Schools

School closures are hard, but they can be done with respect. Families deserve to know why their schools are being closed; families should get support (and preference in unified enrollment systems) in finding a better school; and political leaders should ensure that empty school buildings are put to good use for the community.

Unfortunately, in many cities, political officials do not close schools thoughtfully. Instead of being honest with families about the poor performance of the school, they let failing schools linger for year until enrollment dwindles and the school folds academically and financially.

It is Difficult to Scale Something that Causes Political Pain

It is unclear to me whether or not deliberate school closure will scale. Reforms that cause political pain tend not to do well over time.

However, opening new great schools need not be politically painful, which bodes well for continued charter growth.

Of course, continued charter growth can lead to the closure of failing schools – and this is exactly why charter moratoriums have some political support.

Charter moratoriums have the potential to reduce pain for adults even as they inflict pain on children.

Can New Orleans Continue to Close Schools? Should It?

Over the past few years, academic performance has stagnated in New Orleans:

screen-shot-2016-10-21-at-8-20-25-am

During this time, there has also been a reduction in school closure activity.

So here is an interesting question: has the stagnation in performance been caused because New Orleans ate up most of the low-hanging fruit of closing schools, or has the stagnation in performance been caused because New Orleans has slowed down on closing failing schools?

At this point, I’m not familiar enough with the data to have strong opinions.

But I do worry that New Orleans, especially as it moves toward more local control, may stop using one of the strategies that has proven to dramatically improve the achievement of its students.