Category Archives: Governance

Doubling down when you’ve been wrong

From 2012 to 2016, three reforms took place in Memphis: the Gates Foundation teacher evaluation reform, the Achievement School District (ASD) charter school turnaround effort, and the district run izone turnaround effort.

In private conversations, I predicted that neither the teacher evaluation nor izone efforts would work. I thought the ASD charter effort work would work. The Arnold Foundation funded research to help us understand if our beliefs about how to help children were correct.

Five years later, the results paint a different picture than my predictions: the teacher evaluation work did not improve achievement. The ASD charters have yet to deliver results. And the izone schools have done the best.

I was wrong about the izone and the ASD.

So what to do now?

The short answer: expand non-profit governance to protect the gains of izone schools, and continue to grow the best non-profit public charters and replace the worst.

The longer answer is below.

Responding to Being Wrong

There are three ways to respond to being wrong: ignore that you were wrong, change your behavior, or admit that you were wrong but double down.

Ignoring that you’re wrong is never good.

Changing your behavior is often the most reasonable response.

Doubling down comes with substantive and reputation risk, but it can be the right thing to do.

Memphis Results 

The teacher evaluation reforms went roughly as I expected, so I won’t discuss that too much here, other than saying that my predictions were based on the belief that execution would hamper implementation in big urban districts, which seems to be exactly what happened.

Here are the results of the izone and ASD charter efforts:

 

 

This slideshow requires JavaScript.

The izone had a positive .2 stand deviation effect over five years. This is fairly good impact and it’s great to see these schools helping students.

The ASD schools had no impact.

What I Got Wrong

I made a few mistakes.

First, I underestimated the school district’s strategic and executional competency. In creating the izone, the district successfully recruited a lot of their best educators to their izone schools, and then appointed a talented leader, Sharon Griffith, to oversee these schools. This strategy (concentrating talent) and execution (selecting a great leader) worked in ways I did not expect.

Second, I overestimated the abilities of charter networks to deliver results in the first few years of the turnaround efforts. My assumption was that expert charter leaders would be able to deliver better results pretty quickly, even though they were going into very difficult situations and were often coming from out of town. Unfortunately, a few operators really struggled in the early years, with one of them, Yes College Prep, choosing to not even open up a school (this was a big hit as YES is considered to be one of the best networks in the country).

What I Would Have Done Differently 

I should have realized that great educators in the traditional system could achieve good results if given autonomy in a competitive environment. I have changed my beliefs on this issue based of the work of the izone, the work in Lawrence, Massachusetts, and my lived experience of seeing great teachers and leaders in all types of schools.

Rather than predict the izone wouldn’t work, I should have predicted that the izone might work, but advocated for non-profit governance over the zone so that it could sustain its gains. This remains my worry with the izone: that it will not be able to sustain its impacts over the next 5-10 years due to its governance structure, which makes it vulnerable to political shifts.

I also worry that it can’t really scale. Putting the best educators in the worst school is more of a one time intervention rather than a strategy that can grow over time, and it has some negative effects on the schools that lose their great educators.

Second, I should have realized that early stage charter operators (either newly started or new to Memphis) would struggle to scale through full school turnarounds in a difficult political environment. In hindsight, I should have advocated for more one grade at a time roll outs of newer operators. While this would have been more disruptive to families at the outset, it would have been better for kids over the long run.

What to Do Now

If the above diagnosis is true, it doesn’t really call for a dramatic change in strategy.

Rather, it calls for trying to get non-profit governance over the best district schools, growing the charters that are working, replacing those that aren’t, and starting new non-profit district and charter schools under more favorable conditions.

Getting non-profit governance to the best district educators would (I think) require a change in legislation.

Growing the best charter schools and closing the worst  can be done under current policy.

Evidence to Support Doubling Down

Researchers have found evidence of charter school sectors improving over time. This paper on charters in Texas found significant sector improvements, which led the researchers to write:

The findings suggest the value of taking a longer-term perspective when evaluating the impact of a major educational reform such as the introduction of charter schools, especially when the success of the reform ostensibly depends on parental decisions and market forces.

Additionally, previous research on Memphis charters found strong positive effects:

Screen Shot 2018-06-23 at 8.03.44 AM

These results mirror the results of many other urban charter sectors, which generally find positive effects.

So there is a research based case for predicting that the Memphis charters will improve over the next 5-10 years. We have already seen this with some operators, such as Aspire.

In Sum 

The research has increased my belief that great district school educators, with autonomy and support (and often pressure on local districts from the state to give this autonomy) can really help struggling students.

I still believe that these gains risk backsliding due to political shifts, so I support giving these schools non-profit governance so they can build enduring organizations (both Denver and Indianapolis have mechanisms to give district schools non-profit status).

I also still believe that growing the best charters and closing the worst is one of the best hopes for improving student achievement in Memphis.

So, in many ways, I’m choosing to double down despite being wrong, with the caveat of calling for non-profit governance for more types of public schools.

While this is uncomfortable, I don’t know a better way forward. I think doubling down will help children.

I understand that many of those who I disagree with say “we just need more time.” And I’m making a version of this argument here. So I’ve tried to lay out all my assumptions as clearly as possible, both for the sake of transparency, as well as with the hope that these assumptions can be corrected if they are wrong.

What can we learn from school board meetings?

In forming opinions on policy, it’s good to balance research and lived experience.

Research is an invaluable tool in helping us test our beliefs, but the most rigorous research is limited to what can be measured, and not everything can be measured.

When it comes to public school policy, attending school board meetings has been a major part of my lived experience.

Over the past decade, I’ve attended two different types of school board meetings: elected school board meetings and non-profit charter school board meetings.

 

The elected school board meetings I’ve attended  range from dysfunctional (school board members screaming at each other) to misguided (school board members discussing random topics that have little to do with governing public schools) to sometimes useful (reasoned debate on systems level policy).

Unfortunately, I’ve attended many more dysfunctional and misguided elected school board meetings than I’ve attended useful elected school board meetings.

The non-profit charter school board meetings tend to be much more productive. More often than not, the board members are focused and knowledgeable. This is especially true of larger non-profit charter networks, where the board has been around for over a decade and governed through organization scaling.

Yes, I’ve attended a few dysfunctional charter school board meetings, but they are not the norm.

This lived experience in attending board meetings has shaped my policy views. Based on a decade of attending board meetings, I believe that non-profit boards will be better at governing schools than elected boards will be.

I do believe we need democratic oversight over our public schools. Elections allow us to debate values, and these values should shape how we oversee our public schools.This oversight can come in a variety of forms, from elected boards to mayoral control. The New Orleans elected school board, for example, oversees a nearly 100% non-profit charter schools system.

But while we need elections to debate values, we don’t need elections to govern individual or networks of schools.

My lived experience leads me to believe that this duty is best held by non-profit boards.

Most students in NOLA whose school is closed end up in one of their top choices for the following year. Here’s how.

This post from Ed Navigator is worth reading. It covers schools closures in New Orleans.

Over ten years after Katrina, and under an elected school board, New Orleans continues to selectively close underperforming schools.

I view this as a good thing, given the growing body of research that shows that school closures help kids when the students end up in better schools.

New Orleans uses a unified enrollment system to help kids get into better schools.

The unified enrollment system gives preference to students whose schools were closed the year before. If your school was closed, the algorithm bumps you to the top of the list for any school you want to get into.

Ed Navigator works with families whose schools have been closed, so that they can help select great schools.

The result?

This year, 87% of students who attended a closing school and used the enrollment system received on of their top three choices for the next school year.

94% of the students will now attend a school that is rated higher by the state’s grading system.

The system is by no means perfect. My biggest critique is that the state’s grading system still relies too heavily on absolute test scores (rather than growth). I also understand the counterarguments that government should never close schools and should instead let enrollment patterns (driven by parental choice) determine which schools grow and which close.

But I would rather have the New Orleans enrollment and closure system than just about any other big city system in the country.  In too many cities, really bad schools stay open for too long. And if anything happens to them, kids often end up in schools that are just as bad.

This is not what happens in New Orleans.

It’s also great to see Parag Pathak (and his colleagues) work in action. Parag recently won the John Bates Clark award in part because of his contributions to working on unified enrollment systems.

It’s rare that an idea goes from the ivory tower to think tanks to actual implementation by a democratically elected body to  helping citizens.

This is really great to see. And really great for kids in New Orleans.

The best school district in the United States?

One of the joys of my job is the number of amazing emails that arrive in my inbox.

Below is an email (pasted with permission) from Scott Pearson, the head of the Washington DC Public Charter School Board.

On this blog, as well as on twitter, we debate a lot about regulation. We have a lot to figure out and these debates help me get smarter.

But leaders on the ground have to lead, always with imperfect information and complicated local contexts.

The DC Public Charter School Board has chosen to regulate the charter community fairly tightly on performance, but more loosely on other inputs. As Scott notes in his letter, over 40 charters have closed in Washington DC over the past decade. While I don’t know if this is right for every community, the DC charter community is providing a lot of great options for tens of thousands of children, and they have undoubtedly made DC a better city.

The continuity of the DC charter community’s success also reinforces my belief in the importance of non-profit governance. It’s hard to think of a better school district in the country, and I’m highly confident that a primary key to their success is their structure: the DC Public Charter School Board regulates and non-profits operate.

It’s a winning formula for kids.

___________________

Dear Colleagues,

We have faced nearly a year’s worth of bad news about DC Public Schools, from high teacher turnover, to faked suspension data, from inflated graduation rates, to the resignation of the DCPS chancellor, to residency fraud.  This steady drumbeat has undermined confidence in our traditional public schools – far more than is warranted in my opinion.  DCPS is a vastly better school system than it was in 2007 when Michelle Rhee and Kaya Henderson took the reins.

But I’m writing today about the other half of public schools in DC, the 120 public charter schools serving 43,340 students – nearly half (47.5%) of DC’s public school students.  Our story is mostly one of continued success, growth, popularity, and quality improvement.  I feel the need to write this because I fear that the bad news about DCPS is drowning out what continues to be a remarkable story of charter school success in our nation’s capital.

First, it’s important to say that not one of these bad news stories has been about public charter schools.  Our graduation rates check out – because for years the DC Public Charter School Board has audited the transcript of every graduating senior.  Our discipline data has never been questioned.  And there have been no allegations that anybody has jumped the lottery queue at a DC charter school.

Of course, we’re not perfect. In a sector as diverse as ours there will be unflattering news to uncover.  But at least so far none of the year’s bad news has touched DC’s charter sector.

Second, our quality keeps improving.   The NAEP “flatline” story just doesn’t apply to DC’s charter schools, as David Osborne and Emily Langhornewrite in The 74.  We were up this year in three of four grade/subject areas, with fourth-grade reading scores climbing five scale score points.  That’s more than any state in the nation.  And over ten years our growth overall is faster than any other state or district.  Charter scale scores have grown 17 points in 4th-grade math, 19 points in 4th-grade reading, and 12 points in 8th-grade math.  Each of these represents over a year’s worth of learning gains.   Only in 8th-grade reading are our scores disappointing, down 2 points over the past ten years and lagging far behind our big-city peers.

Our school leaders deserve much of the credit for this growth, but the authorizer gets some credit, too.  Since 2007 we have overseen the closure of 40 low-performing charter schools, all the while aggressively supporting growth for our highest-performing schools.

Third, our improvements are not driven by a change in our demographics.  Charter schools continue to enroll higher percentages of black and low-income students than does DC Public Schools.  And charters schools now educate the same percentage of students with disabilities as does DCPS – and higher percentages of our most disabled children.

Fourth, even as our quality improves, our schools have made remarkable progress reducing out of school suspensions and expulsions.  Out of school suspension rates are down by half since 2011-12, to under 7%.  And expulsions are down over 80% to less than 0.25% – about the national average.  We’ve achieved this through transparency and communications, not through mandates.  (Though, disappointingly, our city council is now threatening to regulate school discipline.)

Finally, demand keeps growing.  Despite the charter board adding nearly 9,000 charter school seats since 2013-14, the number of unique families on charter school waitlists has risen from 7,205 in April, 2014, to 11,317 in April, 2018.  Two-thirds of our charter schools saw their waitlist length increase from last year to this year.   Waitlists are in one sense a measure of our success because it shows families want our schools.  But is also a measure of our failure – and that of the District government – to provide our residents with enough quality schools and the facilities to house them.  We need to do better.

Thank you for reading, and please reach out if there is more information or context I can provide.

Scott Pearson

Executive Director

DC Public Charter School Board

What is California telling us about what parents want from public education?

I sit on the board of the California Charter School Association (CCSA), which is one of most effective charter associations in the nation.

Their data team put this slide together.

It’s a little complicated, but it’s very informative.

Screen Shot 2018-04-05 at 8.08.33 PM

To build the chart, CCSA looked at schools that are serving the same type of demographics and then compares their test scores.

A school that outperforms similar schools, gets a higher score (8-10 is really good); a school that underperforms similar schools, gets a lower school (1-3 is pretty bad).

This methodology – absolute test scores controlled for demographics – is imperfect, but it’s the best methodology you can use given California does not publicly report student growth scores.

Charter schools serving students in poverty outperform district schools on state tests 

43% of charter schools with higher concentrations of poverty outperform similar schools.

In California, if you are a low-income Hispanic or African-American child, you are more likely get a better education (as measured by test scores and parent demand) if you attend a charter school.

This is good news for the educators and families who are working together to create better educational outcomes in at-risk communities.

If parents are demanding schools with good test score impacts, the government’s response should be obvious: let more of these schools open.

Charter schools serving middle class students underperform district schools on state tests 

But not all charter schools are outperforming their peers on tests scores.

47% of charter schools serving middle class students perform worse than similar schools. And only 26% perform better.

So perhaps California should close some of these charter schools that serve middle class families? Research covered in this post shows that schools with negative test impacts tend not to have large positive life outcome impacts.

But here’s the odd thing: presumably, middle class families have a decent amount of information at hand when making school choices. It takes all of two minutes to scan Zillow or Great Schools to get a quick read on the absolute test score performance of any school in the state.

So why are all these relatively well resourced families sending their children to lower-performing charter schools as measured by state tests?

I’m not sure. It would be interesting to focus group and poll them to learn more.

And with regards to closure, while I surely disagree with middle class Californians on many policy issues, I’m not sure that I think I know enough about their children to close schools that have modest negative test impacts but high enrollment demand.

Charter schools don’t exist unless parents want their children to attend them 

One of the best features of charter schools is that they don’t exist unless parents choose them. No one is assigned to a charter school.

So what are we to make of this data where families in living in poverty are choosing schools with positive test scores impacts and middle class families are choosing schools with negative test score impacts?

I think the starting point should be to assume that families, on average, are in a better position to make an informed choice than government is.

Remember, government’s default assignment algorithm is to look at your family’s address and then assign your child to the nearest school. It’s not very nuanced!

My guess is that parent choice will outperform geographic assignment when it comes to finding great fits between kids and schools.

But I do think we should be open to the idea that parents, sometimes en masse, can make mistakes. And, at times, this can warrant government intervention.

Sometimes performance might be an indicator, such as when families keep sending their children to high schools with below 40% graduation rates and /or schools with extremely negative value-add scores. If less than half the kids are graduating, and those that do are barely literate, government should step in.

Sometimes lack of alignment with our nation’s professed values might be an indicator: certain public schools have at times been captured by groups, sometimes religious, that do not teach basic democratic values.

In these cases of significant performance or culture malfeasance, government should consider intervention, ideally by handing over management of the school to a non-profit organization that can achieve better results.

I don’t know enough about individual school performance to know if what’s happening in California with middle class families equates to education malfeasance, but I’m a bit skeptical.

My hunch is that once absolute test score levels surpass a certain floor (as they tend to in middle class schools), families just care a lot about other factors.

Even if I might make a different choice, I don’t know that the situation warrants government intervention.

When government should not intervene 

While it’s difficult to decide when government should intervene, it’s still pretty clear to me when government should not intervene.

When schools have both high demand and high test score impacts with students living in poverty, government should not prevent these schools from serving more students!

California should follow this common sense policy.

Unfortunately, too many school districts do not.

SaveSave

SaveSave

A Crisis Reveals the Obvious in Washington D.C.

The Washington D.C. public school system (DCPS) has been rocked by crisis over the past year.

First, came the suspension scandal. DCPS schools were lowering their official suspension rates by simply not recording when students were suspended.

Then came the high school graduation scandal. An investigative report showed that at a single school over 30% of the graduates received a high school diploma in violation of district policy. With more honest record keeping, it looks like overall district graduation rates might drop over 10 percentage points, if not more.

Then came the enrollment scandal. An investigative report found that the superintendent, Antwan Wilson, had violated the district’s enrollment policy when enrolling his own daughter in highly sought after school.

Despite these three crises, I don’t think anyone should wave their hands and say nothing good has happened in DCPS over the past 15 years. Things have improved. The gains the city made in NAEP do appear to be real and significant (the Laura and John Arnold Foundation is funding a research study to further examine this question).

But there were always two fatal flaws to the DCPS approach.

The first flaw is that it was obvious that DCPS was never going to be able to turnaround its struggling schools.

The second flaw is that it was obvious that the district was not going to be able to handle leadership succession.

Both of these fatal flaws having nothing to do with any individual leaders in DCPS. Rather, both have to do with the structure DCPS itself.

As I wrote back in 2015, if you don’t fix the structure of DCPS, you’ll never be able to fix DCPS.

The Most Damning Chart on Struggling Schools in Washington D.C.

Recently, I worked with some colleagues to apply the local charter school accountability system to DCPS schools, as we thought the charter system was more rigorous than the DCPS’s system. Here’s what we found for grades pre-K to eight (where publicly available data allows for reasonable apples to apples comparisons).

Screen Shot 2018-03-12 at 5.53.15 PM

There is a stark difference in the number of children attending Tier 3 schools ( the lowest performing schools) in each sector.

DCPS still has nearly 8,000 students stuck in failing prek-8 schools, despite over 10 years of reform leadership.

The charter sector, on the other hand, has replaced nearly all of its struggling schools with much better schools, having closed or replaced over 21 schools since 2012.

The reason for this is simple and obvious: it’s very hard to turnaround yourself.

The DC Public Charter School Board oversees, but does not operate, the non-profit public schools under its jurisdiction. DCPS, on the other hand, both oversees and operates its own schools.

The structure of the charters system in DC makes it easier to to replace struggling schools with better ones. The structure of DCPS does not.

Political Leaders are Selected for Political Reasons Under Political Circumstances 

The DCPS approach, which consolidates power in a central bureaucracy, relies heavily on strong Chancellor leadership. Leadership is of course important to all organizations: charter school organizations also lean on great leaders.

But there is a difference between how DCPS and charter organizations pick their leaders. The DCPS leadership transitions run through a political process, while charter leadership transitions run through a non-profit board process.

This is why it is often so hard for districts to manage superintendent transitions: it’s rare that a single superintendent lasts numerous political cycles, and it is even rarer that multiple superintendents in a row will share a common strategic vision.

Non-profit boards have it easier. While they are not immune from horse trading and politics, their boards of directors are not elected and are less subject to acute external political pressures.

The charter sector is also composed of dozens of non-profit boards. So even if one leadership succession goes badly, the whole public school system won’t suffer.

Am I surprised that DCPS has had three superintendents in two years? No, the history of urban public school systems made this an obvious eventual outcome.

Creating a Stable, Community Driven Public School System 

Fortunately, Washington D.C. is better positioned than most cities when it comes creating an amazing public education system.

Over the past fifteen years, the city has been home to both improving traditional and public charter sectors.

Moving forward, it should take the best of what the city has seen in both sectors and unify this under one public system.

From the charter sector, the city should take the idea that schools do best when they are operated by non-profit organizations, and, when a school struggles, the best thing to do is to let another non-profit school try and operate the school.

Innovative governance models with real accountability can be applied to traditional public schools. Both Denver and Indianapolis already allow traditional public schools to build non-profit boards that are held accountable through performance contracts.

From the traditional sector, the city should take the idea that Washington D.C. residents value neighborhood schools and expanded pre-k; two areas where the district has made great strides. Giving every public school a non-profit board does not mean every school needs to be a full open-enrollment charter school. Neighborhood schools and community  based early learning centers should be part of the fabric of the pubic school system.

Non-profit public schools can make DCPS better. They can give great educators more autonomy. They can create more accountability within the system. And they are best set-up to manage tough leadership successions.

SaveSave

Did a federal grant to turnaround failing schools in New Orleans and Tennessee work?

Back when I worked at New Schools for New Orleans, we applied for a $30m federal grant to turnaround failing schools in New Orleans and scale the model to Tennessee.

CREDO just came out with a research study on our efforts. Their findings, and my analysis, are below.

The New Schools Were Much Better than the Ones They Replaced 

Here’s what CREDO found when they compared the schools we created to the schools we replaced:

In New Orleans, we replaced schools (“closing schools”) that were at 26th percentile in the state with new schools (“CRM schools”) that performed at the ~33rd percentile in the state at the end of the study.

In Tennessee, schools went from the ~17th percentile to the ~23rd percentile by the end of the study.

To quote the CREDO report: “the CRM schools in both New Orleans and Tennessee showed significantly higher academic growth compared to the Closing schools they replaced.”

Translated into days of learning, these are large effects: “Closing school students experience 63 fewer days of learning in reading and 86 fewer days of learning in math when compared to students in non-CRM schools… students in CRM schools make comparable academic growth to non-CRM students.”

The New Schools Performed About the Same as Other Schools in the City

When CREDO compared the new schools to other existing schools (rather than the failing schools they replaced), they found no statistically significant effects:

Screen Shot 2018-02-15 at 12.07.57 PM.png

In other words, the new schools that replaced the failing schools performed no better or worse than other existing schools in the city.

On one hand, this is disappointing. Our most ambitious targets included having the new schools be amongst the highest performing schools in the city.

On the other hand, this is still a major improvement: the new schools replaced failing schools and ended up achieving at the same level of most other schools in the city.

Building a System that Keeps Getting Better 

Replacing failing schools with new schools is a process, not a one-time intervention.

Ideally, a subset of the schools you created will do really well, and then, overtime, these schools will continue to grow. The ones that don’t do well will not be supported to do additional turnarounds.

Over the long-haul, gradually increasing the number and scale of high-quality school operators is more important than the average effect of the first wave of replacements.

Here’s what CREDO found across the new schools when they compared them to existing schools:

Screen Shot 2018-02-15 at 12.11.36 PM.pngIn New Orleans, 50% of the new schools had positive effects in both Math and Reading. This is really positive: half of our turnaround schools in New Orleans achieved significantly better results than existing schools across the city.

In Tennessee, only one school had positive effects in both Math and Reading, though a few other schools had positive effects in only reading.

This makes me optimistic that the school operator base in New Orleans will continue to have the capacity to replace more failing schools over time.

The early results in Tennessee are a bit more worrying on the operator quality front, and the next few years will be extremely important in ensuring that a healthy operator base emerges.

Lastly: CREDO found that replacing failing schools with fresh start schools (that opened one grade at a time) had a higher success rate than whole school turnarounds. My takeaway here is that you need a mature operator base to do a lot of whole school turnarounds, and no city had enough capacity to really do whole school at scale. In hindsight, we should have done more fresh starts and less whole school turnarounds.

Was the Effort a Success?

At the outset of the project, I remember debating with our research partners at CREDO about how to set-up the evaluation.

I argued that we should ultimately be judged on whether or not the new schools we created were better than the failing schools we replaced.

I didn’t think we should be primarily judged on whether or not the new schools were better than other existing schools that weren’t failing.

Yes, we did include language in the grant application that had goals of schools performing much better than existing schools. And as we executed the project we tried to pick school operators that we thought could deliver top tier results. Our highest aspirations weren’t met. This is disappointing, but it does not mean the project was a failure.

Rather, I consider the project to be a positive step forward in improving public education in these cities.

Making Things Better

The result of the project strikes at the heart of what’s so difficult about education reform: our aspirations for our most at-risk children are incredibly high, but making progress in creating better educational opportunities is very difficult.

In roughly a five year period, we replaced failing schools with new schools that were on average 7 percentile points higher in state performance, which translates to an extra 60-90 days of learning per year.

If the process of opening and replacement continues, what is a modest success right now may eventually become a great success.

I hope that this occurs and that New Orleans continues on its impressive track record of increasing student achievement. As a reminder, the federal grant was just one piece of an overall effort that has radically reduced failing schools in the city:

Screen Shot 2018-02-20 at 5.47.30 PM

SaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSave