Author Archives: nkingsl

Did a federal grant to turnaround failing schools in New Orleans and Tennessee work?

Back when I worked at New Schools for New Orleans, we applied for a $30m federal grant to turnaround failing schools in New Orleans and scale the model to Tennessee.

CREDO just came out with a research study on our efforts. Their findings, and my analysis, are below.

The New Schools Were Much Better than the Ones They Replaced 

Here’s what CREDO found when they compared the schools we created to the schools we replaced:

In New Orleans, we replaced schools (“closing schools”) that were at 26th percentile in the state with new schools (“CRM schools”) that performed at the ~33rd percentile in the state at the end of the study.

In Tennessee, schools went from the ~17th percentile to the ~23rd percentile by the end of the study.

To quote the CREDO report: “the CRM schools in both New Orleans and Tennessee showed significantly higher academic growth compared to the Closing schools they replaced.”

Translated into days of learning, these are large effects: “Closing school students experience 63 fewer days of learning in reading and 86 fewer days of learning in math when compared to students in non-CRM schools… students in CRM schools make comparable academic growth to non-CRM students.”

The New Schools Performed About the Same as Other Schools in the City

When CREDO compared the new schools to other existing schools (rather than the failing schools they replaced), they found no statistically significant effects:

Screen Shot 2018-02-15 at 12.07.57 PM.png

In other words, the new schools that replaced the failing schools performed no better or worse than other existing schools in the city.

On one hand, this is disappointing. Our most ambitious targets included having the new schools be amongst the highest performing schools in the city.

On the other hand, this is still a major improvement: the new schools replaced failing schools and ended up achieving at the same level of most other schools in the city.

Building a System that Keeps Getting Better 

Replacing failing schools with new schools is a process, not a one-time intervention.

Ideally, a subset of the schools you created will do really well, and then, overtime, these schools will continue to grow. The ones that don’t do well will not be supported to do additional turnarounds.

Over the long-haul, gradually increasing the number and scale of high-quality school operators is more important than the average effect of the first wave of replacements.

Here’s what CREDO found across the new schools when they compared them to existing schools:

Screen Shot 2018-02-15 at 12.11.36 PM.pngIn New Orleans, 50% of the new schools had positive effects in both Math and Reading. This is really positive: half of our turnaround schools in New Orleans achieved significantly better results than existing schools across the city.

In Tennessee, only one school had positive effects in both Math and Reading, though a few other schools had positive effects in only reading.

This makes me optimistic that the school operator base in New Orleans will continue to have the capacity to replace more failing schools over time.

The early results in Tennessee are a bit more worrying on the operator quality front, and the next few years will be extremely important in ensuring that a healthy operator base emerges.

Lastly: CREDO found that replacing failing schools with fresh start schools (that opened one grade at a time) had a higher success rate than whole school turnarounds. My takeaway here is that you need a mature operator base to do a lot of whole school turnarounds, and no city had enough capacity to really do whole school at scale. In hindsight, we should have done more fresh starts and less whole school turnarounds.

Was the Effort a Success?

At the outset of the project, I remember debating with our research partners at CREDO about how to set-up the evaluation.

I argued that we should ultimately be judged on whether or not the new schools we created were better than the failing schools we replaced.

I didn’t think we should be primarily judged on whether or not the new schools were better than other existing schools that weren’t failing.

Yes, we did include language in the grant application that had goals of schools performing much better than existing schools. And as we executed the project we tried to pick school operators that we thought could deliver top tier results. Our highest aspirations weren’t met. This is disappointing, but it does not mean the project was a failure.

Rather, I consider the project to be a positive step forward in improving public education in these cities.

Making Things Better

The result of the project strikes at the heart of what’s so difficult about education reform: our aspirations for our most at-risk children are incredibly high, but making progress in creating better educational opportunities is very difficult.

In roughly a five year period, we replaced failing schools with new schools that were on average 7 percentile points higher in state performance, which translates to an extra 60-90 days of learning per year.

If the process of opening and replacement continues, what is a modest success right now may eventually become a great success.

I hope that this occurs and that New Orleans continues on its impressive track record of increasing student achievement. As a reminder, the federal grant was just one piece of an overall effort that has radically reduced failing schools in the city:

Screen Shot 2018-02-20 at 5.47.30 PM

SaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSave

How should CEOs handle what society demands?

As I get older, my illusion of control weakens.

This is generally a good thing: the more I acknowledge that I can’t change much of the world, the more focus I have on the few areas where I might actually do some good.

For me, this means less time reading the news and more time on work and personal community.

But even within areas of some control, such as work, there are a lot of limits.

One of those limits is how, as a leader, to deal with what society demands.

Society Makes Demands 

I recently had dinner with No Excuses charter founder, and he was describing how his school’s attempts to implement restorative justice have been a train wreck. After two years of cultural decline, the school is now just getting back to having a culture of high expectations that helps children learn.

I don’t know whether his was because of poor implementation or actual serious flaws in the restorative justice model itself. But I do know that there was a lot of pressure for the school to adopt the restorative justice model.

So let’s put aside the idea of whether or not restorative justice is a good cultural model for a school  – rather, let’s consider the larger question: what should a CEO do when society demands something they disagree with?

In this case, liberal society (from which most No Excuses teachers come from) is increasingly demanding a more progressive school culture, with a lot of young white teachers reacting negatively to having to manage (what feels like to them) overly authoritarian cultures for black children.

For the sake of argument, let’s say you’re the CEO of a charter organization and you disagree with what society is demanding. What should do?

Bend or Stay Firm?

What if you think that restorative justice will lead to a decline in culture that hurts children’s lives? What do you do?

This is a very difficult question.

If you stay firm, you might lose the talent pipelines that had made you so successful in the first place. Or you might lose a communications battle that significantly reduces family demand for you school.

In other words, even if you think what society is demanding is wrong, it might still make sense to bend to society’s demands and just try to mitigate the negative impacts.

In this case, you’re basically trying to ride it out. Society’s demands constantly change – and you just have to hope that this moment in time will pass – and you can course correct in the future.

On the other hand, it might be the case that if you stay firm you will prove society wrong: when everyone else’s results plummet, you’ll be knocking it out of the park.

There are never any easy answers when you disagree with what society demands.

Some Advice 

 

Too often, CEOs make the mistake that society, having made its demand, is now willing to hear the CEO’s response.

By the time society has made its demands, it’s usually not in a contemplative mood.

So here’s some advice:

Consider bending: The best course of action is often to bend to society in a way that keeps your organization afloat and lets you live to fight another day.

Have a values conversation: If you decide this is a place where you really need to make a stand, the default position should be to always engage society in a values conversation, not a strategy conversation. Society doesn’t make demands about operations and strategies, it makes demands about values and tribal affiliations. If you’re going to try and convince society you’re right, you need to win on values.

Make sure your team is with you: If you’re making a values argument, you’ll get clobbered if a bunch of people internal to your organization say that they disagree with your values – or provide reporters with a bunch of examples of how you’ve violated the values you’re now professing to hold.

To the extent I’ve had to push back on society’s demands (keep neighborhood schools, don’t close schools), I’ve always tried to do it in a values based way. And I’ve always tried to surround myself with people who share these values. And, at times, I’ve bended: such as supporting enrollment systems that give a partial neighborhood preference to families… and respecting the demands of people who hold different values than I do.

A great report from Princeton on charter schools… with one major mistake

Screen Shot 2018-02-08 at 9.51.57 PM

Sarah Cohodes just published a report on charter schools in a joint Princeton and Brookings publication.

Sarah does a great job of summarizing the research on charter schools: on average, charter school do not outperform traditional schools, but urban charter schools (particularly the No Excuses model), perform much better than traditional urban public schools.

It is so rare in publicly policy that something… works best for disadvantaged students, gets better as it scales, costs 20% cheaper than the existing system, and has positive to neutral effects on the existing system’s performance.

With urban charter schools, we’ve found something that works at scale with almost no trade-offs.

It’s truly amazing and makes me hopeful for public education in this country.

So kudos to Sarah for writing this report.

But she does make one mistake, and it’s a common mistake, so it feels worth addressing.

Sarah writes:

The charter sector is growing by 300 to 400 schools a year. Let’s consider a thought experiment in which further expansion focuses on high-quality charters. What would happen to the achievement gap in the United States if all of those new charter schools were opened in urban areas serving low-income children, had no excuses policies, and had large impacts on test scores like Boston, New York, Denver, and KIPP charters?

So far so good. I think about this thought experiment a lot.

Sarah continues:

Expanding charters in this way certainly could transform the educational trajectories of the students who attend. But if we consider the US achievement gap as a whole, it would have a negligible effect. Charter schools represent too small a proportion of overall enrollment for such an expansion to reduce nationwide achievement gaps.

Notice the mistake? Sarah focuses on the absolute number of annual charter school openings rather than the annual percentage enrollment increase of charter schools.

Yes, the charter sector is growing by 300-400 schools a year.

But the charter sector’s enrollment has grown between 7-10% a year for almost a decade.

So here’s another way to answer Sara’s thought experiment: if the charter sector continues to grow at ~10% annually, it will double in size every 7 years. Currently there are ~2 million students in charter schools. That means in 25 years, charters would serve over ~20 million students, or nearly 40% of students in the United States, if current growth rates continued.

Of course, this might not occur. But it’s surely possible. I have deep experience in working in cities across the country, and I remain convinced that any major city in the America can get to ~50% charter enrollment in a 10-15 year period, if they so desire.

Again, it’s not inevitable that charters will continue to maintain quality and grow at 10% annually.

But it’s also not inevitable that districts can adopt the best practices of urban charter schools.

To the extent that there are major resource allocation considerations, I would bet more on charter school scaling than district adoption.

But I don’t think there are major resource allocation debates. Urban charters are an open book. District best practice adoption is less a matter of money than it is a matter of will.

So I think we should keep on scaling urban charters, with the hope that districts get better and all boats rise. If districts get better, more kids will get in great schools faster. If they don’t get better, charter school expansion can continue to increase opportunity.

Someone once asked me: how much charter school growth is enough?

My answer then, and now, remains the same: we should keep growing charter schools until every kid in American attends an amazing public school.

What will be normal for superintendents in 2028?

Education change happens slowly until it happens everywhere. And then a new normal exists.

Right now, it is normal for superintendents to directly control all the public schools in their city. It also normal for them to fight off educators (either within their system or external charter operators) who threaten this control.

But this is changing. A new wave of superintendents are looking to charter operators to help them solve their toughest problems. And they’re not doing so in light touch ways, like best practice sharing. Instead, they’re handing over control of some of their most struggling schools to charter schools.

Atlanta 

In Atlanta, the superintendent is partnering with the city’s best charter operators to operate underperforming schools in the city.

Here is how one Atlanta high school student responded to her plan:

Omari Hargrove, a junior at Carver School of the Arts, isn’t looking for guarantees.

At a recent community meeting, Carver students spoke about daily fights on campus, gambling at school and classmates more focused on selling drugs, than studying.

“I’m not sure if the changes they’re introducing are the ones they need, but the situation we have now is not acceptable,” Hargrove said.

“Maybe they’ll work,” he added. “At least it’s a plan.”

San Antonio 

The superintendent in San Antonio is taking a similar approach. Texas recently passed a law, Senate Bill 1882, which encourages both superintendents and charter operators to work together to turnaround failing schools.

Denver

In Denver, the district partnered with a charter operator, University Prep, to turnaround a struggling school. A year later, the school posted the highest math growth in the state of Colorado.

Indianapolis 

In Indianaplis, the district utilized the state’s Innovation law to partner with charter operators, and the first wave of schools saw an increase in test scores. One of those schools, while managed by a charter operator (Phalen Academies), was actually led by a group of district leaders who wanted more freedom (Project Restore).

Memphis

Over the past few years, the state, frustrated with local inaction, has forced charter partnerships in Memphis. But last week, the superintendent told the local paper he was willing to lead the partnership effort himself. A locally elected board member also indicated willingness: “As we look at what’s best for kids, we have to look at all options,” she said.

What will be normal in 2028?

That’s hard to tell. But I hope that what we’re seeing from this courageous superintendents becomes the new normal.

A superintendent’s job is to get the highest number of kids in the best schools as quickly as possible.

That’s a job that requires all hands on deck.

Any group of educators who can make things better quickly should be given that opportunity, charter schools included.

Over the coming decade, I hope what we see in Atlanta, San Antonio, Denver, Indianapolis, and Memphis is the new normal.

My favorite books of 2017

I felt this was a pretty weak year for books. I don’t know why. I only really loved three books that came out this year.

Especially with regards to work, I found I learned a lot more by doing rather than reading. I’m curious if this trend will continue.

That being said, the best books was incredibly good: because of the first two books below, I’ve tried to up my meditation to 40 minutes a day and reduce social media to under 30 minutes a day. I’ve also done a lot to reduce iPhone screen time. I’ve also spent much more time contemplating the nature of the self. My meditation practice is a little less tactical and includes more philosophical exploration.

I feel more in control of mind than I have in years.

Why Buddhism is True by Robin Wright

Robin’s thesis is:

  1. Our brains were mostly built during hunter and gather times.
  2. The modern world has hijacked useful desires (for food, sex, stimulation, and status) so that they are no longer that useful (we over eat, watch too much porn, constantly check our phones, etc.).
  3. There is no CEO in your brain. Your brain is made up of a bunch of competing desires / modules. And whichever you feed and reward will grow stronger.
  4. Meditation is a technique that can reduce the power of the feeling -> action sequence. Desires need not be orders if they are observed with distance and objectivity.
  5. The idea that there is no CEO of the brain also fits Buddhism’s core philosophical tenet that the self is an illusion.

I think arguments #1 through #4 are correct. Robin surveys a mounting body of scientific case evidence that makes this case, from evolutionary psychology to neurobiology.

I think #5 is directionally correct, but that ultimately humans do not have a brain that is powerful enough to make hard claims about these types of metaphysical conditions.

iGen – Jean Twenge

Jean’s thesis is that:

  1. Socio-economic conditions (in part families having increased wealth and less children) has led to a lengthening of childhood. High school is the new middle school.
  2. The iPhone has fundamentally altered how teens interact.
  3. Taken together, changing socio-economic conditions and the smartphone has led children to be more tolerant, less risk taking (sex, alcohol, and driving are down… marijuana is up), more insecure, less happy, less religious, more concerned with wealth, and more politically independent.

I’m not an expert in the field, but I found her found her argument compelling.

This generational shift is a striking example of how productivity and technology can combine to change societal values.

For you parents in the crowd, she gives thoughtful parenting recommendations at the end of the book.

The Dark Forest Trilogy

I previously reviewed the books here. Some of the best science fiction I have ever read.

The series is premised on this logic path:

  1. The primary goal of each civilization is to survive.
  2. There are finite resources and space in the universe.
  3. Civilizations tend to expand.
  4. Civilizations tend to advance technologically.
  5. You have no way of truly knowing whether an alien species is peaceful or hostile.

If this ends up being true in our reality, we will likely be destroyed by more technology advanced aliens.

Stanford researchers find that New Orleans and Chicago are doing amazing things for black children

The New York Times / Upshot just took a massive Stanford researcher database and turned it into an easy to use webpage.

This was a big undertaking and these types of projects are some of the best of modern journalism. Kudos to them.

The research methodology is great in its scope (every city in America) though rather crude in its precision (raw proficiency gains from 2009 to 2015). It’s not as accurate as experimental and quasi-experimental research, but it’s still useful in taking a broad look across the country.

Which of the largest 200 school districts in the country saw the most growth?

Here’s the top 10:

Screen Shot 2017-12-06 at 7.00.57 AM

In 5 years, all of these districts achieved at least 5.5 years of academic growth. Of the 200 largest school districts in the nation, these are the highest performers.

Are these cities very similar to each other? 

Not at all. And this is where I think the reporters could have done much more. The New York Times journalists rightfully pointed out Chicago as an outlier, but they did little else to tease out the vast differences between these high growth districts.

Let’s take a look at these top growth districts by free and reduced lunch rates:

FRL3

See any differences?

Only four districts – Salem Keizer, Garden Grove, Chicago, and Orleans – have students bodies where the majority of students are economic disadvantaged.

And Chicago and Orleans are in a category of their own, with 80%+ students receiving free and reduced lunch during the years of the study. The task these districts face is 100x more difficult than that of the low poverty school districts.

Here’s the district African-American student enrollment percentages:

AA rate

See any differences?

None of the highest growth districts in the country except for Chicago and Orleans serve many African-American students.

New Orleans and Chicago serve many students whose lives are still shaped by the deep, generational poverty that is rooted in our country’s horrific history of slavery.

The fact that their academic growth is amongst the very best in the nation should be a huge cause for celebration and hope.

What we talk about when we talk about high growth districts

It took me about 45 min to create the above charts. I did quick google searches so the numbers probably aren’t perfect, but they paint a pretty clear picture.

Yet in the 10+ articles I’ve read about the Stanford research data set not one has mentioned that New Orleans is the only majority black school district to be in the top ten growth school districts in the nation. 

And in case you’re wondering if the New Orleans data includes all of its schools, it does. I emailed the researchers and they confirmed that the data includes all district and charter schools in the city boundaries.

So let’s give a big shout out to the amazing educators, students, and families who achieved unprecedented gains in learning in New Orleans between 2009 and 2015.

The New York Times might not have noticed, but hopefully others eventually will.

Does anyone know why Chicago children are getting smarter?

If you just read the headlines, you might suspect that Chicago’s public schools are in a terrible tailspin. Part of this is the noise of big city politics. Part of this stems from cloud of violence that hangs over the city.

But Chicago has improved on academic test scores more than most other cities in the country. Rather than one of the worst, Chicago is one of the best.

A recent report by Sean Reardon and Rebecca Hinze-Pifer found that between 2009 and 2014:

“This [student achievement] growth rate [of Chicago] is higher than 96% of all districts in the US. Among the 100 largest districts in the country, the average growth rate from third to eighth grade is 0.95 grade equivalents per year; Chicago has the highest growth rate between third and eighth grade of any large district in the United States.”

The authors admit that they don’t know why this occurred.

I can’t prove why Chicago kids are getting smarter, but I have a hypothesis.

What’s Been Going on in Chicago Public Schools? 

One way to try and solve the mystery of why Chicago children are getting smarter is to look at the district’s previous major initiatives.

As this report details, between roughy 1990 and 2010 there were three overarching eras of reform in Chicago: the decentralization era, the the accountability era, and the do a lot of things era.

The authors are very careful to not attribute a causal relationship between reform eras and outcomes. The reforms were messy and not rolled out in an experimental manner – so fair enough.

But in this post I’ll try and make my best guess on what was causal and what was not.

The Decentralization Era

The decentralization era was best known for the creation of Local School Councils. This reform gave local councils real control over decisions about how schools were run. The councils were made up of school leadership, parents, and community members.

The councils always seemed like a terrible idea to me. It’s basically taking all we know about charter schools (good central offices, scalable instructional programs, governance matters) and doing the exact opposite!

Not surprisingly, research on the reforms found that the councils had some positive effects on advantage communities, but were least likely to improve schools in low-income communites. Communities with low social capital didn’t gain a lot from ad-hoc and poorly constructed local boards.

I’m very skeptical that the decentralization era and school councils were the root cause of later gains.

The Accountability Era

In 1995 Mayor Daly put in Paul Valls as the superintendent (I later worked with Paul when he was the superintendent of the RSD in Louisiana). Vallas, who did not have deep instructional expertise, used test driven accountability to try and make things better.

New tests, promotional standards, and interventions for failing schools were all put in place.

The reforms had better impacts for low-performing schools; the researchers noted:

“This was the only era to show large improvements in the lowest- achieving schools. However, the patterns in test scores in the lowest-performing schools suggest that some of the improvements resulted from instruction that was aligned specifically to the high stakes tests.”

This matches other research on accountability reforms: you tend to see gains in the lowest preforming schools, but the high stakes can cause narrowing of the curriculum.

 

The Do a Lot of Things Era

Arne Duncan came in after Vallas, and he instituted a lot of reforms.

Arne launched 100 new schools, implemented internal district instructional and curricular reforms, overhauled school leadership pipelines, and placed a deep focus on on-time high school progression.

Perhaps the biggest initiative of this era was the Renaissance 2010 project, which launched about a 100 new district, charter, and contract schools between 2005 and 2010.

Unfortunately, no one has conducted a full evaluation of the program. Someone should do this!

Two interim research reports came out around 2010. One study, which only included a few years of data from the early Renaissance cohorts, found that the new schools performed about the same as the existing district schools. The other study was inconclusive.

Not much help from the research community.

A lot of work was also done on school leadership. The Chicago Public Education Fund, in partnership with the district, invested heavily in school leader development, placing bets on both district based and non-profit providers.

The lastest research I could find on these programs found that “results indicate that one-year learning gains in elementary and high schools led by Fund-supported principals were not different than those in other similar schools.”

Another major reform, another mediocre result.

All told, researchers found that this era produced more gains in high school than elementary schools, but wrote: “while the effects of the dominant policies of Eras 1 and 2 are largely understood, much research remains to be done to understand both the positive and problematic effects of the policies in Era 3.”

Not super helpful, especially since this is the era that preceded the large gains in test scores that occurred after 2009.

What About the Charter Sector? 

CREDO published a report on Chicago charters that covered test scores from 2010 to 2012, which is right in the middle of the period where Chicago saw a lot of gains.

The study found +.01 effects in reading and +.03 effects in math. These effects amount to about a month or so of extra learning per year, maybe a bit less. Given Chicago’s relatively small charter market share, and the modest size of these positive effects, it’s unlikely that charters themselves accounted for the 2009-2014 gains.

A more recent study, which just looked at charter high school performance from 2010-2013, found much larger effects: +.2 effects on ACT related tests and much higher college enrollment rates.

These are large effects, but they are for high school only. The study lauding Chicago’s gains only covered grades 3-8.

So WTF Happened in Chicago to Make Kids Smarter?

To summarize: Chicago improved its test scores more than any other big city in the country, and researchers really don’t know why.

So why are Chicago kids getting smarter?

Here’s my guess: competition and accountability lifted all boats.

When you put accountability in place (the Vallas era) and then launch a 100 new schools (the Duncan era) you get a city where school leaders know there are consequences for failure and the best of the new schools begin to raise the bar for what’s possible.

This theory helps explain why the Renaissance schools and charter effects were a bit muted. In the studies on these reforms, researchers compared the new schools to existing schools. So if the existing schools were improving due to increased competition, you would not see large relative effects for the new schools.

I can’t prove that accountability and competition caused the results, but in many sectors accountability and competition make everyone better. It also fits stories we’ve seen elsewhere. In place like Denver and Washington D.C. increased competition led to all boats rising in the public school system.

If you have a better theory, let me know.

What Should Chicago Do Now? 

Here’s another tough question: if it was accountability and competition that caused Chicago’s gains, how should this impact Chicago’s future strategy?

Since 2002 (while the district was getting much better!) Chicago enrollment plummeted from 440,000 students to 370,000 students.

This means that there are lot of under-enrolled schools in the district and the city might have to go through another round of painful closures.

This also it means it’s harder to push the very reform (opening new schools) that might have driven Chicago’s previous gains in achievement.

So what should the city do?

Reasonable people can surely disagree, but I would continue to create new schools, albeit in a different fashion.

First, I’d open new schools in the areas where population is increasing. Chicago is made up of a lot of neighborhoods, and not all neighborhoods are losing children.

Second, I would do some replacement work. Instead of closing all the under-enrolled schools, I’d try and select some neighborhoods where there’s enough child density that you could imagine families coming back to the public schools if there were better options. I’d launch replacement schools in these neighborhoods.

There are clear drawbacks to this strategy. Politically, it’s hard to justify opening schools when you’re in the midst of closures. Programmatically, it’s hard sell to get the operators of new schools to open up in neighbors with shrinking enrollment.

But I think it’s the best thing for children.

Lastly, I might also try and launch some diverse by design schools.

In a city as diverse as Chicago, it’s sad that it’s schools are so segregated.

The Last Word

Chicago’s Chief Education Officer, Janice Jackson, recently gave her take on why things are better.

Her list: pre-k, better professional development, better curriculum, competition from private and charter schools, and clear accountability standards.

In her own words:

“I believe the level of transparency we have provided around what a quality school is has been transformational in this district.”