Category Archives: Portfolio

Could Newark have achieved more?

Harvard researchers just published a study on Newark.

In large part due to Mark Zuckerberg’s $100 million gift to the reform efforts, the city has seen a lot of coverage, including the well done book The Prize.

Most of this coverage has been negative.

The Harvard Study: Teasing Out Annual Effects From 2010-2016

The Harvard study found that the Newark reforms, in the most recent year of the study, had a positive impact on ELA (.07 SD) and no impact on math. In the early years of the reforms, the achievement effects were negative.

Unfortunately, the researchers only looked at annual effects (were there positive or negative effects in any given year) rather than calculating cumulative impacts (what was the total impact over these years). Because of this, we can’t perfectly compare the Newark results with results of other city based studies, such as Doug Harris’ evaluations of the New Orleans efforts.

What Caused the Positive Effects? 

While the authors didn’t calculate the cumulative effects of the reforms, they did do something wonderful.

The authors separated out the effects of two different strategies: (1) improving existing schools vs. (2) expanding high-performing schools, closing low-performing schools, and facilitating the transfer of students out of low-performing schools and into high-performing schools.

The improving existing schools approach included replacing large numbers of principals, renegotiating the union contract, implementing new data systems, and extending learning time.

The open / close / shift enrollment approach included adding to the enrollment of high-performing charter and district schools, closing underperforming schools (11 traditional schools and 3 charters), and implementing a unified enrollment system that made choosing schools much easier for families. These reforms increased charter enrollment from 14% to 28%.

The results are striking.

In math, the improve strategy achieved a .08 negative effect in math while the open / close / shift strategy had a positive .04 effect.

In ELA both strategies had positive effects, but the open / close / shift was responsible for 62% of the overall positive effect.

Perhaps most importantly: the open / close / shift strategy achieved positive effects in every year of the study. 

Opening and closing schools, and shifting student enrollment, increased student learning.

Could Newark have Achieved New Orleans’ Level Gains?

In roughly the same number of years as the Newark reforms, New Orleans achieved positive .2-.4 standard deviation effects.

New Orleans, of course, relied much more on the open / close / shift than it did on improvement.

As noted above, we can’t do a direct apples-to-apples comparison of the studies. But I’d be surprised if Newark’s cumulative effect is as large as the New Orleans effect given that Newark saw negative effects in 3 of the 5 years of reforms.

We’ll never know if Newark could have achieved similar gains to New Orleans, but in many ways the Newark circumstances were more enviable than those of New Orleans.

Newark is home to one of the highest-performing charter sectors in the country, while New Orleans had no large effective charter operators in the city right after Katrina.

Newark’s initial philanthropic investment was much larger than that received in New Orleans after Katrina. And the public per-pupil in Newark is about 3x that of New Orleans.

Newark students also didn’t have to go through the immense trauma of Katrina.

That being said: perhaps New Orleans style reforms would have not worked in Newark.

Maybe the politics would not have allowed for it… though it’s worth remembering that Shavar Jeffries, now the CEO of Democrats for Education Reform, was only about 2,000 votes away from beating Ras Baraka in the 2014 Newark mayoral race.

Or perhaps the charter sector would not have been able to grow much faster than it did; notably, charter effects did decline as they enrolled more students… though perhaps more charter capacity would have emerged under better conditions.

All told, my guess is that Newark did not increase educational opportunity as much as it could have.

A 2010 Prediction 

On this blog, I try to be honest about both my mistakes and successes.

It would be disingenuous for me to write a post arguing that, in 2010, no one could have predicted which strategy would have been better.

The fact is that I did predict this. In 2010, when it became clear that there was going to be major reforms in Newark, I created a powerpoint deck to give advice to national educational leaders. In the deck, I modeled out how quickly I thought you could expand great schools and close failing schools.

Given the relatively small size of Newark public schools (50,000 students), as well as the number of high-quality charter schools, I felt confident that the vast majority of Newark students could attend a high-quality school within ten years.

This ended up being the strategy we took in New Orleans.

It was not the strategy taken in Newark.

Learning 

I have a lot of love for the philanthropists, political leaders, and educators who spent the good part of a decade trying to make things better for kids. I know many of them well and have had nothing but positive interactions.

In terms of rigorous third party research results, we also knew a lot less in 2010 than we do now. There have been a wave of recent studies documenting the positive effects of expanding the best schools and closing the worst.

And while we still suffer from a dearth of research on citywide turnaround efforts, outside of the early results in Lawrence, Massachusetts, we really haven’t seen a citywide turnaround reform effort significantly raise achievement without a focus on opening and closing schools (research has shown that Washington D.C. improved their public schools, but the city also leaned heavily on opening and closing schools).

So perhaps the better question is not what should have Newark leaders done in 2010, but what should similarly situated civic leaders do in 2017?

Civic leaders should embrace strategies that maximize the expansion of the best schools and close the worst schools.

Of course this need not be the only focus of a city turnaround effort, but the Newark study has increased my belief that it should be the primary focus.

CREDO’s school closure research validates portfolio and golden tickets

 

CREDO just came out with a study on school closures. Matt Barnum gives a good write up in Chalkbeat (and continues to far surpass NYT and WAPO in his analysis of complicated research).

Achievement increases when you close low performing schools and students transfer to better schools 

Overall, the research increased my belief in the idea that great schools should expand and failing schools should be closed or transformed (the basis of the portfolio model).

My only reservation with this study is that it defined low-performing schools by absolute performance rather than growth in achievement; however, student achievement still grew when students moved from lower to higher absolute performing schools, so perhaps many of the low absolute schools were low growth schools as well.

The research adds to the body of evidence that shows: if you…

(1) Close lower performing schools;

(2) Increase the number of high-performing schools;

(3) Ensure students from the closed schools get into the higher-performing schools; then

(4) Educational opportunity will increase.

See below from CREDO on the student achievement effect of a student transferring from a closed school to a superior school.

Screen Shot 2017-08-27 at 9.12.18 AM

Screen Shot 2017-08-27 at 9.12.29 AM

One other thing: this type of analysis doesn’t capture the positive effect of the failing school no longer existing; a city implementing these strategies should see additional gains from no new students ever having to attend the failing school.

“Golden ticket” policies can ensure students attending closed schools get into better schools

Cities that use unified enrollment systems can easily guarantee that students leaving closed schools have access to better schools by a “golden ticket” policy. This policy gives students exiting closed schools first access to any open seats in high-quality schools in the city.

While I find the policy name “golden ticket” to be crass, it’s a policy that can do wonders for educational equity.

Too often students in failing schools are shuffled from one underperforming school to another; a golden ticket policy can prevent this.

Should we support increased school closures in majority white communities?

Numerous commentators pointed out that schools with +80% minority students were more likely to be closed than schools that had lower minority enrollment.

Most took this as a sign of inequitable treatment toward minority communities.

However, a growing body of research indicates that school closure increases educational opportunity so long as student have access to better schools. And although I wish this wasn’t the case, my hunch is that majority white communities likely have a higher concentration of better schools than your average minority community: this means that school closures are likely to be even more effective in majority white communities.

My guess is a lack of closures in majority white communities is leading to reduced educational opportunity.

School closure is incredibly hard

While I believe that thoughtfully implemented school closure policies will benefit children, I know that closing schools is hard for students, families, educators, and politicians.

But sometimes doing the hard thing can help students.

And a growing body of evidence is pointing to the idea that a well implemented school closures is one of those hard things that can ultimately make things better for students and communities.

__

Lastly: thinking of everyone in Houston right now. Really hoping that friends, colleagues, and everyone else in city are ok.

SaveSave

SaveSave

Phase II of the charter and choice research agenda is extremely important

In a recent post, I summarized much of the research on charter school and high choice cities.

The short of it: there’s a strong body of evidence that urban charter schools outperform traditional schools, and a nascent body of evidence that thoughtfully implemented high choice cities can deliver strong achievement gains.

It’s worth taking a moment to celebrate this: the research on urban charter schools is an impressive body of work built on twenty years of studying the incredible efforts of educational entrepreneurs. It’s wonderful to see these schools working.

___

But as charters scale, additional questions need to be studied.

More specifically:

  1. Do urban charter school students achieve better long-term life outcomes?
  2. What happens to traditional schools when charter schools expand?
  3. What happens to students when under-enrolled traditional schools eventually close?
  4. What happens to cities that transition to majority charter systems?

A few recent studies have shown that charter school students achieve better long-term life outcomesthat charter school expansion can improve traditional schools; that student attending schools that are closed actually benefit from the closure so long as good new schools are being continually opened; and, as noted in the previous post, that majority charter cities can deliver strong achievement gains.

But there are also studies that indicate the opposite. Charter students in Texas did not have greatly improved life outcomes; students in Baton Rouge did not increase achievement after their schools were closed; and cities such as Detroit continue to struggle despite high charter penetration.

___

Personally, I most care about questions (1) and (4). If charter schools end up delivering better life outcomes, and majority charter cities work, I’d be willing to accept some interim negative effects of charter expansion and school closures. But I understand that the latter effects could cause hardship for families, and I’m heartened to see early examples that show that charter expansion can be coupled with gains for all students.

So here’s to holding out for the best scenario: I truly hope that phase II of the research agenda shows that charters increase life outcomes, that charter school expansion and failing school closure benefits all students, and that majority charter systems deliver benefits for all students.

Initial research indicates that this might be possible, which is really exciting.

Applying portfolio reforms to postsecondary problems

I just got back from a trip to New Orleans, which continues to be a well of friendship and inspiration.

I. Where should you spend the next philanthropic dollar? 

In a few conversations, the following questions came up:

  1. Are the kids we serve going to succeed in life after high school? What will their lives be like when they are 30? Will they be living meaningful and happy lives?
  2. Is the marginal dollar of philanthropy best spent on making the K-12 system better (after 10 years of improvements) or trying to overhaul the post-secondary landscape?
  3. If you wanted to radically improve post-secondary, what would you do?

II. Post-Secondy portfolio 

The K-12 portfolio mindset entails viewing an educational system in terms of operators (running schools) and seats (how many students are served).

This mindset could also be applied to post-secondary.

By 2020 or so, New Orleans will be graduating around 3,000 students a year.

Let’s say that about 1,500 of them will be prepared to succeed in a four year college; 1,000 of them will be prepared to succeed in a 1 to 2 year credentialing program; and 500 of them will need deep support to enter the workforce and exit crisis situations.

Of the four year college students, you might need 500 to 1,000 “KIPP to College” type supports to ensure students make it through.

For the credentialing programs, you’d need 1,000 seats that can reliably produce students with employable credentials.

For the crisis students, you’d need employment and social service operators that could transition students into jobs.

III. Post-Secondary investment intermediaries 

Instead of assuming this will naturally happen in New Orleans (or any other city), you could capitalize a new or existing non-profit intermediary to launch, recruit, and support post-secondary providers.

At the outset, the intermediary would create a business plan where it laid out how money it would need to get X% coverage on the aforementioned 3,000 seats.

High-Quality existing local providers (like the coding bootcamp Operation Spark) could cover some of the seats, and national providers like Match Beyond could be recruited in.

Overtime, you’d expand what was working, close what wasn’t, and support new entrepreneurs to keep innovation going.

IV. Getting funding streams right

Most states subsidize mediocre public universities; the federal government tops this off with Pell grants.

To make the 3,000 seat post-secondary strategy viable, you’d need to blend a mixture of public support and tuition to make providers sustainable.

Louisiana’s course choice provides a revenue stream for programs that started working with kids while they’re in high school.

Creating a new university that housed many of these programs could allow for the accessing of Pell grants.

Wage contigent loan programs could also be an option for programs that were consistently placing graduates in high-performing jobs.

V. Who are the entrepreneurs that will seek out the 10x play?

The early New Orleans K12 entrepreneurs felt that they could deliver something to students that was significantly better than the existing system.

They were right.

A post-secondary transformation won’t happen on its own.

It will take a set of entrepreneurs to put forth a plan, galvanize funding, and spend a decade building the new system.

Is this the right play? If so, who will step up?

Only Normal Things Scale

Sometimes, when people ask me what I do, I’m tempted to say that I’m trying to make the concept of relinquishment normal.

Right now, it is not normal to let educators operate their own schools; to let families choose amongst these schools; and to transition to government to more of a regulatory role.

Rather, it is normal for the government to directly operate schools; for families to be assigned a school based on their address; and for regulation to be handled by the same entity that is operating schools.

Every Adopter Reduces Psycho-Social Barriers for the Next Adopter 

New Orleans was the first city to build an education system on the aforementioned principles, and it did so in a very abnormal situation.

New Orleans (in so many ways!) is not normal.

Perhaps if 8-10 cities adopt similar principles, then these ideas will become modestly normal.

And then perhaps 10-20 other cities will start pushing that direction.

The more normal it is, the more quickly it will be adopted. Each marginal user slightly lowers the psychological barrier for the next user.

“Vote for Abnormality!” is Not a Winning Slogan 

Most people react negatively to abnormal things. This is why in Massachusetts, the home of the nation’s highest performing charter schools (in Boston), people in the suburbs will likely vote against eliminating the charter school cap.

For people in the suburbs, neighborhood public schools – as well as private schools – are normal. Charter schools are not normal.

When you’re in a referendum, you don’t want to be the abnormal option.

People’s Fidelity to Normality is Much Higher than Their Fidelity to Ideas 

Many people disagree with an idea when it is abnormal and then agree with the idea when it becomes normal.

This, for example, is why I think getting to ~50% charter market share is so important. All of a sudden, charters become normal – and the people who previously did not like charter schools become ok with them.

Other policies such as unified enrollment, unified accountability, the transformation of operators for failing schools…. all these things can become normal over time, as New Orleans has shown.

Abnormal is for the Moment of Disruption, Normal is for Scale

Entrepreneurs who come up with amazing ideas are often very abnormal people; however, to scale their disruption, they generally have to do a lot of normal things, including convincing others that their new idea will become the new normal.

Sometimes they fail to make this transition.

Equally problematic: sometimes people who are trying to disrupt things act too normal. They say they want to change the world, but ultimately they want to be liked… and be normal.

This doesn’t work either.

So you need congruence between your current level of normality and the level of normality that the situation requires for you to be successful.

This can be tricky to pull off.

The Answer is 6.7 Miles. What is the Question?

Screen Shot 2016-06-15 at 8.01.52 PM

The question is: how far, on average, would a family send their child to attend a school that is in the highest category of the state accountability system compared to a school in the lowest category of the state accountability system?

This is from a recent report on the DC public school system. The analysis, while useful, isn’t perfect in that it only includes families who utilized the enrollment system, but it does add to the emerging literature on the revealed preferences of families that participate in transparent enrollment systems.

 

___

Here’s another answer: it increases racial integration.

The question is: does DC’s unified enrollment system increase or decrease racial segregation?

Screen Shot 2016-06-15 at 8.06.39 PM

Shockingly enough, assigning families to neighborhood schools that are zoned by property values is not a great way to decrease segregation.

 

___

Answer: Unclear.

Question: Do parents care about a school’s academic growth (as opposed to absolute test scores)?

Screen Shot 2016-06-15 at 8.20.46 PM

Interesting but not shocking. Parents probably care a lot about peers and status.

Also interesting, this seems more true of low-income families:

Screen Shot 2016-06-15 at 8.25.26 PM

This raises an interesting question for policy makers: given that growth more accurately measures a school’s impact, should they design grading systems that prioritize growth (as DC’s charter framework does) even though low-income parents might care more about absolute scores?

Or perhaps not – maybe low-income families aren’t considering the growth based performance framework because the government is hiding this information:

Screen Shot 2016-06-15 at 8.27.05 PM

___

One last answer: Families who aren’t assigned to a school in the lowest performance category, as well as the politicians and superintendents who seek their favor.

The question: who loves neighborhood schools?

It remains shocking to me that public leaders in cities such as Oakland are vehemently opposed to unified enrollment on the grounds that such systems will undermine public education.

The only thing a unified enrollment system undermines is the privilege of those who benefit from institutional racism and widespread income inequality.

 

 

NOLA Return Bill: Is it Good?

Screen Shot 2016-05-13 at 2.09.37 PM

So far, many articles focused on the new bill have concerned themselves with raising or lowering the status of specific players – but what about the bill itself?

___

Overall, I think the bill is strong on equity, democracy, and protecting existing academic gains.

I worry about what the future holds for maintaining a sense of urgency on increased academic performance and innovation. A cementing of structure will likely favor the incumbents.

But a law can only do so much.

A good framework has been set. The return bill enshrines much of what has led to NOLA’s gains in equity and performance.

In a world of toxic national politics, it’s heartening to see complex and important legislation being crafted and adopted by a politically and racially diverse coalition.

___

More reflections:

  1. This is nationally important: we now have a law on the books that clearly defines how a portfolio district can be implemented. A debt is owed to Paul Hill.
  2. The bill is strong on equity: the weighted student funding, enrollment, and expulsion provisions are extremely important. As I’ve written before, I think NOLA’s greatest innovations have been in equity.
  3. The bill is sticky: with fairly broad support, the bill codifies 10 year’s worth of work – this should make it stick. Moreover, the marketshare limits and charter autonomy provisions should prevent roll backs to a one best system.
  4. The return is really a rebalancing: the school board now has much more power, and the state has much less – however, the state still has the RSD as a check on the local school inaction on underperforming schools. Democratic power exists at both local and state levels, and this is a readjusting rather than a jettisoning of state power.
  5. Who will be the leaders? Many of the local leaders in education reform have been unelected: non-profits and civic organizations have held a lot of power. This will shift as the local school board gains more power, but with elections happening this fall, we don’t yet know who this slate of leaders will be. The first wave of local leaders post-return will set a vision and culture for what local control means…
  6. Incumbents are protected – but what about future schools? The bill outlines clear powers for the school board, superintendent, and existing schools, but less ink was spilled on ensuring there is a continually pipeline of new (and hopefully innovative) schools. I think this amongst the biggest risks in NOLA over next decade: will the incumbents of the system (government, charter schools, non-profits, etc). utilize their hard and soft power to block new entrants?

___

Lastly, here are what I think to be the most important parts of the bill – with excerpts of the exact language for those who want to understand the mechanics.

Equity Provisions 

Student based budgeting w/ weights for specific needs: “…establishes a process to determine the district-level funding allocation to be effective beginning July 1, 2017, and as revised in subsequent years as appropriate, based upon student characteristics or needs…”

Unified enrollment and expulsion: “…shall require all charter schools under the board’s jurisdiction to participate in the parish-wide enrollment system and student expulsion process, according to policies established by the board…”

Allows for some (but not exclusive) neighborhood preference: “May provide a lottery preference for enrollment at elementary middle schools under the board’s jurisdiction for students residing with defined geographic zones as one of the factors to determine student assignment, according to policies adopted by the board. Such preference shall be applied to not more than one-half of the seats available in each grade level…”

Promotes integration: “…so that such schools shall be exempted from the minimum enrollment percentages…”

Performance Provisions

Prevents monopoly / too big too fail: “shall adopt a policy establishing a process which allows the local superintendent to limit the percentage of system enrollment that any single operator of schools or charter governing authority may serve to ensure that a diverse system of schools led by multiple high quality operators exists at all times.”

Empowers superintendent as portfolio manager: “superintendent shall present recommendations to the local school board regarding the approval, extension, renewal, or revocation of the charter for any charter school under the board’s jurisdiction…Unless vetoed by a two-thirds vote of the full membership of the board, the local superintendent may implement any such recommendation submitted to the board.”

Protects charter autonomy: “the local school board shall not impede the operational autonomy of a charter school under its jurisdiction.”

Promotes test security: “each charter school under the local school board’s jurisdiction shall provide for independent test monitoring from third-party entity approved by the school board for the testing period immediately preceding the board’s consideration of renewal of the charter school’s contract.”