Category Archives: Advice

5 Replies

Harvard researchers just published a study on Newark.

In large part due to Mark Zuckerberg’s $100 million gift to the reform efforts, the city has seen a lot of coverage, including the well done book .

Most of this coverage has been negative.

The Harvard Study: Teasing Out Annual Effects From 2010-2016

The Harvard study found that the Newark reforms, in the most recent year of the study, had a positive impact on ELA (.07 SD) and no impact on math. In the early years of the reforms, the achievement effects were negative.

Unfortunately, the researchers only looked at annual effects (were there positive or negative effects in any given year) rather than calculating cumulative impacts (what was the total impact over these years). Because of this, we can’t perfectly compare the Newark results with results of other city based studies, such as Doug Harris’ evaluations of the New Orleans efforts.

What Caused the Positive Effects?

While the authors didn’t calculate the cumulative effects of the reforms, they did do something wonderful.

The authors separated out the effects of two different strategies: (1) improving existing schools vs. (2) expanding high-performing schools, closing low-performing schools, and facilitating the transfer of students out of low-performing schools and into high-performing schools.

The improving existing schools approach included replacing large numbers of principals, renegotiating the union contract, implementing new data systems, and extending learning time.

The open / close / shift enrollment approach included adding to the enrollment of high-performing charter and district schools, closing underperforming schools (11 traditional schools and 3 charters), and implementing a unified enrollment system that made choosing schools much easier for families. These reforms increased charter enrollment from 14% to 28%.

The results are striking.

In math, the improve strategy achieved a .08 negative effect in math while the open / close / shift strategy had a positive .04 effect.

In ELA both strategies had positive effects, but the open / close / shift was responsible for 62% of the overall positive effect.

Perhaps most importantly: the open / close / shift strategy achieved positive effects in every year of the study.

Opening and closing schools, and shifting student enrollment, increased student learning.

Could Newark have Achieved New Orleans’ Level Gains?

In roughly the same number of years as the Newark reforms, New Orleans achieved positive .2-.4 standard deviation effects.

New Orleans, of course, relied much more on the open / close / shift than it did on improvement.

As noted above, we can’t do a direct apples-to-apples comparison of the studies. But I’d be surprised if Newark’s cumulative effect is as large as the New Orleans effect given that Newark saw negative effects in 3 of the 5 years of reforms.

We’ll never know if Newark could have achieved similar gains to New Orleans, but in many ways the Newark circumstances were more enviable than those of New Orleans.

Newark is home to one of the highest-performing charter sectors in the country, while New Orleans had no large effective charter operators in the city right after Katrina.

Newark’s initial philanthropic investment was much larger than that received in New Orleans after Katrina. And the public per-pupil in Newark is about 3x that of New Orleans.

Newark students also didn’t have to go through the immense trauma of Katrina.

That being said: perhaps New Orleans style reforms would have not worked in Newark.

Maybe the politics would not have allowed for it… though it’s worth remembering that Shavar Jeffries, now the CEO of Democrats for Education Reform, was only about 2,000 votes away from beating Ras Baraka in the 2014 Newark mayoral race.

Or perhaps the charter sector would not have been able to grow much faster than it did; notably, charter effects did decline as they enrolled more students… though perhaps more charter capacity would have emerged under better conditions.

All told, my guess is that Newark did not increase educational opportunity as much as it could have.

A 2010 Prediction

On this blog, I try to be honest about both my mistakes and successes.

It would be disingenuous for me to write a post arguing that, in 2010, no one could have predicted which strategy would have been better.

The fact is that I did predict this. In 2010, when it became clear that there was going to be major reforms in Newark, I created a powerpoint deck to give advice to national educational leaders. In the deck, I modeled out how quickly I thought you could expand great schools and close failing schools.

Given the relatively small size of Newark public schools (50,000 students), as well as the number of high-quality charter schools, I felt confident that the vast majority of Newark students could attend a high-quality school within ten years.

This ended up being the strategy we took in New Orleans.

It was not the strategy taken in Newark.

Learning

I have a lot of love for the philanthropists, political leaders, and educators who spent the good part of a decade trying to make things better for kids. I know many of them well and have had nothing but positive interactions.

In terms of rigorous third party research results, we also knew a lot less in 2010 than we do now. There have been a wave of recent studies documenting the positive effects of expanding the best schools and closing the worst.

And while we still suffer from a dearth of research on citywide turnaround efforts, outside of the early results in Lawrence, Massachusetts, we really haven’t seen a citywide turnaround reform effort significantly raise achievement without a focus on opening and closing schools (research has shown that Washington D.C. improved their public schools, but the city also leaned heavily on opening and closing schools).

So perhaps the better question is not what should have Newark leaders done in 2010, but what should similarly situated civic leaders do in 2017?

Civic leaders should embrace strategies that maximize the expansion of the best schools and close the worst schools.

Of course this need not be the only focus of a city turnaround effort, but the Newark study has increased my belief that it should be the primary focus.

Education philanthropists should not take advice from Larry Summers

4 Replies

Screen Shot 2017-09-20 at 7.28.42 PM.png

Tyler Cowen just Larry Summers.

In a blog post about the interview, Tyler wrote: “if you think you know someone who is very smart, Larry is almost certainly smarter.”

This may be the case when it comes to economics. While I’m in no position to evaluate his economic policy claims, I found Summers to be reflective, curious, and thoughtful. He seems like the kind of person I would enjoy working with.

But Summers also discussed education philanthropy, and I came away with a strong belief that I almost certainly smarter than Summers on this subject.

I don’t say this because I for sure know that I’m right and Summers is wrong; rather, I say this because I have a firmer grasp of the research, more hands on experience, and a clearer strategic vision for scalable and sustainable change.

Given that Summers likely has a good 20-30 IQ points on me, and that he has risen to the top of an extremely competitive field, the fact that I’m likely smarter than Summers in this area is a testament to the powers of specialization and the domain specific nature of knowledge.

How should you spend a $100 million?

In the interview, Tyler asked Summers how he would advise a philanthropist in St. Louis who wanted to give away a $100 million to help her city. After admitting the he knew little about St. Louis, Summers answered the question more generally, and said that he would focus on public education.

As it happens, my job is to advise philanthropists who want to improve public education. Currently, our team manages the philanthropic giving for Reed Hastings and the Laura and John Arnold Foundation.

Given my day job, I was curious to hear how Summers would respond.

Moreover, while he may not remember, Summers was once asked about relinquishment during an interview on education reform (I was in attendance and vividly remember him saying “Ah, relinquishment…”), and I was curious to see if my ideas had impacted him at all.

They have not!

Summer’s advice: avoid charter schools and work outside the system

In the interview, Summers gave two pieces of advice to education philanthropists:

Avoid charter schools: Too many philanthropists set-up charter schools that cream students, pay teachers salaries that are not sustainable on the public dollar, and then ultimately cannibalize the traditional system of good students, good teachers, and public funds.
Avoid the K12 system: Instead of trying to tackle the core K-12 system, it’s better to fund efforts that work around the system, such as after school or summer school.

I think both of these points are wrong.

Summers ignores a large evidence base on charter schools

I recently summarized the evidence on charters schools on this blog. Summers ignores most of this research:

Achievement: Urban charter schools outperform traditional public schools, posting annual effects of .05-.1 standard deviations. This holds true with both quasi-experimental designs (where researchers try to control for student selection) and experimental designs (where student selection is randomized). Charters are not achieving their impacts because of student creaming.

Funding: Charter schools, on average, receive much less funding than traditional schools. As I previously wrote about, in numerous cities where charter schools receive less money, they still outperform the traditional system.

Teacher Pay: Nationally, traditional school teachers have higher average salaries than charter school teachers. And while some of this is due to the effect that charters hire younger teachers, I have seen no research that indicates that, at scale, charters are picking off the best teachers by offering them unsustainable salaries.

Impact on Traditional Schools: Lastly, most research shows that charter schools have positive or neutral effects on traditional school achievement. Moreover, cities that have improved their educational systems over the past decade have often seen rising charter school enrollment during the same period. Washington D.C. and Denver stand out as primary examples of cities where all schools got better as charter schools expanded.

All boats rising – and not cannibalism – is the norm.

It appears that Summers is reasoning from anecdote rather research.

I am sure there are some charter organizations that cream students and spend way above the public dollar (I can think of a few!), but these are outliers.

At scale, urban charter schools achieve more and spend less than traditional public schools.

Working outside the system is low impact and not leveraged with existing public funds

Summer’s second piece of advice – work outside the system rather than fix the system itself – is also flawed.

Yes, fixing the system is hard. But kids spend a lot of time in the system. It will be very difficult to improve public education if you ignore what happens to students from 8 AM to 3 PM for 13 years.

Moreover, to the extent that a philanthropist funds an outside the system intervention that works, the only way to scale the intervention is with more philanthropy or increased public revenues. There is no leverage with existing public dollars.

While I am not against raising additional public revenue for things that work, I think we should spend most of our energy improving the effectiveness of the dollars we already spend, especially given that systems level K12 interventions (like urban charter schools), are achieving success at scale.

If there was no evidence that the system could be fixed, I would tend to agree with Summers. But as more and more cities breakthrough and achieve citywide gains, the logic of working mostly outside the system is increasingly flawed. The one exception I’d make to this claim is pre-school, which has a reasonably strong evidence base and is increasingly funded with public dollars.

If you are a philanthropist who wants to improve public education in your city, please contact me

In the event that Tyler’s question was not hypothetical in nature, and that there is a philanthropist in St. Louis who wants to donate a $100 million, I do hope she contacts me (neeravkingsland at gmail) rather than takes Summers’ advice.

I am a firm believer that philanthropy well spent can forever positively alter the trajectory of a city’s public educational system.

And while those of us advocating for systems level change still have much to prove, we now have numerous examples of cities achieving citywide improvements for their most at-risk students. Philanthropists should double down on their successes, evolve the model based on local conditions, and continue to fund further research so we can keep on learning.

Career advice for young people who care about education

2 Replies

Bill Gates recently just tweeted his career advice to you people:

Screen Shot 2017-05-20 at 11.10.38 AM

It’s great advice.

But to the extent you want to work in education, here’s where I would focus on.

I. 10 Million Students

There are about 50 million public schools students in the United States.

According to the NCCP, about 44% of these students live under 200% of the poverty line (~$50K for family of 4). And about 22% live under the cover line (~$25K for family of 4).

For this modeling exercise, let’s set the goal scaling great education opportunities to the poorest 30% of public schools students.

In most cities I work in, usually around 25% of low-income students are currently being served by high-quality schools.

That leaves about 11.25M students underserved.

To make the math a little easier, let’s call it 10M students.

To date, charter schools are one of the few interventions that have consistently shown positive effects.

Roland Fryer recently reviewed 196 studies and identified charter schools as one of four interventions that seem to work.

If we want to meet the needs of these 10M students, scaling high-performing charter schools is a solid big bet to work on.

Of course, it’s not the only possible path to better serving 10M students, but I think it’s the most likely path for success.

If I was young and figuring out what to do, this is where I’d begin my career.

II. Getting to 10 Million: the 50K CMOs

KIPP, which serves around 80K students, has already passed the 50K mark. This is amazing.

Others, such as IDEA public schools, and Success Academies, have publicly stated they want to get near or surpass the 50K mark.

A few organizations I know of our considering this type of growth as well.

All in, let’s say that over the next decade we’ll serve 500K students with the highest growth CMOs.

III. Getting to 10 Million: the 10K CMOs

I did a quick scan of CMOs I’m familiar with and identified another 20 CMOs or so that have growth plans for around 10K students.

There are also a lot of 1-2K CMOs coming out of various incubators, so let’s assume another 20 or so emerge.

Give or take, that’s another 500K students.

IV. Where Will the Other 9 Million Come From?

Assuming current trends continue, and we don’t see that many +100K CMOs emerge, then we’ll need to build a lot of 10K CMOs.

About 900 of them… are there 900 people in this country who can operate high-quality 10K student CMOs?

Or 1,800 5K CMOs… are there 1800 people in this country who can operate high-quality 5K student CMOs?

Or maybe CMOs will start to scale and we’ll need 20 500K CMOs… are there 20 people in the country who could accomplish this amazing feat?

You get the idea.

I don’t know how the sector will develop.

As a career choice, it’s interesting to think about helping a single CMO scale to 500K or to try and lead a CMO that gets to 10K.

V. The Incentives to Scale

The more I ponder this question, the more and more I keep coming back to incentives.

Places like Silicon Vally intentionally construct every incentive toward scale: founder wealth comes from equity and investor wealth comes 10-20% of investments being home-runs.

Even more physical companies (fast food chains, retail stores, etc.) operate under similar incentives.

In the charter world, the way to get large amounts of philanthropy is to grow, but this money is different: the investors are losing money (they give it away) and the founders are personally gaining nothing (all the money goes to their organization).

Spot the difference?

All that being said, we do have for-profit charter schools in this country, and they have failed to produce great outcomes at scale for children.

Another twist: the best emerging international school organizations have often been for-profit organizations.

So why has the profit incentive had more effect intentionally than domestically?

I’d try to think about that if I was young and trying to scale great schools.

VI. The Knowledge and Technology to Scale

Even if the incentives are right, sometimes a job is just too hard to achieve with our current knowledge and technology.

Perhaps the reason we only have ~50 high-quality scaled CMOs is that right know our knowledge and technology significantly restricts the amount of people who can succeed as a CMO leader.

It’s possible that further codification of knowledge and better software could increase the number of high-quality CMOs.

Maybe that’s a problem you could spend your life solving.

VII. It’s a Hard Problem

This is why we need great people working on it!

How to make strategy less confusing by getting rid of the word strategy

2 Replies

For many years I thought strategy was one thing when in reality it is two things.

I thought strategy was answering the question: (1) how will you achieve your mission?

In reality, strategy is answering two questions: (1) what will you do? and (2) how will you succeed?

I thank Patrick Lencioni for helping me achieve this clarity, as well as our current team for thoughtfully working through both of these questions when initially I had only worked through the first.

___

For example: a non-profit’s mission might be to increase economic opportunity for low-income families.

This non-profit might then say that their strategy is to operate charter schools, raise money for post-secondary scholarships, and partner with local employers for job placements.

In this answer, the organization is answering the question: what will we do?

Alternatively, the non-profit might not mention the above and instead say that their strategy is to form incredibly deep partnerships with their families, develop the best teacher coaching program in the country, and use data analysis to find great college and career matches for their students.

In this answer, the organization is answering the question: how will we succeed?

Because “strategy” has evolved in to too broad of a concept, either answer might be deemed acceptable.

But it’s vial that an organization answers both questions.

As a leader, you need to be very clear about what is that your organization will do to achieve your mission.

As a leader, you also need to be very clear about how your organization will out-perform other organizations that are doing the exact same thing.

___

Most often, I see leaders in the non-profit sector make the same mistake I made: they have only thought hard about what they will do.

Too often, leaders don’t spend enough time answering the question: how will we outperform everyone is who doing the exact same thing?

A taxonomy for how to get better faster

3 Replies

This may seem a little harsh, but I think it’s mostly true. And it’s important in that constant learning is a vital competitive advantage.

All feedback welcome to make the framework better.

Screen Shot 2017-03-06 at 7.36.37 PM

Advice -> Endorphins or Habit Change?

1 Reply

There are many inspirational posters on many walls across the world.

Most of these posters do not change the behaviors of those who purchase them, frame them, hang them, and look at them.

Rather, most of the posters deliver a sequence of endorphin boosts that very quickly fade.

In a short matter of time you pass the poster and you feel nothing.

Why Do People Seek Advice?

Most people who seek advice are not that serious about changing their behavior.

They want to change their behavior, and they want to feel the endorphin rush of wanting to change their behavior, but they do not want to put in the work that behavior change requires.

People like to feel that they can change.

People like to get advice on how to change.

People do not like the process of change.

If you are seeking advice, you’d do well to be aware of why you are seeking advice.

If you are giving advice, you’d do well to clearly articulate what it will require to implement this advice.

Or, at the very least, understand an advice session for what it is: a form of human banter that makes everyone feel good at the time but has little lasting effect.

What Advice Have You Received that Has Become a Habit?

It’s worth reflecting on what advice you have received that has become a mental habit, both to reflect if you’re doing the hard work of behavior change, as well as to understand why some advice leads to change and some does not.

Here are some pieces of advice that have truly change how I think and act:

Nobody promised us anything (my father): As my father was dying from Parkinson’s disease, I asked him if he was sad, and he responded: nobody promised us anything. I think about this phrase a lot – as well as a sister phrase that I’ve incorporated into my thinking: the world is not ordered for my own happiness. When I am feel frustrated, indignant, or consumed with self-pity, I say this phrase to myself and, in the moment, reorient my mindset as best as I can, which is often a good amount.

Workout and mediate everyday (self-help books): Basically every self-help / self-improvement book I’ve read and have hammered home the importance of exercise and meditation. And for good reason! It’s taken time, but I’m not at least 5-6 days a week for both. Working out and meditation have become long-term habits.

Lead congruent organizations / teams (Nancy Euske, NSNO management team): After a few initial failures, I am now very deliberate about leading teams and organizations that have an explicitly aligned mission, strategy, culture, structure, tactics, people management systems, and goals.

I’m sure there are other pieces of advice that have become habits – but these standout to me in that they have greatly impacted my life, five years ago I did none of these three with any regularity, and I continually track my behavior to ensure the habits stick.

The Biggest Mistake I Made with Habits

For years, I would read books for knowledge rather than behavioral change. For some types of books (novels, history, etc.) this is fine. For many other types of books (business, self-help, etc.), this is not fine.

But I get high off learning new information, which is dangerous. I used to rip through dozens of business and self-help books a year – and while this made be an interesting dinner party companion (or terrible depending on your conversation tastes) – it rarely led to behavior change.

Now, I only read a business or self-help book if: (1) I have the time to read it deliberately enough to draw out behavior change possibilities; and (2) I have the time to practice and implement the behavior changes.

The Ability to Adopt New Habits is an Incredible Competitive Advantage

Most people will spend over forty years working.

Over the course of a career, being able to adopt a few important habits a year will provide an incredible competitive advantage over people who do not adopt important habits.

Growth mindset and intellectual curiosity are amazing mindsets – but it’s the ability to form habits that unleashes their true power.

Every Mentor Should Become Your Peer

1 Reply

I. Rigorous Thinking and Effectiveness

The most effective people that I’ve worked with, or engaged with online, are extremely rigorous thinkers.

This, in some sense, is satisfying.

To the extent that the connection between effectiveness and rigorous thinking is causation rather than correlation, then increasing the rigor of one’s thinking can help lead to greater effectiveness.

II. Implicit and Explicit Instruction

Of the extremely rigorous thinkers I’ve interacted with, only some of them are good at explaining why they think the way they do.

All of them, on the other hand, have been good at telling me when my thinking has not been rigorous enough.

Given the above, I’ve spent a lot of time trying to take their implicit and explicit instruction in order to improve myself.

One incredibly positive aspect of the internet is that it opens up thousands of avenues to learn how other people think.

III. Mentorship

Another way to say all this: the value of a mentor (either in person or online) is that she should significantly increase the rigor of your thinking.

When you communicate with her, you should run as many cycles of feedback as you can – pitching your ideas, getting her critiques, and learning.

But, at some point, this can lead to laziness: you might rely on your mentor to do the thinking for you rather than doing it yourself.

One way to test your potential laziness is to continually take stock of the delta between the rigor of your thinking and the rigor of your mentor’s thinking. If you are not closing the gap between the rigor of your thinking and your mentor’s thinking, then you are, in some sense, exploiting your mentor.

Ultimately, every mentor should become your peer. This should be the goal of all involved.

IV. My Journey in Unrigorous Thinking

For whatever it’s worth, my biggest obstacles to rigorous thinking have come from the following:

1. Yearning for Silver Bullets: Very few quick fixes exist for hard problems. In part, I think I was drawn to law school because of the idea that passing the right law could fix things quickly. As it happens, this is very rarely true. Hard problems tend to have decade long answers.

2. Yearning to Have My Problems Solved: Related but slightly different: when confronted with thorny day-to-day problems, I’ll sometimes gravitate toward the easiest or first solution rather than taking the time to rigorously analyze the issue. This is nothing more than intellectual laziness but it is difficult to control, especially when you’re moving fast and making a lot of decisions each day.

3. Ignoring Politics: I’ve also succumbed to spending a lot of time on ideas or programs that never had a chance of being political viable.

4. Ignoring Executing at Scale: Even ideas that might be politically viable may not be possible to scale operationally. Especially earlier in my career, I deeply underestimated execution as a limiting variable.

5. Wanting to be Agreeable: The desire to please or get along with others sometimes trumps my effort to push to get to the right answer.

Those are the big ones: I could have avoided many of the biggest mistakes of my career had I been more rigorous at avoiding these pitfalls.

I suspect I’m not the only person in my line of work to make these types of mistakes.

V. Thinking More Rigorously Hurts Until It Becomes a Habit

For all of the above issues, there has either been a professional failure or extremely direct piece of feedback that has provided a wakeup call that I need to be thinking more rigorously.

Even after knowing I have a problem with the way I think, many times the only way I’ve been able to make progress has been by being exposed to leaders who are adept at avoiding these pitfalls.

This often all rather painful.

For me, it literally hurts to think more rigorously in an area where I have not previously been a rigorous thinker.

In some instances it feels like being a child who knows his parents are right but doesn’t want to admit it even though it is in his best interest to (1) admit it and (2) incorporate his parent’s way of thinking into his world view.

Eventually, it gets less painful – and then it becomes a habit.

And then it’s on to the next one.

VI. Accountability is an Accelerant

One last point: putting yourself into situations with high accountability for outcomes is one of the best ways to increase the rigor of your thinking.

If you are not accountable for outcomes, then you will be tempted to avoid the pain that comes with thinking more rigorously.

If you are accountable for outcomes, then the pain of potential failure helps offset the pain of thinking more rigorously.

Meditating Away Education Reform

relinquishment

writing about handing power back to educators and families – and other thoughts