Category Archives: Uncategorized

Senior signing day: from a bar in Houston to the White House

Chip and Dan Heath are brothers and co-authors. Chip teaches at the Stanford Graduate School of Business. Dan teaches at Duke.

The most recent book is called The Power of Moments.

The book opens with a story. The story begins with Chris Barbic (a friend and a colleague) and Donald Kamentz sitting in a bar after another long day in the founding of YES Prep, a charter school network serving low income students in Houston. They were watching ESPN’s coverage of “senior signing day” – the event where star high school athletes announce where they’ll be attending college.

Chris and Donald believed what their students were accomplishing was every bit as impressive as what these star college athletes had accomplished.

So a few months later (charter schools founder work quickly), Yes Prep held it’s first senior signing day. Each senior took the stage and announced where he or she would attend college in the fall, dropping a t-shirt or pennant with their chosen school’s mascot. Leading up to the day, students kept their final school decision a secret from friends. After each announcement, the room erupted with applause. Everyone cried, and a tradition was born.

You can watch a YES Prep senior signing day highlight reel here:

Soon other schools charter school began to hear about YES Prep’s amazing event.  If YES Prep could create an amazing moment to celebrate their students, why couldn’t other schools do the same?

So steal they did.

Here’s Achievement First’s signing day (grab a tissue).

Here’s IDEA’s signing day video (grab some more tissues):

Then, in New Orleans, Josh McCarty, a friend and colleague of mine at New Schools for New Orleans, stole the idea for the whole city, and, I believe, we had the country’s first citywide signing day.

But we were soon out scooped, with senior signing day going to another (national) level: Michelle Obama made college signing day a signature part of her Reach Higher campaign. My guess is that Arne Duncan, the Secretary of Education under Obama, spread the idea, as he had spoken at YES’ signing day in 2010. In the video below, Michele Obama asks high school seniors to take selfies with their new college gear and tweet it with the hashtag #reachhigher.

Here’s a picture of her and her husband celebrating signing day together:

Screen Shot 2018-06-08 at 5.36.44 PM.png

Charter schools have been an incredible source of innovation for public schools. From Doug Lemov’s Teach Like a Champion to Summit’s personalized learning platform to Achievement First’s open curriculum to Valor’s social learning program – great public charter schools are making all public schools better.

The senior signing day story, which begins in a bar in Houston and goes all the way to the White House, is an amazing example of what happens when you let great educators open new public schools.

Every once in a while, a school opening is the origin of a new national tradition.

So please: go to a senior signing day. And while you’re celebrating incredible students, give a silent thanks to Chris and Donald.

 

 

Response to Matt Ladner and Jay Greene

Over at Jay’s blog, Jay and Matt wrote two critical posts of portfolio management and harbormasters (our team calls them quarterbacks).

Jay and Matt think differently than I do, have different political orientations, and are sharp. Their writing makes me smarter.

I also find their post titles, writing tone, and evidence analysis to be a bit over the top. They sometimes overstate their claims and under appreciate the other side of the argument.

In these two posts, Jay and Matt use NAEP charter sector gains in Arizona, Michigan, and Texas – as well as the mediocre NAEP scores seen in Louisiana’s charter sector – to argue that portfolio management and quarterbacks aren’t working.

I found their analysis to be overly narrow. Instead of taking some new evidence in and synthesizing this with the broad set of evidence available, they anchored on to one set of data points and made too strong of claims (especially in the titles of their posts).

Don’t Look at NAEP in Isolation 

Matt is right to point out that some states with fairly loose charter regulations saw a lot of charter gains in NAEP between 2009 and 2017.

I think this should modestly increase our belief that being loose on charter openings and closings can lead, over time, to a healthy charter sector.

But the story is not that clean.

This CREDO paper, which looked at charter school performance in Texas between 2011 and 2015, found a small positive effect in reading and no effect in math. Given that CREDO tracks individual students across time, and NAEP does not, the CREDO data should make us cautious in interpreting the NAEP gains as a huge victory.

Screen Shot 2018-04-24 at 9.57.03 PM

Another study found that it took Texas charters ten years to achieve the performance gains of their traditional school peers.

While it’s great to see the sector improving in Texas, perhaps with better regulation the sector wouldn’t have had to improve for ten years just to achieve neutral effects.

The story of the Texas charter sector is much more complicated than Matt’s piece indicated. The same is true for Arizona and Florida, where quasi-experimental research has found muted test score effects.

Given the different results of NAEP and CREDO, we should be trying to figure out the puzzle rather than claiming victory, as Matt tends to do.

Don’t Look at States in Isolation 

Looking at state level gains is not a great measure of whether portfolio and quarterbacks are working, as portfolio (to some extent) and quarterbacks (almost always) are city based endeavors.

For city based data, this CREDO study measured student learning effects in a bunch of cities across the country.

Washington D.C., Denver, New Orleans, and Newark all did very well. These cities are also some of the most mature portfolio cities.

Phoenix, Austin, El Paso, Forth Worth, Mesa, and San Antonio did not do well. These cities are all found in loose regulation states.

It hardly seems like a slam dunk to me that portfolio / quarterbacks are bad and loose regulation is good.

Of course, the CREDO analysis is not perfect: test scores aren’t everything and the virtual twin methodology may miss unobserved differences between students.

But looking at state based NAEP scores to make broad judgements on portfolio and quarterbacks is unwise, especially with so much other evidence available.

The portfolio / quarterback model seems to be doing some good in many cities.

Quarterbacks are a Step in the Right Direction  

Jay and Matt often criticize quarterbacks as vehicles for people who think they are smarter than everyone else (especially educators and families). I find this to be an overly simple critique.

Quarterback originated as a way to use expertise to aggregate and allocate philanthropy.

In many cities, philanthropists were funding low impact activities, often wasting it on the  pet projects of district leadership. A lot of money was spent for very little academic gain.

Quarterbacks have helped improve philanthropy: instead of just passively giving money to the district, philanthropists partner with expert management teams to try and launch and grow great non-profits.

I think this is a major improvement on the status quo. Of course, there are some drawbacks, and too much centralization of philanthropic capital poses risks. This is why I don’t think all of a city’s philanthropic capital should flow through one organization.

But quarterbacks are increasing, not decreasing, educator entrepreneurship and family choice. Yes, they do often use test score results selecting who to fund, but I suspect this will change if a better way to invest is developed over time.

In Sum

The NAEP data should not be ignored. It’s made me more open to the idea that looser regulations can lead to charter scale and quality, especially at the state level. And I found Matt’s data analysis to be quite helpful. I love it when smart people who think differently than me play with complex data sets and come to novel conclusions.

But I think there’s plenty of other state based evidence that should make us cautious, such as the CREDO Texas study.

I also think there’s a lot of evidence that the charter sectors in portfolio / quarterback cities are making a lot of gains. The NAEP data Matt and Jay site, which is state based and does not track individual students, is not convincing enough to make me deeply question our city based work.

All that being said, I look forward to reading more from Jay and Matt in the future.

I don’t take smart, critical friends for granted.

The Case Against My Own Education

Bryan Caplan just released a new book: The Case Against Education: Why the Education System is a Waste of Time and Money.

Instead of a doing a regular review of Bryan’s book, I thought I’d do a little introspection. Bryan’s argument is that education is a major waste of time and money.

Does this hold true for parts of my own education? If so, which parts?

Pre-K: Not wasteful!

My formal education started at a Montessori pre-k. It’s a little difficult to use introspection to determine whether this was a waste of time and money, as I don’t remember much about pre-k. I do have a vague memory of being confused most of the time. I didn’t understand what I was supposed to do there. But perhaps this is the point of Montessori. I don’t know.

But I don’t view this as a waste of time (what else was I supposed to do at the age of 3?) or a waste of money (the pre-k was not that fancy so I assume it was priced just a bit above the cost of babysitting). So seems like a decent use of mine and my parent’s resources. It allowed me to be confused in a safe environment and it allowed my parents to work.

Elementary School: Not wasteful!

At Parkview Elementary, I learned to read and write and do math, which have all been very useful in my life. Me being at school also allowed my parents to work, which provided our family with a home, food, and the comforts of a middle class lifestyle, which made for a happy childhood. If I had not been at school, I can’t really think of many productive uses of my time, so I don’t see many trade-offs in having attended Parkview Elementary. The combination of the school teaching me the basics and providing cost-effective babysitting (Indiana is not an extravagant spender on elementary schools) seem well worth the time and money.

Middle School: Not wasteful! 

At Ben Franklin Middle School, I honed my basic writing and math skills, as well as picked up some basic science and social studies, which probably helped ground me in the modern / liberal world order (science, democracy, etc.). I also was put in an orderly environment which helped prepare me for a society that values conscientiousness, agreeableness, and the maintenance of civilized social coalitions. If I had not been a school, I suppose I could have worked in nearby farms (labor laws permitting), which would have also reinforced conscientiousness, but probably have been lacking in math, writing, and more advanced form of social coalition building. I don’t think I was prepared to work at the types of firms that would have developed my professional skills, nor do I think that most firms would have found it cost effective to teach me math and writing, which would have been hard to teach myself.

Early High School: Not sure 

9th and 10th grade at Valparaiso High School were also good educational  years: I learned Algebra (which I still use) and further practiced writing, with an additional emphasis on research (which I still use).

However, at this age there were some real trade-offs in going to school. By the age of 14, I could have started contributing to companies at a rate that would have been worth paying me a minimum wage (if not more!) for roles that would have both helped the company and helped me build a lot skills. This is probably true at free market rates, and definitely true if the government took some of the money they were spending on me in education and used it subsidize employers paying my wages.

On average, I think you learn more about how to succeed in skilled jobs rather than in school, so I imagine I would have picked up a lot of useful soft and hard skills (goal setting, data analysis, project management, giving and receiving feedback, etc.) that I didn’t really pick up at school. And while I doubt most employers would have taught me Algebra, I imagine I could have taught myself in the future if the job required it.

So this feels like a toss up: I was learning things in schools that have helped me, but I also could have learned a lot by working at interesting jobs.

Late High School: Waste of time and money!

Most subjects I learned in high school (advanced math, science, literature, etc.) have been of very little use to me in life. Of course, I didn’t know what I would end up doing for a career at the time, but taking a bunch of advanced coursework seems like a pretty inefficient way to keep doors open for a wide breadth of future careers. For the most part, given my strong foundation in reading and math, I could have learned many subjects down the road if my chosen career had required it.

Probably 90% of what I learned in late high school I’ve forgotten and don’t really use.

I do think going straight to the work force would have been a much better education than school, but I worry a bit about making career decisions at such a young age. But a bunch of 3-12 months internships / travel experiences / short-term jobs likely would have been much better than learning Calculus, both terms of intellectual and social development.

Had I been working, I would have become a better person (in all senses of the word) faster.

College: Complete waste of money!

I was an English major at Tulane. I learned very little. Writing papers about novels is not a very transferable skill; the courses weren’t that rigorous; and most of the good novels I read I probably would have read eventually throughout my lifetime. I would have learned so much more (and been happier) had I been working at a few great companies over this time.

People also always argue that college is a time for intellectual exploration, but I don’t buy that. Life is a time for intellectual exploration, and you either enjoy being curious or you don’t. Even if I had been working, I would have still read a ton and had a bunch of great conversations, which probably would have allowed me to explore more topics at deeper levels than I did at Tulane.

1st Year of Law School: Not wasteful!

The first year at Yale Law School is basically a one year bootcamp in a mental model (how lawyers think) and logic (outline the arguments of legal cases). Even though I don’t practice law, both of these things have been helpful to me. I’m a big believer that mastering professional mindsets (lawyer, entrepreneur, teacher, VC, etc.) helps you solve a diverse set of problems as you move up in your career, and I do think that logically ordering arguments is a generalizable skill in the modern day workforce.

I sometimes wonder if schooling from ages 16 to 20 should alternate between internships and 3-6 months of curriculum from a variety of graduate degrees that provide useful mental frameworks. This would also be a great way to meet a lot of interesting people.

2nd and 3rd Years of Law School: Wasteful

I just got deeper and deeper into a knowledge base that I never use.

In Sum

My personal experience has been that school was really valuable until about 10th grade, and then, save for the first year of law school, was pretty wasteful relative to what I could have learned in a bunch of internships and jobs.

Of course, what is true for me might not be true for others.

One last point: from a policy perspective, I do think that grades K-10 are very important for both individuals and society, and I’m grateful to be working at a job that is trying to make that experience more pleasant and productive for millions of children.

Did a federal grant to turnaround failing schools in New Orleans and Tennessee work?

Back when I worked at New Schools for New Orleans, we applied for a $30m federal grant to turnaround failing schools in New Orleans and scale the model to Tennessee.

CREDO just came out with a research study on our efforts. Their findings, and my analysis, are below.

The New Schools Were Much Better than the Ones They Replaced 

Here’s what CREDO found when they compared the schools we created to the schools we replaced:

In New Orleans, we replaced schools (“closing schools”) that were at 26th percentile in the state with new schools (“CRM schools”) that performed at the ~33rd percentile in the state at the end of the study.

In Tennessee, schools went from the ~17th percentile to the ~23rd percentile by the end of the study.

To quote the CREDO report: “the CRM schools in both New Orleans and Tennessee showed significantly higher academic growth compared to the Closing schools they replaced.”

Translated into days of learning, these are large effects: “Closing school students experience 63 fewer days of learning in reading and 86 fewer days of learning in math when compared to students in non-CRM schools… students in CRM schools make comparable academic growth to non-CRM students.”

The New Schools Performed About the Same as Other Schools in the City

When CREDO compared the new schools to other existing schools (rather than the failing schools they replaced), they found no statistically significant effects:

Screen Shot 2018-02-15 at 12.07.57 PM.png

In other words, the new schools that replaced the failing schools performed no better or worse than other existing schools in the city.

On one hand, this is disappointing. Our most ambitious targets included having the new schools be amongst the highest performing schools in the city.

On the other hand, this is still a major improvement: the new schools replaced failing schools and ended up achieving at the same level of most other schools in the city.

Building a System that Keeps Getting Better 

Replacing failing schools with new schools is a process, not a one-time intervention.

Ideally, a subset of the schools you created will do really well, and then, overtime, these schools will continue to grow. The ones that don’t do well will not be supported to do additional turnarounds.

Over the long-haul, gradually increasing the number and scale of high-quality school operators is more important than the average effect of the first wave of replacements.

Here’s what CREDO found across the new schools when they compared them to existing schools:

Screen Shot 2018-02-15 at 12.11.36 PM.pngIn New Orleans, 50% of the new schools had positive effects in both Math and Reading. This is really positive: half of our turnaround schools in New Orleans achieved significantly better results than existing schools across the city.

In Tennessee, only one school had positive effects in both Math and Reading, though a few other schools had positive effects in only reading.

This makes me optimistic that the school operator base in New Orleans will continue to have the capacity to replace more failing schools over time.

The early results in Tennessee are a bit more worrying on the operator quality front, and the next few years will be extremely important in ensuring that a healthy operator base emerges.

Lastly: CREDO found that replacing failing schools with fresh start schools (that opened one grade at a time) had a higher success rate than whole school turnarounds. My takeaway here is that you need a mature operator base to do a lot of whole school turnarounds, and no city had enough capacity to really do whole school at scale. In hindsight, we should have done more fresh starts and less whole school turnarounds.

Was the Effort a Success?

At the outset of the project, I remember debating with our research partners at CREDO about how to set-up the evaluation.

I argued that we should ultimately be judged on whether or not the new schools we created were better than the failing schools we replaced.

I didn’t think we should be primarily judged on whether or not the new schools were better than other existing schools that weren’t failing.

Yes, we did include language in the grant application that had goals of schools performing much better than existing schools. And as we executed the project we tried to pick school operators that we thought could deliver top tier results. Our highest aspirations weren’t met. This is disappointing, but it does not mean the project was a failure.

Rather, I consider the project to be a positive step forward in improving public education in these cities.

Making Things Better

The result of the project strikes at the heart of what’s so difficult about education reform: our aspirations for our most at-risk children are incredibly high, but making progress in creating better educational opportunities is very difficult.

In roughly a five year period, we replaced failing schools with new schools that were on average 7 percentile points higher in state performance, which translates to an extra 60-90 days of learning per year.

If the process of opening and replacement continues, what is a modest success right now may eventually become a great success.

I hope that this occurs and that New Orleans continues on its impressive track record of increasing student achievement. As a reminder, the federal grant was just one piece of an overall effort that has radically reduced failing schools in the city:

Screen Shot 2018-02-20 at 5.47.30 PM

SaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSaveSave

To renew or to relinquish? That is the question for New York City

DB

Sometimes the research gods shine down upon us. In this case, they gave us two NYC studies, each on a different strategy, within a few months of each other.

The studies tell us much about what’s happening in New York City. They also illuminate the rational for why I believe in relinquishment.

What to do with struggling schools?

Some people believe you should give struggling schools more support. Others believe you  should either replace them with charter schools or close them.

Bill de Blasio is spending $582 million on 78 Renewal Schools, a model built upon the support strategy.

Charter school supporters tend to support the replacement strategy.

Early evidence on Renewal Schools

Marcus Winters of the Manhattan Institute authored a recent study that found that Renewal Schools have seen around a .1 standard deviation increase in math and ELA scores. This is a healthy bump and it’s great to see modestly positive effects.

Winters also notes, however, that Mike Bloomberg’s closure strategy achieved about the same results at little to no cost.

Recent CREDO study on NYC charter schools

CREDO’s research found that charters in NYC achieved about .1 standard deviation effects as well.

And while CREDO did not put a price tag on the charter reforms, most estimates I’ve seen have charters coming in at a lower per school cost than the ~$8 million per Renewal School.

Is it worth it?

It can be painful to close schools and grow charter schools. This process undoubtedly causes some disruption to families and educators.

If I believed the Renewal Schools would continue to improve, I’d be open to the idea that they were worth it. Perhaps spending a few hundred million more dollars is worth the cost to avoid significant disruption to communities.

The Renewal Schools will not keep improving

Sometimes these blog posts write themselves.

Elizabeth Harris ended her NY Times piece on Renewal Schools with the following quotation:

“The thing we’re nervous about is losing any of these resources,” said Mr. Bradley of Renaissance School of the Arts. “Renewal is the bomb. I want to be renewed forever.”

I’m sure receiving $5 to $10 million to improve your school is the bomb.

But of course these resources will not last.

Additionally, because these schools are subject to the whims of changes in district policies, the strategies they implemented will likely fall out of the favor of a future administration.

Charter schools, on the other hand, receive most of the philanthropic support for start-up costs, not continuing costs.

Additionally, charter schools are governed by non-profit boards which can maintain strategic consistency through self-perpetuating boards.

To renew or to relinquish? In the long-run, relinquishment will deliver sustained gains for students. Renewal will not.

Philanthropy is a low accountability sector (and what to do about it)

In terms of accountability to stakeholders for continued survival, my ranking of sectors is as follows (from most accountable to least accountable):

  1. Politics: the people you serve vote on your fitness every 2-4 years.
  2. For-profit: the people you serve must give you their money.
  3. Non-profit: the people who care about your issue must give you their money.
  4. Philanthropy: people must be willing to accept your money.

Depending on the specific office / company / non-profit, these rankings might shift a bit, but at the median I think they hold true.

If you disagree, think about this question: which sector is the most ruthless? This is likely where this is the greatest accountability.

___

At the structural level, philanthropy is an extremely unaccountable sector. All you have to do to stay in business is have someone be willing to accept your money!

This is not the fault of anyone involved, it’s just the nature of the beast.

I think a few things follow from this structural condition:

Culture matters: Philanthropists, like all humans, care about what people (especially their peers), think of them. Given that direct accountability is not as strong of a lever in philanthropy, culture pressures will play a bigger role. I think the rise in status of evidence based giving has been a positive development on this front.

Governance matters: Certain organizational structures, such as foundations whose governance outlive their founders by 100+ years, should probably limited, as this further decreases accountability in an already low accountability sector.

Rigorous exits matter: In most cases, philanthropy can’t cover the permanent costs of its subsidies, and over time government will generally pick up the tab of continued social programs. As such, it’s government that controls when philanthropy can exit, and the more rigorous government is in its spending, the more accountable philanthropy will be for producing results.

___

Again, I don’t think philanthropy is to blame for operating in it a low accountability sector.

It is what it is.

But citizens and governments can increase accountability, and they should do so when feasible.

 

 

Personalized learning is a transformative idea without a transformative technology

Screen Shot 2017-07-11 at 4.59.53 PM.png

I just got back from vacation, which was a great time to read the The Three Body Problem science fiction trilogy, a wonderful series that revolves around the protoganist using first principles thinking to negotiate with an alien species.

Upon return, I read this Rand report on personalized learning, which was funded by the Gates foundation. The report covers a small set of schools in the early years of implementation, so best not to draw too firm of conclusions.

The report found:

  • Charters that adopted personalized learning strategies saw a +.1 effect in math and no statistically significant in reading.
  • District schools (very small N) saw no achievement gains.
  • Charter schools implemented personalized learning strategies with more operational fidelity.

Perhaps most interestingly, the authors noted:

In this theoretical conception, schools that are high implementers of PL [personalized learning]  approaches would look very different from more traditional schools. In practice, although there were some differences between the NGLC schools and the national sample, we found that schools in our study were implementing PL approaches to a varying degree, with none of the schools looking as radically different from traditional schools as theory might predict.

So in this sample, charters outperform traditional schools (thought by a lesser margin than urban charters as a whole outperform traditional schools); charters execute better; and the schools themselves don’t look radically different than traditional schools.

Hence the title of this post: personalized learning is a transformative idea without a transformative technology.

Without a technological breakthrough, the current personalized learning efforts will, at best, lead to modest improvements on the execution of common place ideas (using data to drive instruction, executing leveled small group instruction, investing children in goals, etc.). School will look the same and be a little more effective and pleasant for all involved.

This is fine and the world is in many ways built on modest improvements.

But for personalized learning to live up to its hype (as well as to its philanthropic investment), it will need a technological breakthrough.

Instructional platforms might be the first breakthrough, but even here I think the primary effects will be more around scaling great school models and content rather than deep personalization.

The crux of the issue is this: computers are simply not as good as humans in coaching students through instructional problems.

Your average person off the street remains a more effective grade school tutor than the most powerful computer in the world.

Until this changes, personalized learning will never realize its promise. The problem is one of technology, not practice.