What is Amazon’s flywheel and why does Bezos so often ignore it?

I just finished reading The Everything Store by Brad Stone. It is an amazing book and I learned a ton. I highly recommend it.

But here is the question I kept coming back to: why does Bezos appear to so often get strategically distracted?

My presumption here is that I’m missing something and am wrong in thinking he gets distracted. He is the one running one of the most successful companies in the world.

But here’s my wondering:

Amazon seems most successful when it stays focused on building out world’s biggest online retail store and spinning out services that it had to create in order to scale (AWS, distribution, allowing 3rd party sellers on its platform).

This appears to be an amazing flywheel:

  • Capture the largest economies of scale retail has ever seen by being a global online everything store.
  • Anytime you create a technological or operational breakthrough, turn it into a platform available to third parties.
  • Repeat.

From afar, this seems like an amazing formula that Bezos has perfected with brilliant vision and aggressive execution.

So why doesn’t Bezos just stick to the formula?

Time and time again Bezos has deviated it from it and suffered major losses: including his dot.com spree of buying stakes in other companies, many of which eventually went bankrupt (investing distraction); the failure of Amazon Fire (hardware distraction); the failure of Amazon Wallet (veering into payments); the failure of Askville (search distraction). I’m sure there are more that I’m missing.

Note that in the above I’m not naming retail mistakes: experiments with online travel, fashion, auctions, and so forth were within in strategy and didn’t work out, but they weren’t really distractions. They were retail bets.

Kindle and Alexa are the most interesting counterpoints. With these devices, Amazon built hardware products that ended up being market leaders when the flywheel strategy dictates that they should have stayed out of the hardware game.

Amazon video another big bet that has not yet failed, but seems like it would have just been better for them to be the platform rather than be their own content channel.

Yet even if Kindle, Alexa, and Amazon video are long-term profitable, they (and all the distractions that failed) have to be weighed against the opportunity cost of time and resources that could have been plowed into the flywheel.

Amazon has so much retail growth ahead of them, why not just devote 100% of Amazon energy into being the everything store?


Is Sweden’s education system falling apart? If so, what is to blame: vouchers or progressivism?

Sweden adopted an expansive school voucher system in 1992.

Since then, Sweden’s PISA scores have tumbled.

Screen Shot 2018-12-19 at 9.58.39 AM.pngSweden’s PISA scores shot up a bit in 2015, which is good to see.

But it’s worth exploring why they dropped in the first place. There are two main theories why the scores dropped: some blame the voucher system and some blame Sweden’s embrace of progressive / constructivist educational methods (less teacher directed lecturing, more student inquiry and discovery, less focus on knowledge building).

Who’s to blame? Vouchers or constructivism?

Was it the Vouchers?

There’s been a decent amount of research on Sweden’s voucher program.

This paper found small positive to neutral effects on academic achievement and no effects on long-term outcomes. This paper found positive to neutral effects.

This paper found positive effects due to increased competition, though it took about ten years for these effects to kick-in. They also tackled the PISA issue head on:

Our positive estimates might appear surprising given Sweden’s relative decline in scores on international tests such as PISA since the mid 1990s… we do not find any support for the belief that an increase in the share of independent school students provides an explanation for Sweden’s relative decline.

It seems that vouchers aren’t the main culprit. They’ve only scaled to educating 20% of students, and most research finds neutral to positive effects.

Was it Constructivism?

Another set of researchers say that Sweden has embraced an extreme form of progressive / constructivism, “which holds that an objective reality does not exist and that the objects and phenomena themselves—and not just our perceptions and interpretations of these phenomena—are socially constructed.”

The authors note that this philosophy really began to take hold in the early 1990s, when large groups of older teachers retired and were replaced by younger teachers who had been trained in progressive-constructivist ideas.

This was accompanied by unwinding the existing detailed national curriculum, and, in 1994, replacing it with high level standards that were more constructivist in nature. These standards also had a deeper focus on student’s directing their own learning instead of teacher directed lessons.

Or was it both? 

These same authors said that move to constructivism was accelerated by the voucher program: because of fluffy curricular standards and a lack of accountability from the national government, the new private schools inflated grades, which drew students in, which then caused the public schools to respond by inflating their own grades.

Without national accountability, the researchers claim that it was race to the bottom.

Looking at Finland

Sweden is not the only country to drop on PISA of late. So has Finland, which previously was at the top of the PISA charts (and even after dropping still does very well).

Screen Shot 2018-12-19 at 1.33.24 PM.png

I’ve seen no rigorous research on why Finland dropped on PISA; some point to declining economic conditions, others to a reduced focus on core subjects like math. Others note that many of Sweden’s more constructivist / progressive reforms took place right when Sweden was at its peak, and now that they’ve been fully implemented scores have dropped.

No Excuses vs. Constructivist Charter Schools

In the United States, “No Excuses” charters schools, which tend to be more standards and teacher driven, generally outperform more constructivist / progressive school models when it comes to state tests.

I’d add this as another data point in favor of the idea that constructivist / progressive schools tend not to do as well on standardized tests.

My Hypothesis

I’m not an expert in any European education systems, so consider this a guess: I think that constructivist / progressive education models are the leading contender for why test scores are falling in Sweden and Finland.

This hypothesis fits within both anecdotal and research on progressive approaches not resulting in high test scores; see table below from meta analysis on effect teaching, which identified approaches that tend to raise / lower test scores:

Screen Shot 2019-01-02 at 7.39.23 AM.png

What Do Swedes Want? 

Given that Swedes embraced progressivism in both their traditional and voucher schools, perhaps they care less about PISA scores than they do other aspects of education.

It’s also interesting that Sweden’s PISA scores shot up a bit in 2015. So maybe the system is correcting toward balancing practices that produce good test scores and allow for student directed learning. Or perhaps the constructivist strategies do work and they just took some time to kick in. Either way, things don’t seem to be falling apart.

As vouches expand in Sweden, it will be interesting to watch what families desire as they have more information about which types of practices lead to which types of results.

Grasping in the Dark 

Unfortunately, both in Sweden and the United States, we have very little idea what types of education models lead to great long-term outcomes, so Swedish families (and the rest of us) will still be grasping in the dark for the foreseeable future.

City Fund updates

Check out this 74 Million interview with me and two other City Fund partners to get some updates on our work.

Kameelah Shaheed-Diallo and Kevin Shafer, who both joined our team over the summer, provide some great context on their past work in Indianapolis (Kameelah) and Camden (Kevin).

I really value how much earned wisdom our partners have. Kameelah and Kevin were incredible local leaders in their own communities, and it’s great that they can now support other leaders trying to make change.

In the interview, we also give updates on how much money we have raised and which cities we’ve made initial grants to.

We will continue to be fully transparent about where our money comes from and where it goes. We are a charitable institution with significant resources and we want our work to be understood and scrutinized.

We also want to be transparent because we’re big believers in this effort. We think that a few cities have made significant and sustained gains for children. And we’re excited to work on a philanthropic effort that helps see if these strategies can work in select group of other cities across the country.

If it works, a dozen or so cities will have shown that there might be a better way to do public education in cities. That will be amazing for children.

If it doesn’t work, we’ll understand that what worked in a few cities is simply not scalable.

The only way to find out is to do the work.

How much I gave to charity this year -> and to which cause -> and why my giving might be mistaken

Every year I write a post about how much I give to charity. I consider this an act of positive virtue signaling. If we’re going to compete on something, competing on how much we give to charity is the right kind of competition.

This year I’m slightly altering my reporting. Instead of reporting this year’s giving, I’m reporting a five year charitable giving percentage. I consider this a more honest reporting, as it smoothes out year to year fluctuations.

Over the past five years, I’ve given away 8.5% of my total five year pre-tax earnings.

How does this compare to your giving? I’d love to hear about how much you give and what you give to in the comments.

What I Give To: Expanding Bed Net Access to Reduce Malaria 

Most of my giving goes to the Against Malaria Foundation. They are recommended highly by Givewell.

I donate to AMF because there is good evidence that bed nets save lives and because my marginal contribution increases the number of people who have bed nets. Despite the massive success of bed nets, there is still an on-going need.

Researchers studied the decline of cases of malaria in Africa between 2000 and 2015. They found that the single most important contributor to the decline were insecticide-treated bed nets.

Bed nets were responsible for the aversion of 68% of the 663 million averted cases in Africa between 2000 and 2015. These are 451 million averted cases. Given that children make up 72% of malaria fatalities, this is a truly remarkable impact for families.

Screen Shot 2018-11-22 at 12.57.25 PM

Additionally, researchers estimate that the malaria “penalty” to GDP ranges from 0.41% of GDP in Ghana to 8.9% of GDP in Chad, all of which could be regained following elimination of malaria.

Not only does my gift potentially saves lives, it is also positively impacts economic productivity.

All together, Givewell estimates (very roughly), that every $4,000 spent on bed nets saves a life.

If this is true, and I keep up my giving, I will be able to save a lot of lives.

As a new father, I can barely comprehend what it would be like to lose our child. I hope my giving will over time help hundreds of families avoid the pain and suffering caused by one of life’s worst tragedies.

When I think about whether or not it’s worth it to give, I think about our daughter.

Two Reasons (Out of Many) I Might Be Wrong

It is hard to help other people. I’ve tried to minimize this risk by giving in an area with  lots of evidence, low operational complexity, and clear health benefits.

But the fact is bed nets are never going to get people out of extreme poverty.

The only way for people living in extreme poverty to get out of extreme poverty is through rapid economic growth. Bed nets will not cause rapid economic growth.

The problem is that I have no idea what will cause massive economic growth in Africa.

But here are somethings I have considered funding:

Economic Research: Lant Pritchett makes the case that economic research allows us to learn truths that help countries escape poverty; i.e., the research on the benefits of trade, property rights, and other liberal economic principles have led to many countries adopting these policies, which has led to massive increases in wealth. Perhaps the same could be said of the research on domestic industry subsidization that forces subsidized companies to export competitively (some say this is a key driver for the Asian tigers). I could fund this research in the United States, or work with others to set-up research programs in local universities. A few friends and I could probably cobble together enough money to fund a full-time professor at a prestigious African university to work on these issues.

Technological Innovation: Technological progress is a primary cause of wealth creation. People living in Africa have much longer lifespans today because of technological innovations invented elsewhere. While my giving alone probably isn’t enough to impact technological research or venture capital investing, I’ve wondered about trying to get a group of 50 people or so and invest alongside established funds that are dedicated to technological innovation in globally important areas, such as energy. It’s plausible that in the case of investing, I could even get my money back and do a lot of good.

How I Feel About Giving

For the most part, giving makes me feel good. It feels morally correct to reduce my consumption so I can save the lives of children living in poverty.

But it also stings a bit. If you put together all the various taxes I pay, my tax burden is somewhere between 40-50% (such is life in California!). When you add my charitable contributions to this, that’s nearly 60% of my income out the door.

I also sometimes worry about my family. I live a very comfortable life and don’t want for anything. But life is unpredictable and this could change. If I or a loved one were in a severe accident, it’s quite plausible that I could run through my savings in under a decade. Giving to charity now reduces my ability to withstand big shocks later. Ultimately, I view the ability to withstand big shocks as a privilege that shouldn’t trump my duty to help others now, but it’s still something I worry about.

So there it is.

I give away 8.5% of my pre-tax income and I allocate much of it to malaria reduction. I hope this helps others in need.



Rearranging how parents get information about schools may increase educational opportunity at no cost

A .2 standard deviation increase in academic performance is a pretty good sized effect.

In this study, the .2 effect equates to attending a school that is 5 percentile points higher in ranking in academic performance.

Most interventions that achieve a .2 effect cost money.

This intervention costs nothing.

Unified Enrollment Systems 

Over the past decade many cities have adopted unified enrollment systems. These systems allow families to go online and view information on all public schools in their city, and then submit their ranked preference of schools to the government. New Orleans, Indianapolis, Denver, Chicago, Newark, Camden, and New York all have some version of this system for at least some grades.

These systems are great in that they give parents more information, allow them to easily apply to schools online, and help policy makers get information on which schools are most in demand by parents.

I’ve previously written about how the user interfaces for these systems diverge greatly in in quality. Some feel like you’re using a great iphone app and some are barely better than opening PDF files.

How Does User Interface Affect School Selection?

In this study, researchers worked with a consumer testing company to recruit a group of parents to use a generic unified enrollment system to select a school.

They then broke the sample into groups and presented a different user interface to each group, with the aim of testing how presenting information would impact school selection.

Here’s what they found:

Screen Shot 2018-11-07 at 8.27.24 AM

You can see the biggest effect (.19 standard deviations) is in the row “default sort order” and the column “distance.”

The researchers created two default sort orders: in the first case, you put in your address and then you are shown the schools nearest to your address; in the second case, you are shown the highest performing schools available to you.

The researchers found that if you make academic performance the default sort order, parents ended up picking schools that were +5 percentile points higher ranked on academic performance.

Making academic performance the default order costs no money.


The study has some real limitations.

First, the stakes weren’t real. The parents weren’t actually selecting a school that they would send their child to. This probably meant they put less effort into the school selection. They also weren’t able to get information from other sources (like friends and family).

Second, having parents pick schools with higher academic performance ratings is only useful if those ratings accurately measure student learning. In cities that use value-added methods for school rankings, I’d feel more comfortable with this nudge. In cities that mostly use absolute test scores, I’d feel less comfortable.

Third, academic performance isn’t everything, and parents select schools for a variety of reasons. Ultimately, designers of the interface do have to make choices, so I’m ok with a bit of nudge toward academic performance, but I don’t think we should make this nudge at the expense of giving families a wholistic picture of schools.

All that being said, the study shows that small and easy to make changes in user interface may have an impact on how families select schools.

If I was a government official that managed a unified enrollment system, this study would lead me to experiment with similar interventions for my own city’s system.

At the very least, I’d want to make sure that my user interface decisions were deliberate and values based rather than ad-hoc and random.

Access and Supply

Lastly, great unified enrollment systems are about equalizing access to great public schools.

They do nothing to increase the number of great public schools.

Cities would also do well to do all they can to help their best public schools expand.

Should leaders give praise to teammates?

Managing people didn’t come naturally to me. I struggled early on. I eventually became competent at it. But it’s not something I’m exceptional at.

I now work in a partnership. We don’t use a traditional hierarchical management structure, but I still have management duties. And I try to do them well.

The Best Managers of People I Know 

I’m close friends with two exceptional managers of people. If the United States somehow created an accurate national ranking of managers, I’d be shocked if they were not in the top .1%.

So I listen to them when they talk about managing people. And when I’m struggling with something, I often reach out to them. Both have a knack of telling me truth I wasn’t willing to tell myself because it would cause pain.

These two people strongly disagree on the role of praise. I don’t know who I agree with.

Praise as a Signal 

One of my friends strongly believes that praise is one of the most effective ways to incentivize mission aligned behavior.

According to this friend, if a manager knows what behaviors will lead to team success (and they should), then the manager should praise those behaviors whenever she sees it. And she should attempt to build a culture where others do this to.

The psychology behind this is fairly simple: people want to be praised, so if you praise people for something, they will do more of it.

My friend believes praise is one of the best reenforcement mechanisms a manager can use – and that it should be used frequently, at least weekly.

Praise as Sugar

Another friend says praise is like sugar: it gives you a quick dopamine hit, then you want more of it, and the more you get the you need to get high again.

According to this friend, a manager should try to create a culture where people are constantly trying to figure out what it will take for the organization to succeed, and then they do these things because they deeply care about the organization and have an internal desire to be the best version of themselves.

This friend also believes that praise gets the incentives all wrong. Because a manager can easily be fooled, if team members are just seeking out praise, they may act in ways that gets praise from the manager but is not actually in the best interest of the organization.

The psychology here is simple: people should have a deep ownership of the organization’s mission and their own personal self-actualization, and external praise short circuits this ownership.

When Brilliant People Disagree

My two friends are both brilliant managers of people and they disagree on this issue. Whenever two brilliant people disagree, and they are experts on the issue, and they have every incentive to be right on the issue because their mission depends on it…. then my first thought is that it’s a really really hard problem, and my second thought is that the answer might be situation specific and they might both be right.

Operational Clarity vs. Operational Uncertainty

I sometimes wonder if praise is most useful in organization’s with a lot of operational certainty and least useful in organization’s with a lot of operational uncertainty.

In other words, the less the manager actually knows what should be done, the less useful praise is.

If, for example, you’re managing a team to create a strategy for a complicated product launch, you probably don’t know what the right answer is for everyone on your team. If you give a lot praise, you could very well praise bad ineffective actions.

If, on the other hand, you’re managing 1,000 people to do the exact same job, and this is the third year in a row these people have executed on this task, you probably know exactly what you want to see. So it’s probably pretty easy to praise a good performance.

I’m not sure that this distinction matters for using praise, but I think it might.

Should Leaders Give Praise?

I don’t really know where I land on this.

My guess is that I significantly praise each person I work with 3-5 times a year.

I think my model for this is:

(1) I do want people to know when I think they did something pretty amazing, both because I care deeply about them and I want to signal that those types of accomplishments will help us succeed for children; and

(2) I do kind of think praise is like sugar, so I don’t do it too much.

Perhaps I’m trying to have my cake and it eat it too. I’m not sure.

One last thought: I sometimes think people confuse praise with care. I think it’s very important for leaders and colleagues to care for one each other. Care is the foundation for trust and trust is the foundation for good conflict. Also, life is short, and it’s better lived by surrounding yourself with people who care about you and who you care about.

But praise isn’t the only way to show care. It’s probably not in the top five. I would rather work for someone who cares deeply but praises sparingly rather than someone who praises effusively but cares shallowly.

Public education 25 years from now

What might public education look like 25 years from now?

The future is hard to predict, but it can be fun to try. The below is a mix of hope, curiosity, thought experimentation, and wild speculation. I’m not confident any of it will happen; for some of it I’m not confident it should happen.

Early Childhood Education

Starting at the age on one, means tested vouchers are offered to every family to spend on childcare that meets a basic level of accreditation. All young children get access to nurturing care. And low-income parents who wish to work and continue building their careers don’t face childcare costs that eat up most of their income. The United States moves from an international laggard to getting close to international leaders, such as Sweden. This is expensive, but growing productivity gains have made us wealthier and Americans wisely decide to spend some of this wealth on young children.

Kindergarten to 10th Grade

Public schools morph to significantly change the power dynamics between the school district, educators, curricular providers, and families.

School district bureaucracies are paired back. A system of great schools replaces the traditional school system, with families having access to an array of autonomous district schools, charter schools, magnet schools, and private schools – all of which have much more operational control. The lines between these types of schools begins to blur, and the political fights between these types of schools subsides. Non-profit organizations run a lot more schools than today, but school districts still serve the majority of children.

A lot of entrepreneurship, a little bit of competition, and a lot of best practice sharing make all public schools better. It’s a great time to be a public school educator.

While schools get more autonomy, many schools begin to gravitate to the best learning platforms. Over a few decades, many incredible academic models are put on cloud-based learning platforms that transparently support students, teachers, and families through a rigorous curriculum and independent assessments. Some models, such as Montessori and IB have been around for awhile. Others, such as Achievement First Navigator, Khan Academy, and Summit Basecamp, are born from more recent entrepreneurship. All of these models combine both academic and social emotional development into one programmatic model. Most schools choose one learning platform but tailor it the kids they serve.

Many teachers specialize in one or more of the learning models that they are most passionate about. They gain deep content expertise in the model and grow expert at intellectually and emotional supporting students through the model’s progression. Teacher training in colleges is more grounded in supporting students in these models rather than being so heavily focused on theory. Many of the best learning platforms open their own teacher training programs for on-going educator development.

Families have a lot of great information about public schools as well as how their children are doing. Online enrollment systems provide transparent information on every public school in the area and allow for more equal enrollment access. 3rd party providers like Great Schools provide independent analysis. Because most schools are on one of the top learning platforms, families are better able to distinguish between the approaches of different public schools. The school down the road is no longer a black box. Families know what they are being promised. And the big data provided by the internal assessments of the learning platforms give families very transparent information about how their children are progressing.

In places with high rates of poverty, support services are ramped up. Physical and mental health services are provided at the school site. A mix of onsite and offsite programs serve students with severe physical or emotional needs. Social workers provide intensive services to families who are struggling the most. Data-driven Strive type programs coordinate an array of services and prevent kids from falling through the cracks.

The major increases in educator autonomy and family information do not cost much more than the current system. Overall public expenditures for K12 stay fairly flat in terms of real dollars. However, spending on support services for very at-risk students rises significantly.

Grades 11 and 12

The traditional public school system changes dramatically after grade 10. Once a student has shown mastery of secondary material, they can begin experimenting and specializing.

Grades 11 and 12 are funded with universal vouchers. Families are able to spend the vouchers on any accredited education institution, including a regular high school, a higher education institution, an apprenticeship program, working abroad, or being a subsidized employee at a great for-profit or non-profit organization.

Some kids continue at their high schools. Some kids go straight to college. Some kids begin trade school in data and analytics. Some kids begin trade school in nursing programs. Some kids join together to start local businesses. Some kids move to San Francisco and join start-ups. Some join writing camps where they write their first novels. Most kids do a few of these things. It’s a time of exploration.

Once You Turn 18 

Every adult gets access to a low interest loan for up to 6 years of schooling or subsidized employment through their adult life. Very few people do this all at one time.

The subsidized loans can only be used at institutions that agree to repay the government for 50% of unpaid loans. If you educate a student who can’t afford to pay back the government, you share the losses. This reduces the number of students who go to four year colleges (as many of these students are too risky to take on) and increases the number of programs that provide sound job training to students who would have likely attended but dropped out of four year universities. Many of these programs are income contingent pay-back models where a student only has to pay back their loans if they earn a certain salary.

More students than ever make the jump to a meaningful career. Higher education expenditures fall as fewer dollars are wasted on programs that provide little value to many of their students.

In Sum

Early childhood is vastly expanded. All kids get access to a nurturing early childhood environment.

Empowered educators run schools serving grades K-10. A public system of schools is operationally decentralized, more programmatically centralized through great third party content providers, and greatly expanded in scope of services provided to at-risk kids. Families have much more information about how schools, and their own children, are doing.

Grades 11 and 12 allow for young adults to experiment across a variety of learning experiences.

Post high-school education is more accountable, specialized, and on-going. Kids are not left with loans that they can’t pay back. Institutions survive by adapting their offerings to what different kids need to cross the bridge into secure, meaningful adulthood.


Should states use test score based accountability systems? If so, how? If not, why?

Over the past decade, I’ve deepened my belief in the power of letting educators form non-profits to run public schools. Both experience (walking into amazing public schools) and research (a track record of reading and math gains) have shown me that non-profits are an incredibly valuable tool in making public education better.

I’ve also deepened my belief in unified enrollment systems. They can give families a lot of information about public schools and make enrolling in public schools much easier.

I do not have deep confidence in my views on accountability. I often find myself moving up and down the spectrum of: no accountability (just let parents choose), to accountability-lite (require testing, share this information, but don’t intervene), to accountability heavy (require testing, give schools letter grades, intervene in lowest performing schools).

I think reasonable arguments can be made for all three approaches.

Recent NWEA Research

NWEA just published a new report using a national data set from the tests they license to schools. Many schools we work with use these tests. I’m not expert enough in statistics to evaluate the reliability of their findings, but the report raised some important issues.

Absolute test scores are highly correlated with poverty. The chart below shows that test scores rise as income increases. This is not new information.

Screen Shot 2018-10-04 at 7.17.05 AM

Student growth is not tightly correlated with poverty. Unlike absolute achievement, individual student growth does not rise significantly with income. Many high poverty achieve growth that mirrors those of their wealthier peers.

Screen Shot 2018-10-04 at 7.19.26 AM.png

Schools with similar levels of poverty perform very differently on growth. The red line in the chart below represents how schools with high poverty perform on academic growth. It is a fairly wide curve. Many schools achieve low growth, while others achieve very high growth. To the extent you believe that growth is a pretty good measure of school performance (the researchers do), this performance spread might increase a policymaker’s willingness to intervene in low-performing schools and expand high-performing schools.

Screen Shot 2018-10-04 at 7.23.15 AM.png

Focusing on absolute test scores will cause you to misidentify many, many schools. The graph below is tricky to read, but it’s very important. The red line represents all schools that are in the bottom 5% for absolute test scores. And it shows that 77% of these schools (the bottom 5% on absolute) are close to the average or better on growth. In other words, if you just closed the bottom 5% of schools based on absolute achievement, nearly 80% of the schools you’d close probably would be mistakenly closed (given their growth scores). This is pretty damning evidence against those who want to focus mostly on absolute achievement in accountability measures.

Screen Shot 2018-10-04 at 7.27.08 AM

When Does a Good Policy Idea Become Indefensible Because of Bad Practice?

Over the past few years, most states reworked their accountability systems during the reauthorization of No Child Left Behind.

Unfortunately, this report found that only 18 states weighted growth for at least 50% of the total accountability score, with another 23 states weighting growth at least at 33%.

On one hand, this is an improvement over old accountability systems. On the other hand, this means a lot of states are unfairly rating high poverty schools that have decent growth but low absolute scores.

I think a fair critique of test based accountability is that it’s a reasonable idea that has very little hope of being reasonably implemented.

My Own Thoughts

Again, I do believe deeply in letting non-profit organizations operate public schools. And I do believe deeply in enrollment systems that make it easier for families to find a great school for their children.

I’m uncertain about accountability, but here’s what I think I’d do if I were superintendent of a school district:

  1. Calculate a letter grade score for growth and a letter grade score for absolute achievement score.
  2. Publish the higher of these grades as the letter grade that appears most prominently on the online enrollment system. I would also include the lower letter grade, as well as a bunch of information about school programs and curriculum, on the school’s online profile.
  3. Allow for government intervention in schools that are in the bottom 5-10% for both growth and absolute (you need to perform bad on both).

This type of accountability system gives parent’s good information, avoids the political war of giving low letter grades to schools with high absolute scores, and avoids the error of intervening in schools that have low absolute scores and higher growth scores.

It does give an accountability pass to schools with high absolute scores and low growth, but I view this ok in that it’s both politically useful and it does reflect the notion that parents really want to get into these schools.

It also still uses test scores as the primary way to evaluate schools. This sits uneasy with me, as I think schooling is about much more than tests, but I haven’t seen any other way to measure schools that feels more reliable. I hope this changes.

I’m not very confident that this is the best system, but I think it’s the best of a bunch of options that all have reasonable drawbacks.

Another hard question would be what to do if local politics did not allow for the creation of a system like this. At some point, if the drum beat for absolute scores was too much, I’d probably walk away from accountability as a superintendent.

But I’m not sure. If you scan this blog’s history, I’m sure you can find me saying conflicting things about accountability. I’m conflicted about it. But the above reflects my current thinking of what makes for a good accountability system.

Lastly, if you want to hear a good version of the argument against test based accountability, see here.

Does research matter?

A common critique of public charter schools is that they hurt traditional schools.

As of 2016, this questions had been studied by researchers in 16 regions. In 15 regions, they found that public charter school growth had positive or neutral effects on student learning in traditional schools. The table below summarizes the results of these studies.

Screen Shot 2018-09-25 at 7.55.15 PM

In 2016, a ballot initiative in Massachusetts called for “lifting the cap” on charter schools. In many cities in Massachusetts, it is illegal  for public non-profit charter schools to serve more students, even if parents want to send their children to these schools.

This is despite the fact that Boston is home to some of the best charter schools in the nation. You can check out this NYT article to learn more.

By a large margin, the ballot measure was rejected by voters in Massachusetts. Prominent political leaders, such as Elizabeth Warren and Bernie Sanders, came out against it.

A post-ballot analysis showed that the measure lost because many people believed that expanding public charter schools would hurt traditional public schools.

Yet, as I noted at the beginning of the post, in 2016 we had a good amount of evidence that this wasn’t likely to be true.

Research did not matter to the Massachusetts outcome.

Now a new study has come out looking at this exact issue in Massachusetts. And the researchers found that between 2011 and 2015, charter school expansion did not negatively impact traditional schools in Massachusetts (and may have had a small positive effect).

Voters in Massachusetts hold incorrect beliefs.

So now there have been studies in 17 regions, and in 16 of these regions public charter growth has had either positive or neutral effects on traditional schools.

Will votes in Massachusetts, and elsewhere, change these beliefs?

A cynic might say no. She might argue that voters don’t vote based on research, they vote based on emotion and tribal affiliation.

There is much truth to this argument.

But I have been influenced a lot by another argument. Lant Pritchett, in this podcast, argues that while a single research study rarely changes the world, hundreds of them often do.

In other words, if a bunch of studies pile up, and they all roughly show the same thing, at some point it becomes hard to argue against them. Sooner or later, you find yourself in the anti-vaccine tribe.

I think there is a lot of merit to this argument.

So while those of that support great public charter schools should develop communication strategies that pull at the heartstrings and speak to the tribe, I also think we should keep building our evidence base.

I think research does matter.

At some point, if the research studies keep piling up, they will increase the probability that the voters of Massachusetts, and elsewhere, change their minds.

Group check-ins, flashcards, and other experiments in our virtual organization

The City Fund is a virtual organization made up of mostly senior leaders.

Some standard organizational practices don’t seem best suited for our situation. So we are beginning to experiment.

I think that reinventing organizational management is a fools endeavor, but modifying at the margins can lead to better performance.

Here are some things we are trying:

Alternating 1-1 and group check-ins: We are no longer conducting 1-1 weekly check-ins with managers. Instead, every other week will be a group check-in with a 3 person pod. We think this will increase the effectiveness of problem solving, reduce knowledge silos, increase pattern recognition, and increase team member investment in other people’s work.

Once a month fly-ins: We are finding that quarterly retreats are too infrequent to tackle pressing big issues. So now once a month we all fly into a city, have dinner, and then spend the next day tackling our toughest problems together. We then fly out in the afternoon. We hope that this will allow us to solve big problems quicker as well as increase team bonding.

Monday optional discussions: We are finding that our 2 hour video team call on Friday is not great for informal brainstorming and debating. The agendas tend to be tight and based on making decisions. So we have now have a one hour optional video call on Monday. The video calls never have more than 1-2 topics and are meant for more open discussion.

Slack summaries: A lot of our work happens when an individual team member is a visiting a city. This makes it a bit difficult to learn together. So now we use slack to record visit observations, allowing other team members to chime in with advice. Simply reading these reports also increases building pattern recognition.

Flashcards: This quarter we are piloting flashcards. Each flashcard deck will be on a topic that everyone on our team should be smart on; i.e., the most important educational research on our strategies. During the pilot, we’ll ask each team member to spend ~10 minutes a day working through some flashcards.

We’re not sure what will work and what won’t. We’ll try things for a quarter and stop whatever is not working.

A common theme of these practices is that we’re trying to get smarter as quickly as possible. We hope that this will help us be more effective in making public education better.

When you’re building for the long-haul, quick marginal improvements matter a lot.

Over the years, they can compound into major improvements in effectiveness.