Category Archives: Decision Making

The Current Brutal Reality of Education Reform and Wage Growth

edu wealth

Going from 16K to 18K in Annual Wages 

Last week, I did a post on Fryer and Debbie’s excellent new study on the Texas charter sector.

I emailed the authors about my hypothesis that the growth of high-quality charters – even if they aren’t that much better than average traditional schools – could still be of great value if these new charters displaced chronically failing schools.

Roland was kind enough to respond but pushed that even if my hypothesis is true, the story still might be a depressing one.

His point: the numbers from his study indicate that even if we replaced all these failing schools with high-performing charters, we’re still only talking about ~1-2K in extra earnings per year for these students.

Given that many of these students end up lower income brackets, this might mean going from 16K to 18K a year in annual salary. Hardly game changing in terms of life outcomes – and surely not a ticket to the middle class.

Confronting this Potential Reality

When a study tells you what you don’t want to hear, the first reaction is often to not deal with it (in some ways I did this in my previous post).

So everyone in education reform needs to deal with this potential reality: there is some possibility that the best that education reform has to offer can only, on average, move a student from 16K to 18K a year.

Of course, this is only one study of one state. We don’t yet know if these numbers will hold under different contexts, methodologies, or timeframes.

But, at the very least, your belief that a great school can radically increase wages should be a little lower after reading this study.

Other Considerations

I’m still mulling this over, but in conversations with Roland and folks I work with, certain ideas bubbled up:

The data doesn’t capture recent improvements: A lot of the best charters have only really started focusing on college and career over the past 5 years or so. As such, the students who received the full suite of redesigned high schools, counseling, and career support aren’t represented in this study. To the extent you believe the best charters are problem solving machines, you might believe this to be true.

The work is generational: Perhaps reformed schools can only, on average, push students who would have been in deep poverty to achieve average poverty / lower middle-class status. And perhaps their children, who will grow up in better educated environments, will the be ones  to more fully make it into the middle class. But this story could be unwound through raised expectations: if we told kids they were going to make it to the middle class, and they don’t, how will they react?

Colleges are the bottleneck: Perhaps these real gains in learning are being wasted by ineffective two year and four year colleges – and that without higher education reform we won’t be able to translate K-12 gains into wage increases.

Society is tough: Just because you’re better educated doesn’t mean you can overcome racism, lack of social capital, and an over-reliance on signaling.

More interventions are needed: Great schools can’t solve everything; interventions that work on family poverty, health, and parenting are needed for schools to really move kids as far as they need to be moved.

The schools aren’t really that good: A bunch of teaching to the test just jacks up crystallized knowledge but doesn’t really give kids the human capital qualities they need to succeed in the workforce.

What Do you Do in the Face of Ambiguity?

Leaders need to make hard decisions in the face of incomplete data.

Often times, this means relying on some combination of probabilistic thinking, intuition, ideology, and philosophy.

But, at some point, you need to walk away if the data is telling you what you’re doing is not working.

I don’t think one study is enough to walk away from the promise of urban charter schools, especially since they’ve achieved so much on less penultimate markers.  I think there’s a lot more experimentation and research that needs to be done to help us understand if we can translate academic gains into wage growth.

But it’s worth thinking about when you would walk away.

Because if there is no point at which you’d walk away, then what do you really stand for?

The Answer is 6.7 Miles. What is the Question?

Screen Shot 2016-06-15 at 8.01.52 PM

The question is: how far, on average, would a family send their child to attend a school that is in the highest category of the state accountability system compared to a school in the lowest category of the state accountability system?

This is from a recent report on the DC public school system. The analysis, while useful, isn’t perfect in that it only includes families who utilized the enrollment system, but it does add to the emerging literature on the revealed preferences of families that participate in transparent enrollment systems.

 

___

Here’s another answer: it increases racial integration.

The question is: does DC’s unified enrollment system increase or decrease racial segregation?

Screen Shot 2016-06-15 at 8.06.39 PM

Shockingly enough, assigning families to neighborhood schools that are zoned by property values is not a great way to decrease segregation.

 

___

Answer: Unclear.

Question: Do parents care about a school’s academic growth (as opposed to absolute test scores)?

Screen Shot 2016-06-15 at 8.20.46 PM

Interesting but not shocking. Parents probably care a lot about peers and status.

Also interesting, this seems more true of low-income families:

Screen Shot 2016-06-15 at 8.25.26 PM

This raises an interesting question for policy makers: given that growth more accurately measures a school’s impact, should they design grading systems that prioritize growth (as DC’s charter framework does) even though low-income parents might care more about absolute scores?

Or perhaps not – maybe low-income families aren’t considering the growth based performance framework because the government is hiding this information:

Screen Shot 2016-06-15 at 8.27.05 PM

___

One last answer: Families who aren’t assigned to a school in the lowest performance category, as well as the politicians and superintendents who seek their favor.

The question: who loves neighborhood schools?

It remains shocking to me that public leaders in cities such as Oakland are vehemently opposed to unified enrollment on the grounds that such systems will undermine public education.

The only thing a unified enrollment system undermines is the privilege of those who benefit from institutional racism and widespread income inequality.

 

 

Is No Excuses or Personalized Learning the Low Hanging Fruit of School Improvement?

On average, I visit a school every other week or so. For the most part, these schools are equal to or better performing than the median urban district school.

During these visits, one question I usually mull over is this: if I was leading the school, what would I focus on to drive the next phase of improvement?

Often times, what the school leader is focusing on and what I would focus on are at odds.

I don’t have extremely high confidence in my analysis, so consider the below speculative.

____

Here are the things I most often here school leaders saying they need to improve on: personalization, student ownership, and critical thinking. Tactically speaking, this often leads them to experiment with new models of instruction and technology.

All good things.

But I’m often thinking that the school really needs to get better at: instructional delivery, higher ratios of student intellectual engagement, and more effective use of small group instruction.

Most school leaders seem to believe that they have the basics down and need to go from good to great.

I tend to think that most schools are mediocre at the basics of things such as cold calling, wait time, efficient time on task, and tutoring – and the other hall marks of the no excuses model.

___

So here’s some questions on my mind:

  1. Should the median charter school be focusing on getting better at the basics of the no excuses model or experimenting with deeper innovation?
  2. If it’s true that the median charter school is still mediocre at the no excuses basics, what should we take from this? That high fidelity to the no excuses basics is operationally hard to scale for either intellectual, emotional, or human capital reasons? That many leaders don’t think the no excuses basics work?
  3. Is there a progression of improvement (i.e., you need to get the basics right before you work on deeper innovation) – or does shifting to more innovative models allow you to bypass the no excuses basics and still get academic gains?

___

My hunch is this: only the top tier charter organizations and the very best entrepreneurs should be working deeply on the margin of innovation.

Most charter schools should be working on the margin of better adoption of the tenets of the no excuses model.

Once new models are hammered out and refined – and get better results than the no excuses model – then the median charter school should begin adopting these new models.

But not before that.

In sum, I think better fidelity to the no excuses model is the low-hanging fruit of school improvement.

Maybe I’m wrong? Maybe I’m very wrong?

The Challenge of Separating Emotional and Intellectual Agreeableness

There is a decent amount of research showing that agreeableness (as measured by the five factor personality test) is not always associated with strong professional outcomes.

Specifically, agreeableness can reduce results orientation and create opportunities to be taken advantage of by colleagues who better use power to achieve their desired ends.

That being said, agreeableness need not be all bad: to the extent that it helps cultivate large, loose networks, agreeableness is likely of use to leaders in attracting talent and coalition members, especially in the non-profit sector.

___

Many times, I’ll be in a conversation with a colleague, grantee, or potential grantee and there will be a small war going on my head: part of me wants to nod my head, smile, and ask probing but pleasant questions – while another party of me wants to dig in very hard on everything that might be wrong about what we’re discussing.

I have a strong desire to be both emotionally agreeable and intellectually disagreeable.

Which begs the question: is it possible to be emotionally agreeable while being intellectually disagreeable?

___

I’m not sure. But here’s some things I try to do:

  • Utilize processes that create a safe space for intellectual aggression (i.e., assigning someone to be the devil’s advocate in a meeting).
  • Using hedging phrases such as “I might have this wrong, but….” that soften the blows of intellectual aggression.
  • Trying to separate my empathy for a person with my disagreement with her ideas – so that my intellectual disagreeableness does not bleed into full blown personal animosity.

If you have any other tools, let me know.

I struggle to get the balance right.

Sometimes I feel like I’m too agreeable, and sometimes I feel like I’m too intellectually aggressive.

I Think High Schools Should Teach More of These Two Things

Trying to answer of what, on the margin, high schools should teach more of gets at the heart of numerous subquestions, including:

  • What is the role of schooling?
  • Where is the economy heading?
  • What level of content can high schoolers handle?
  • What is already being covered well?

I will steal from a bunch of bloggers, academics, and practitioners in a way that will be a little hard to cite, but it’s fair to say that little that follows below is that original.

A Primary Goal of Schooling is to Normalize that which is Not Normal

In hunter and gather societies, Algebra is not normal. Nor is conscientiousness. Nor is abstract logic.

You get the idea. As societies advance, there’s a bunch of things that may need to be taught because physical and cultural evolution do not provide enough guiding instruction.

Of course, school is isn’t the only way to learn things that aren’t normal, but we’ve chose to make it a major source of normalization.

So one way to answer the question – what should high schools teach more of – is to consider what is not normal now but that we would wish to be normal in the future.

Closely related: what is close enough to normal now that schooling could provide a meaningful nudge to normalization?

What Would Make the World Better if It Were Normal? 

A lot of recent books and blog posts have influenced me on this, perhaps most notably Superforecasters,  Scott Alexander’s post on how hard things are for a lot of people, and Triggers.

The thesis of each of these sources are as follows:

  1. Superforecasters = always start with baseline research and data before making any decision.
  2. Scott Alexander = even in America, a lot of people are suffering with major issues such as chronic pain, drug abuse, and unemployment.
  3. Triggers = adult behavior change is very hard but possible.

Over the past few years, recent events in my own life have driven home the importance and relevance of these theses.

What would make the world better if it were more normal?

I think these two things:

  1. An increased internalization that opinions should be formed based on baseline data and research. I mean this both in the policy sense (should we raise taxes?), the business sense (which website design should we use?), and personal life (which nursing home should I use?).
  2. Adult behavior change is possible and specific techniques can increase the chance of success. I mean this in the professional sense (I need to ask more questions in meetings), the interpersonal sense (I need to talk less about myself in groups), and the spiritual sense (I need to meditate more frequently).

Can High Schoolers Handle This?

I don’t know, but my guess is yes. Research trials could tell us. But none of the above are intellectually taxing in terms of complexity.

Rather, these two learnings are more just specific applications of conscientiousness, humility, and growth mindset.

It’s my strong guess that strong marginal improvement could be made in these areas without students ever entering a college campus. It feels like high school material, but I might be wrong.

Providing the Cultural Pressure and Intellectual Tools to Achieve Normalization 

With a few exceptions, high school curriculum covers a lot of important material. My guess is that, on the margin, more traditional content is not what is needed (i.e., more advanced math).

Rather, at the margin, I think there may be a lot of gains to be made in providing cultural pressure – and giving student the intellectual tools – to normalize the tools of data usage and adult behavior change.

But all this is surely speculative, and I don’t have strong confidence that I’m right.

Perhaps some high schools might push in this direction and see if it makes a difference in students’ lives.

Book Review(s): 6 Books on Our Mental Limits

I’ve had some good reading time over the past two months and have been able to get through six books (as well as the new Dragon Tattoo book, which will not be reviewed here):

  1. Triggers: Creating Behaviors that Last (adult behavior change)
  2. Superforecasting (predictions)
  3. Work Rules! (Google’s HR systems)
  4. Simple Rules (utilizing simple rules to guide decisions)
  5. The Evolution of Everything (evolution as a principle for all change)
  6. Hive Mind (how national IQ is more important than individual IQ)

All are worth reading.

Here are some major themes that ran through them all:

We Have Weak Minds

Triggers pushes hard on how much environment impacts us.

Super forecasting details how badly pundits do at prediction because they rely on situational judgment rather than baseline data.

Simple Rules makes a convincing case that the world is too complex to navigate by fully analyzing every situation.

The Evolution of Everything rightly argues that even our geniuses are most often well situated for breakthroughs due to past intellectual evolution, not because they along were capable of achieving such breakthroughs.

Collectively, We Have Better Minds

Hive Mind demonstrates how individual minds are made more effective by having other good minds around.

Triggers lays out an accountability regime whereby other people hold you accountable for your behavior commitments.

Superforecasting talks about how even the best forecasters improve when working together.

The Evolution of Everything narrates how it is our collective knowledge, built over the ages, that allows to enjoy the fruits of modernity.

Those Who Use Data Effectively Will Win 

Work Rules! vividly portrayed how heavily Google relies on data analysis to make any decision, be it about people or anything else.

One memorable quote went something like a manager saying this: “If you don’t give me data, I will give you my opinions, and you don’t want that.”

Super forecasting is all about how baseline data is needed to anchor any situational judgment.

Triggers recommends systematic daily tracking of any desired behavioral change.

How I’ve Changed Because of these Books

  1. After reading Triggers, I created an end of the day 10 question checklist to hold myself accountable for the behaviors I’m trying to implement (I use an app to record them every night).
  2. After reading Superforecasting, I’ve tried to ensure that we conduct a  research review of any issue before even beginning to make judgments, to ensure we understand baseline data.
  3. After reading Work Rules! I reflected on how much I over relied on my own judgment when I led NSNO. I should have done a better job of always asking for the data before making any managerial decisions.
  4. After reading Simple Rules, I revised a decision checklist I had made for grant making to include a priority rule (most of them were boundary rules and stop rules).
  5. After reading Hive Mind, I reflected on my strong preference for very open immigration. While I still hold this belief, the book helped me understand where and why I might draw limits.

I don’t know if I will be successful in sustaining any of these behavior changes. But I hope I can.

If you have a chance, I recommend picking any of the books up for holiday reading.

Pattern Recognition vs. Gut Instinct vs. Mood Affiliation vs. Tribe Affiliation

feelings

If I had to pinpoint the reason why people make poor decisions (myself included), I would probably put mood affiliation and tribe affiliation at the top.

All of this got me thinking of a taxonomy of sorts for quick decision making, none of which is terribly novel, but which might be of interest, as it both impacts leadership and education policy, two topics covered often on this blog.

Some definitions:

Pattern Recognition: Pattern recognition occurs when you make a decision, often without deliberate reflection, because something in your subconscious recognizes the situation, as well as the best way forward. Often, the decision is made through an emotional response – something “feels right” or “feels wrong.” Ideally, pattern recognition is built upon years of experience with the pattern at hand, so it can be a very effective way to make decisions.

Gut Instinct: I view gut instinct (there might be a better name for it), as following the exact same process as above without the foundation of historical experience. In short, it’s an emotional response triggered by a novel situation. Often times, this is a terrible way to make a decision, with the only exception perhaps being a situation that is novel to you but not novel to your ancestors, so the instinct might be hardcoded in your DNA.

Mood Affiliation: Mood affiliation occurs when you have an emotional reaction to an idea (rather than a situation) – and then you either supper or reject the idea based on how the idea makes you feel. Mood affiliation has numerous triggers: a writer’s tone or language; the idea being similar to other ideas you also support; the idea fitting into the identity you have created for yourself. Instead of analyzing the idea on the merits, you accept or reject the idea based on how it makes you feel.

Tribe Affiliation: Tribe affiliation is similar to mood affiliation, but more narrowly defined. Tribe affiliation is born out of a desire of wanting to belong to a group, with ideas being accepted or rejected based on the group’s opinion of the idea, rather than any individual analysis.

In sum: pattern recognition is a good heuristic, while gut instinct, mood affiliation, and tribe affiliation are not.

Personally, mood affiliation is the one I struggle with the most. When I read something positive about charter school, immigration, or driverless cars, my mood elevates. This is clearly dangerous, as it prevents me from soberly analyzing the issue at hand.

I find that twitter is basically live streaming mood affiliation, while blogging, at its best, allows for more reflective analysis.

So be careful out there.