Category Archives: Teacher Quality

The low ROI of focusing narrowly on teacher quality

Eric A. Hanushek, Marc Piopiunik, and Simon Wiederhold just published a study on international teacher quality.

They looked at an international sample of teacher cognitive ability (as measured by OECD’s Program for the International Assessment of Adult Competencies) and student achievement (as measured by PISA scores).

They found a lot of variation of teacher cognitive quality across countries.

Teachers in Finland and Japan score very well. Teachers in Italy, Russia, and Israel score poorly.

After running regressions and controls, the authors estimate that a one standard deviation increase in teacher quality leads to a .15 standard deviation in student achievement.

To put a one standard deviation increase in context, a one standard deviation bump from the mean in IQ scores is about 15 points.

I can’t think of a realistic intervention that, at scale, will lead to that kind of increase in teacher cognitive ability across the entire United States.

For me, this is just one more study that validates the idea that institutional change is the best hope we have for increasing educational opportunity.

High-performing urban charter school sectors across the United States often achieve a ~.1 standard deviation effect over the course of three to four years.

I believe we can thoughtfully grow these non-profit public schools in a manner that maintains quality at scale. I also think we can empower amazing district leaders by letting them adopt non-profit governance for their schools.

I really can’t think of a way that we increase teacher cognitive ability by one standard deviation at scale.

In every city we work in, we do invest in recruiting and supporting great teachers. Teachers should be recruited hard, paid more, and supported better.

But I’m skeptical that focusing most of our energy on improving our teacher pool is the best way to help low-income kids.

Instead, I think we should focus much of our energy on creating and expanding great public institutions where teachers can work.

Another way to say it: I don’t blame our teachers for the dysfunction that plagues many schools in cities across the United States.

It’s their employers that should be the focus of our attention.

 

 

 

In New York, Black and Hispanic Students Access Higher Quality Teachers by Attending Charter Schools

Some interesting data just came out of New York.

The state looked at teacher growth ratings across a variety of governance types and student demographics.

Below are graphs for Black and Hispanic students in Math and ELA.

The purple lines indicate highly effective teachers. Charters teachers who teach Black and Hispanic students are significantly more likely to be highly effective than teachers in others governance structures.

Screen Shot 2015-04-15 at 7.38.51 AM

For ELA, the charter results are less dramatic, though Hispanic students seem to gain real benefit in teacher quality from attending charter schools.

Screen Shot 2015-04-15 at 7.19.58 AM

Increasing the quality of teachers in our public schools is extremely important. This is especially true for those teachers that serve disadvantaged student populations.

New York has been a leader in increasing the rigor of teacher credentialing, which has led to an increase in the quality of teacher pipelines.

This data set appears to show another, or at least complimentary, route: expand the charter school sector.

Teacher Cognitive Skill Seems to Matter, Except in South Korea

Despite my tendency to drone on about charter schools, I also believe we should be spending a lot of time and energy on increasing the quality of our nation’s teaching force.

Yesterday, this study came out. The authors argue that teacher cognitive skill (as measured by adult PISA tests) is an important variable in understanding country academic performance; in their own words:

Our analysis consistently indicates that students living in the countries at the top of the PISA rankings perform better in math and reading in part because their teachers have higher numeracy and literacy skills

The methodology used is quite complex, and I don’t fully understand it, so I apologize in advance if I get things wrong below.

From the study…

Comparing Teachers from Around the World to People in Canada

The literacy skills of the lowest-performing teachers (in Italy and Russia) are similar to the literacy skills of employed Canadian adults with only a vocational degree (278 points). Teachers in Canada, the Netherlands, Norway, and Sweden have similar skills than adults with a bachelor degree (306 points). The literacy skills of the best-performing teachers (in Japan and Finland) are even higher than the skills of Canadian adults with a master or doctoral degree (314 points).

In short, the authors use comparisons to Canadian adults to demonstrate that there is a wide variety of cognitive skill in teachers across the globe.

Here’s a chart:

Screen Shot 2015-04-13 at 2.13.09 PM

Comparing Teachers to Other College Graduates In Their Home Countries

As most teachers are college graduates, it is also illuminating to compare teacher cognitive skills with the skills of all college graduates in a country (see Figure 2). While median teacher cognitive skills fall in the middle of the 25th-75th percentile skill range of cognitive skills of college graduates in most countries, teachers come from the upper part of the skill distribution in some countries (e.g., Finland and Japan) and from the lower part of the college graduate skill distribution in other countries (e.g., Poland and the Slovak Republic).

In short, teachers in some countries are near the top of cognitive performance for college graduates in their country, while teachers in other countries are closer to the bottom.

Here’s a chart:

Screen Shot 2015-04-13 at 2.12.14 PM

Korean Culture > Korean Teachers

I thought this was of interest: Korean students dramatically outperform what you would expect given their teacher quality.

Screen Shot 2015-04-13 at 2.16.06 PM

Culture matters.

How Much More Would USA Students Achieve if Our Teachers Had as Much Cognitive Skill as Teachers in Finland?

The U.S. would be expected to improve by roughly 0.55 SD in student math achievement… Of course, these are long-run impacts since they presume that the quality of students’ teachers in the first ten grades would improve to the level of Finland – something that would take some time and effort to realize.

This is a very large effect.

Does More Money Get You Better Teachers?

In other words, while making it clear that a more skilled teaching force will require higher salaries, the evidence says nothing about either how salaries should be structured or the responsiveness of teachers to higher salary offers.

It’s probably necessary but not sufficient.

Who Do Better Teachers Help the Most?

Our results suggest that the benefits of better cognitive skills of teachers mainly accrue to students with low socioeconomic background, while parental skills are more important for students with high socioeconomic background.

What is Optimal for a Country?

The authors did not really address the issue of cross-sector trade-offs. Presumably, a nation could boost its test scores by ensuring that their schools were saturated with teachers in high cognitive skill.

I imagine that, at some point however, this a diminishing returns to teacher cognitive skill. Additionally, some industries are probably made worse off by high cognitive skill college graduates going into teaching.

I wonder how you might try and model out what the sweet spot is.

So What Should We Do?

Two routes could get us to higher performing teachers: regulation or deregulation. I’ve written about this before. In certain states (New York) and countries (Finland), increasing the rigor of credentialing seems to have increased teacher cognitive skill.

In New Orleans, deregulation seems to have had the same effect.

One other twist: it’s unclear how economic shifts might affect labor supply, as well as how this factors into how much we would need to pay teachers to get a higher-quality workforce.

I think this is an area that’s right for experimentation.

Potentially, a combination of rigorous credentialing coupled with school deregulation could be the best way forward…. but much remains unknown.

Should You Bet on American Education Getting Better?

Two pieces of data, if they hold, could bode very well for the future of education in the United States of America.

1. Teacher SAT Scores on the Rise 

As this Education Next piece notes, entry teacher SAT scores are rising.

Screen Shot 2015-01-10 at 4.40.43 PM

2. Teacher Turnover is Decreasing 

According to this report, 70% of teachers now stay in teaching for at least five years. This is a marked increase from the 50-60% numbers we’re used to.

Screen Shot 2015-01-10 at 4.46.41 PM

Reflections

1. I previously have written that the combination of technology and globalization might make teaching a more attractive job for higher performing students (as other middle class jobs are lost). It’s probably too early to tell if technology and globalization are causing the recent increases in academic performance and retention. But, if the economic recovery doesn’t reverse these numbers, we may be in entering into a new labor equilibrium for the teaching force.

2. On this blog, I’ve staked my professional reputation on the idea that the academic performance will increase if we transition to well regulated, non-profit run, public school systems. This doesn’t mean that I think this is the only way we will increase student learning in this country. Better teachers who stay in the classroom longer will be good for students, regardless of governance structure.

4. If I was going to bet on whether American education will improve, flatline, or get worse – I would look very hard at the academic performance of teachers entering the profession, as well as how long these better qualified teachers stayed in the classroom. The aforementioned data makes me more bullish on American education.

What’s the Best Way to Attract Great Teachers: Labor Market Regulation or School Level Decentralization?

teacher

The Case for Professional Licensure Regimes

Generally, I think professional licenses are only necessary in cases where (1) it is difficult for consumers to gauge the quality of service (2) the cost of error can result in very serious harm (3) and the license is a good proxy for quality.

In medicine, you can make decent (though not air tight) argument that these conditions apply.

The Case Against Teacher Licensure Regimes 

I don’t think teaching meets these requirements, especially considering that licensure (especially master degree requirements) have not seemed to be a great proxy for quality. Moreover, I think parents and students can probably identify the worst teachers (who can inflict the most harm) in a school. Admittedly, judging between average and excellent might be more difficult.

In Louisiana, the only requirement you need to teach in a charter school is a college degree. While this means Bill Gates couldn’t teach in the state, I think a four year degree requirement probably doesn’t restrict too many potentially great teachers, and the lack of licensure requirements means that teachers don’t have to jump through meaningless hoops to be in the classroom.

I’ve generally felt that a four year degree was all we needed for licensure.

The Case For Teacher Licensure Regimes

However, there is an another argument to consider. Creating more sophisticated barriers to entry could (1) create a regime where licensure requirements effectively screen for what makes a great teacher (2) or, simply though selectivy, make teaching a higher status profession, which thereby increases the caliber of people who are drawn into the profession.

The Case Against Licensure Regimes

You might grant that the best licensure regimes could potentially lead to there being more effective teachers.

But you also might say: I’m pretty sure that, over time, any licensure regime will be either poorly designed at the outset, become outdated quickly, or be corrupted by inside interests; i.e, look what good masters in education have done us.

You might argue: we should leave it up to schools, and not the government, to determine who is a great teacher. As long as truly hold schools accountable, they, over time, will best be able to determine what makes a great teacher.

Data from New York

This study just came out. They authors find:

Since the turn of the 21st century, however, a number of federal, state, and local teacher accountability policies have been implemented toward improving teacher quality over the objections of some who argue the policies will decrease quality. In this paper we analyze 25 years of data on the academic ability of teachers in New York State and document that since 1999 the academic ability of both individuals certified and those entering teaching has steadily increased. These gains are widespread and have resulted in a substantial narrowing of the differences in teacher academic ability between high and low poverty schools and between white and minority teachers. We interpret these gains as evidence that the status of teaching is improving.

The report attributes this to regulation, not the economy:

To bolster the evidence that teacher accountability policies drove the turnaround in the average academic ability of teachers, we examine and subsequently rule out the competing hypothesis that these trends could perhaps result from changes in the characteristics of the teacher labor market such as the size of the market or salaries. For example, as salaries increase, the quality of the teacher supply should increase. Similarly, if the market demands fewer teachers, schools should be better able to restrict their hiring to the higher end of the ability distribution. While these labor market changes (themselves influenced by the macroeconomic cycle and declines in enrollment) are likely behind some of the changes in academic ability, they are probably not the dominant driver.

Reflections

I’m still spending time with the study, so I might get some stuff wrong here:

1. There’s a case to be made that, in NY, regulations drove up the average academic performance of teachers entering the profession.

2. I don’t view this as a slam dunk case. When considering economic factors, the authors seem to focus more on the teacher labor market and recessions rather than how technology and globalization might be reducing the number of middle class jobs available (and thereby pushing higher-performers into teaching). But I might have this wrong.

3. Additionally, as the author’s note: “academic or cognitive ability is one of the few observable teacher characteristics prior research has shown to be positively and consistently (though not strongly) associated with student achievement.” So the regulations might have caused in increase in teacher academic ability which is positively but not strongly correlated with teacher performance.

3. Are all these regulations worth it? It’s hard to say. Especially when you consider an alternate world: what if NY had instead passed a bunch of laws on school level accountability (with charter autonomies and real consequences for failure) and totally deregulated teacher licensure. This is what happened in New Orleans, and my guess is that teacher academic performance rose more in New Orleans than it did in New York. Someone should do a study on this.

4. I think the researchers should have considered these different policy regimes, rather than just studying the New York policy regime in isolation.

In Sum

Given the choice, I’d choose (1) effective charter school autonomy and accountability with labor market deregulation [NOLA model] than (2) ineffective school level autonomy and accountability with labor market regulation [NY model].

If this choice wasn’t on the table, I might choose (1) ineffective school level autonomy and accountability with labor market regulation rather than (2) ineffective school level autonomy and accountability with labor market deregulation.

I’m torn on whether I’d want (1) effective charter school autonomy and accountability with labor market regulation over (2) effective charter school autonomy and accountability with labor market deregulation.

Much to think about.