Category Archives: Professional Development

Roland Fryer and the Root Cause of Good Management

Screen Shot 2017-05-30 at 3.44.45 PM

Roland Fryer is one of the top education researchers in the country. His research is always thought provoking and whenever we talk I learn something.

If there’s one area Roland and I may disagree on, it’s the potential for school districts to sustainably adopt the best practices of charter schools (which Roland has been instrumental in helping us understand).

This issue is of course wrapped up in the bigger question: will the greatest value of charter schools be the birth of  innovative practices or the scaling of a better governance model?

I. Roland’s New Research: MGMT Matters

Roland just came out with a fascinating study on the importance of effective principal management.

The experimental research project was set in Houston and provided principal management training (much of it borrowed from Paul Bambrick-Santoyo of Uncommon Schools) to a treatment group of school district principals.

The researchers found:

Overall, the estimates suggest that management training was effective in year one – increasing efficiency approximately 7% — but produced precisely estimated zeros in year two. Pooling the two years produces marginally significant results that fall on the other side of significance with more conservative standard errors. Management training tends to be more effective with more flexible, stable and higher human capital principals and teachers. The most robust partitions of the data are whether a principal was employed for both years of the experiment and fidelity of implementation of the management training.

In sum: they found impressive effects with talented principals who stayed in the job for two years but no effects overall due to principal turnover and too many low human capital leaders.

II. Why Did Fryer Need to Conduct an Experiment in Houston?

Data driven instruction and teacher feedback, which were key to the intervention, are not new ideas. Bambrick wrote Leveraged Leadership in 2012. And he surely wasn’t the first to implement these management practices.

So why did Fryer need to construct an experiment to apply these sound management practices in Houston?

Why wasn’t the Houston school district applying these techniques already?

As it happens, some other researchers (Nicholas Bloom, Erik Brynjolfsson, Lucia Foster, Ron Jarmin, Megha Patnaik, Itay Saporta Eksten, John Van Reenen) just published a paper on this very subject – with the aim of trying to understand the root causes of good management practices.

III. What are the Root Causes of Good Management Practices?

It’s hard to do a controlled experiment on management practices in the private sector, so caution is warranted in interpreting the results.

The authors used survey data and business results to determine whether sound management practices are correlated to increased business success (they are), and then tried to figure out what business conditions led to better management practices.

While the methodology is inherently tricky, it did reaffirm my priors.

The researchers found:

What could cause these huge differences in management practices across establishments? We found several major factors. First, establishments in more competitive industries (measured by the Lerner index) adopt more structured management practices. Second, those in more pro-business states (proxied by states with ‘right to work’ laws, as in Holmes 1998) tend to use more structured management practices. Third, establishments with more college graduates and firms located near universities (building on the work of Moretti 2004 for identification) tended to adopt more structured management practices. Fourth, being located near a successful large new entrant (using the ‘million dollar plants’ identification strategy of Greenstone et al. 2010) is correlated with more structured management practices, likely because it allows local companies to learn about practices from these large, successful firms.

All these factors matter, but they explained less than half of the variation in management techniques, which means that many other factors matter, too. One hypothesis is that individual managers and CEOs themselves are another critical driver (e.g. Bandiera et al. 2017).

To summarize: good management practices were most often found in (1) competitive industries (2) with less restrictive labor laws (3) located near universities and (4) successful new start-ups.

I know a city educational system that meets all these conditions.

It happened to achieve some of the best student achievement results the country has recently seen.

IV. Yes And

It if it ever occurs, it will take a few decades to scale the charter sector serve the vast majority of low-income students.

For this reason, I appreciate Roland’s efforts to see if charter practices can increase achievement in districts. While I don’t think this is the long-term game, there might be short-term benefits to be had.

But if you want these achievement gains to be sustained, you have to address root causes.

And the root cause of good management is not really about intellectually understanding good management practices.

Rather, it’s about creating the enabling conditions to sustainably execute these management practices.

I believe that non-profit governance will prove to be one of the most important enabling conditions in the public education sector.

Teachers Could Have it So Much Better If….

Screen Shot 2015-08-20 at 12.53.40 AM

Screen Shot 2015-08-20 at 1.03.22 AM

Two studies just came out on professional development: one by TNTP and one on Leading Educators via Rand Corporation.

Big shout to TNTP for focusing attention on an extremely important issue and to Leading Educators for having the discipline to formally evaluate their work. Also, the CEOs of both orgs (Dan at TNTP, Jonas at Leading Educators) were generous enough to review and give feedback on this post (all thoughts below are mine only).

Highlights and analysis below.

The highlights from TNTP’s report, which studied three districts and one CMO:

  • Professional development in the three districts did not lead to significant gains in teacher effectiveness.
  • Professional development in the CMO improved teacher performance at much higher rates.
  • In the districts, most teacher improvement occurs within the first few years of teaching and then most teachers plateau in performance (before becoming very effective). See first graphic above.
  • In the CMO, teachers continued to improve overtime. See second graphic above. Keep your eyes on the blue bars.
  • A lot of money is being spent (and in many times wasted), with districts spending an estimated 18K per teacher on professional development, and the CMO in the study spending 33K.

The highlights from study on Leading Educators, which covered fellows in New Orleans (nearly all in charter schools) and Kansas City (nearly all in district schools):

  • The early findings are promising but mixed, and overall do not yet conclusively demonstrate that the program has affected student achievement.
  • There was leadership and management skill attainment across the board to a statistically significant positive degree.
  • With the exception of the positive impacts among fellows (Leading Educator participants) who teach math in New Orleans, the student achievement findings are generally inconsistent across different analysis.
  • The New Orleans math results, depending on which methodology is used, roughly equate to the benefit that students experience from attending a highly effective urban charter school.
  • There is some suggestive evidence of beneficial program impacts among mentees (teachers supported by Leading Educators fellows) in Louisiana, in two of four subject areas.

Major Takeaways:

Professional development only seems to lead to student achievement increases in charter schools!

This appears to be the major takeaway of both studies, though I haven’t seen much commentary on this point. But in both studies positive achievement effects were only found in the charter sector. In the TNTP report, it is unclear whether the teacher growth was a direct results of professional development (in that not all teachers who received PD got better). This may because even good PD will not work for everyone, or that other factors (hiring, org culture, etc.) were really driving the gains.

Both TNTP and Leading Educators hold hope that this can change.

In talking with the CEOs of both organizations, each expressed a belief that effective professional development can occur in districts.

My guess is that gains in district professional development are attainable but will be very modest.

I am not a district nihilist. I think reports such as those put out by TNTP, as well as support provided by groups such as Leading Educators, can increase district performance at the margins. I just think these improvements, as we see with most other district improvements, will be small and pale in comparison to the gains of effective charter schools.

I think we are more likely to scale effective charters than we are to see major gains in districts.

 A common retort to the aforementioned analysis is: “well, districts are where he kids are at.” In other words, it’s better to work for small gains in districts rather than large gains in charters because charters only serve ~5% of public students in our country. This is the wrong way to think about it! The question you need to ask is: is it more likely that we see can achieve major gains in districts or scale highly effective charters?

Both strategies have steep odds against them.

But I think it is more likely that we will be able to scale effective charter schools.

As such, I think focusing our efforts on charter growth is the best way to increase the effectiveness of professional development.