Category Archives: Charter School Authorization

Response to Matt Ladner and Jay Greene

Over at Jay’s blog, Jay and Matt wrote two critical posts of portfolio management and harbormasters (our team calls them quarterbacks).

Jay and Matt think differently than I do, have different political orientations, and are sharp. Their writing makes me smarter.

I also find their post titles, writing tone, and evidence analysis to be a bit over the top. They sometimes overstate their claims and under appreciate the other side of the argument.

In these two posts, Jay and Matt use NAEP charter sector gains in Arizona, Michigan, and Texas – as well as the mediocre NAEP scores seen in Louisiana’s charter sector – to argue that portfolio management and quarterbacks aren’t working.

I found their analysis to be overly narrow. Instead of taking some new evidence in and synthesizing this with the broad set of evidence available, they anchored on to one set of data points and made too strong of claims (especially in the titles of their posts).

Don’t Look at NAEP in Isolation 

Matt is right to point out that some states with fairly loose charter regulations saw a lot of charter gains in NAEP between 2009 and 2017.

I think this should modestly increase our belief that being loose on charter openings and closings can lead, over time, to a healthy charter sector.

But the story is not that clean.

This CREDO paper, which looked at charter school performance in Texas between 2011 and 2015, found a small positive effect in reading and no effect in math. Given that CREDO tracks individual students across time, and NAEP does not, the CREDO data should make us cautious in interpreting the NAEP gains as a huge victory.

Screen Shot 2018-04-24 at 9.57.03 PM

Another study found that it took Texas charters ten years to achieve the performance gains of their traditional school peers.

While it’s great to see the sector improving in Texas, perhaps with better regulation the sector wouldn’t have had to improve for ten years just to achieve neutral effects.

The story of the Texas charter sector is much more complicated than Matt’s piece indicated. The same is true for Arizona and Florida, where quasi-experimental research has found muted test score effects.

Given the different results of NAEP and CREDO, we should be trying to figure out the puzzle rather than claiming victory, as Matt tends to do.

Don’t Look at States in Isolation 

Looking at state level gains is not a great measure of whether portfolio and quarterbacks are working, as portfolio (to some extent) and quarterbacks (almost always) are city based endeavors.

For city based data, this CREDO study measured student learning effects in a bunch of cities across the country.

Washington D.C., Denver, New Orleans, and Newark all did very well. These cities are also some of the most mature portfolio cities.

Phoenix, Austin, El Paso, Forth Worth, Mesa, and San Antonio did not do well. These cities are all found in loose regulation states.

It hardly seems like a slam dunk to me that portfolio / quarterbacks are bad and loose regulation is good.

Of course, the CREDO analysis is not perfect: test scores aren’t everything and the virtual twin methodology may miss unobserved differences between students.

But looking at state based NAEP scores to make broad judgements on portfolio and quarterbacks is unwise, especially with so much other evidence available.

The portfolio / quarterback model seems to be doing some good in many cities.

Quarterbacks are a Step in the Right Direction  

Jay and Matt often criticize quarterbacks as vehicles for people who think they are smarter than everyone else (especially educators and families). I find this to be an overly simple critique.

Quarterback originated as a way to use expertise to aggregate and allocate philanthropy.

In many cities, philanthropists were funding low impact activities, often wasting it on the  pet projects of district leadership. A lot of money was spent for very little academic gain.

Quarterbacks have helped improve philanthropy: instead of just passively giving money to the district, philanthropists partner with expert management teams to try and launch and grow great non-profits.

I think this is a major improvement on the status quo. Of course, there are some drawbacks, and too much centralization of philanthropic capital poses risks. This is why I don’t think all of a city’s philanthropic capital should flow through one organization.

But quarterbacks are increasing, not decreasing, educator entrepreneurship and family choice. Yes, they do often use test score results selecting who to fund, but I suspect this will change if a better way to invest is developed over time.

In Sum

The NAEP data should not be ignored. It’s made me more open to the idea that looser regulations can lead to charter scale and quality, especially at the state level. And I found Matt’s data analysis to be quite helpful. I love it when smart people who think differently than me play with complex data sets and come to novel conclusions.

But I think there’s plenty of other state based evidence that should make us cautious, such as the CREDO Texas study.

I also think there’s a lot of evidence that the charter sectors in portfolio / quarterback cities are making a lot of gains. The NAEP data Matt and Jay site, which is state based and does not track individual students, is not convincing enough to make me deeply question our city based work.

All that being said, I look forward to reading more from Jay and Matt in the future.

I don’t take smart, critical friends for granted.

The best school district in the United States?

One of the joys of my job is the number of amazing emails that arrive in my inbox.

Below is an email (pasted with permission) from Scott Pearson, the head of the Washington DC Public Charter School Board.

On this blog, as well as on twitter, we debate a lot about regulation. We have a lot to figure out and these debates help me get smarter.

But leaders on the ground have to lead, always with imperfect information and complicated local contexts.

The DC Public Charter School Board has chosen to regulate the charter community fairly tightly on performance, but more loosely on other inputs. As Scott notes in his letter, over 40 charters have closed in Washington DC over the past decade. While I don’t know if this is right for every community, the DC charter community is providing a lot of great options for tens of thousands of children, and they have undoubtedly made DC a better city.

The continuity of the DC charter community’s success also reinforces my belief in the importance of non-profit governance. It’s hard to think of a better school district in the country, and I’m highly confident that a primary key to their success is their structure: the DC Public Charter School Board regulates and non-profits operate.

It’s a winning formula for kids.

___________________

Dear Colleagues,

We have faced nearly a year’s worth of bad news about DC Public Schools, from high teacher turnover, to faked suspension data, from inflated graduation rates, to the resignation of the DCPS chancellor, to residency fraud.  This steady drumbeat has undermined confidence in our traditional public schools – far more than is warranted in my opinion.  DCPS is a vastly better school system than it was in 2007 when Michelle Rhee and Kaya Henderson took the reins.

But I’m writing today about the other half of public schools in DC, the 120 public charter schools serving 43,340 students – nearly half (47.5%) of DC’s public school students.  Our story is mostly one of continued success, growth, popularity, and quality improvement.  I feel the need to write this because I fear that the bad news about DCPS is drowning out what continues to be a remarkable story of charter school success in our nation’s capital.

First, it’s important to say that not one of these bad news stories has been about public charter schools.  Our graduation rates check out – because for years the DC Public Charter School Board has audited the transcript of every graduating senior.  Our discipline data has never been questioned.  And there have been no allegations that anybody has jumped the lottery queue at a DC charter school.

Of course, we’re not perfect. In a sector as diverse as ours there will be unflattering news to uncover.  But at least so far none of the year’s bad news has touched DC’s charter sector.

Second, our quality keeps improving.   The NAEP “flatline” story just doesn’t apply to DC’s charter schools, as David Osborne and Emily Langhornewrite in The 74.  We were up this year in three of four grade/subject areas, with fourth-grade reading scores climbing five scale score points.  That’s more than any state in the nation.  And over ten years our growth overall is faster than any other state or district.  Charter scale scores have grown 17 points in 4th-grade math, 19 points in 4th-grade reading, and 12 points in 8th-grade math.  Each of these represents over a year’s worth of learning gains.   Only in 8th-grade reading are our scores disappointing, down 2 points over the past ten years and lagging far behind our big-city peers.

Our school leaders deserve much of the credit for this growth, but the authorizer gets some credit, too.  Since 2007 we have overseen the closure of 40 low-performing charter schools, all the while aggressively supporting growth for our highest-performing schools.

Third, our improvements are not driven by a change in our demographics.  Charter schools continue to enroll higher percentages of black and low-income students than does DC Public Schools.  And charters schools now educate the same percentage of students with disabilities as does DCPS – and higher percentages of our most disabled children.

Fourth, even as our quality improves, our schools have made remarkable progress reducing out of school suspensions and expulsions.  Out of school suspension rates are down by half since 2011-12, to under 7%.  And expulsions are down over 80% to less than 0.25% – about the national average.  We’ve achieved this through transparency and communications, not through mandates.  (Though, disappointingly, our city council is now threatening to regulate school discipline.)

Finally, demand keeps growing.  Despite the charter board adding nearly 9,000 charter school seats since 2013-14, the number of unique families on charter school waitlists has risen from 7,205 in April, 2014, to 11,317 in April, 2018.  Two-thirds of our charter schools saw their waitlist length increase from last year to this year.   Waitlists are in one sense a measure of our success because it shows families want our schools.  But is also a measure of our failure – and that of the District government – to provide our residents with enough quality schools and the facilities to house them.  We need to do better.

Thank you for reading, and please reach out if there is more information or context I can provide.

Scott Pearson

Executive Director

DC Public Charter School Board

The Great Charter School Clean Up?

 

Florida, Arizona, and Texas are known for having large charter school markets with large variation in quality.

Taking the first letter from each state name, let’s call these the FAT states.

All told, charter sectors in FAT states serve about 750,000 students (AZ = 180,000,FL = 280,000, TX = 280,000) – or about 25% of all charter students in the country.

Results in the FAT states have been mixed.

Here is what CREDO found in 2015 when they studied Texas charter school data:

Screen Shot 2016-02-06 at 10.13.34 AM

Even in urban areas, where charters usually perform the best, Texas charters lag compared to their traditional peers.

Previous CREDO studies in Arizona and Florida have found negative to mediocre results; however, more recent studies, especially those focusing on attainment, have found more positive results.

But, in terms of matched test results, the FAT states tend to poorly when compared to the charter sectors of Louisiana, Colorado, and Massachusetts.

___

The FAT states split the reform community.

For many of those whose posts show up on Jay Green’s blog, the FAT states are exactly what we need: high levels of entrepreneurship, disruption, and parent choice.

For many of those whose posts show up on CRPE’s blog, the FAT states have serious shortcomings: they represent the triumph of free market mania over the pragmatic restraints of quality control.

___

But something interesting is happening in the FAT states.

They are closing a lot of charter schools.

Last year, Florida closed 35 charter schools; Arizona closed 30 charters schools; and Texas closed 62 charter schools.

In Florida, regulators closed ~5% of all charter schools in a single year.

In Arizona, regulators closed ~6% of all charter schools in a single year. 

In Texas, regulators closed ~8% of all charter schools in a single year.

These rates are higher than the national charter school closure rates of ~4%.

In the case of Texas, their closure rate was double the national average.

___

We will learn much from the behavior of the FAT states (the next great charter paper must be lurking in this data).

Here are questions for which I would love to know the answer:

  1. How do the schools that are being closed compare to district schools peers in terms of academic growth, post-secondary attainment, earnings, and parent and student satisfaction?
  2. Over a long-period (25 years?) is it better for a state to let a thousand flowers bloom and then clean up the sector or to put on tight quality controls at the outset and then  allow for measured replication? Or somewhere in-between?
  3. How does the size of a charter sector affect its political support in the state legislature?
  4. How does the quality of a charter sector affect its political support in the state legislature?
  5. How does support in the state legislature affect quality control measures?

I’m sure there is more to be mined from the behavior of the FAT states.

The MySpace Strategy

my space

The Philadelphia School Reform Commission just approved five new charters. It received 39 applications.

All new charters were granted to existing operators.*

I don’t know enough about Philadelphia to weigh in on the optimal political strategy for increasing high-quality charters. Perhaps this was all that was politically feasible at the moment.

More worrisome is the implicit strategy of these approvals: only expand what already exists.

In other words: don’t allow Twitter to form; keep Facebook from going live; squash Tumblr before it begins; reject LinkedIn; no need for Instagram.

Why take risks on new organizations when you have MySpace?

*Note: nothing in this post is meant to diminish the great work of existing operators such as Mastery. Their accomplishments are irrelevant to the point I’m trying to make.

Is DC PCSB the Best Charter Authorizer in the Nation?

Unknown

A strong argument could be made. The case for the DC Public Charter School Board would look something like this:

1. There are many authorizers, working in cities such as Boston, Newark, and New Orleans, that have created charter sectors with strong effect sizes.

2. However, there are few authorizers that inherited a weak charter sector (due in part to poor district authorizing), and then turned the sector around to the point where the sector now achieves ~.1 effect sizes when compared to the traditional sector.

3. PCSB did exactly this.

4. And they did it at scale: achieving nearly 50% market share in a major urban area.

If you want to learn more, Dell just did a case study on PCSB. You can read about it here.

One last note: the following table was also in the report:

Screen Shot 2014-10-27 at 8.21.19 AM

This data is from 2012-13, so not sure how things have changed, but I was surprised to see that only 61% of district students are eligible for free and reduced lunch. In the aggregate, DCPS is socioeconomically diverse district.

My guess is that this, masks three distinct subgroups: schools with very high free and reduced lunch rates, schools with mixed socioeconomic rates, and schools that primarily serve the wealthy.

In many ways, each of these subgroups is probably a district unto itself.

I’d be curious to see DCPS performance broken out by schools with below 35% FRL; 35-75% FRL; and 76%+FRL.

Does anyone have this data?