Category Archives: Predictions

What if unified enrollment platforms were 10x better?

An emerging group of cities – including Washington D.C., Newark, Camden, New Orleans, and Denver – have adopted unified enrollment systems. With these systems, families can enroll in schools across the city via an online application system.

This is a huge step forward. For too long, parents have not had enough information or access to the public schools in their cities.

However, the new enrollment systems are still in their infancy. The best version of these systems could radically improve public education. Unfortunately, we’re very far from this endgame.

I. Early Wins: Access, Equity, and Ranking

Access: With the best open enrollment systems, families who can’t afford a house in a fancy neighborhood can now finally transparently apply to a school in a more wealthy neighborhood.

As a result of increase in access, a recent study in Washington D.C. found that the new enrollment regime would likely reduce segregation over time:

Screen Shot 2017-03-17 at 9.23.22 AM

Equity: In most cities, traditional and charter schools play a ton of games with enrollment. Traditional magnet schools use opaque entry requirements. Bad apple charters don’t take in kids with special needs. There is no equity.

With online enrollment platforms, these problems go away, as schools are no longer in control of their enrollment.

Quite simply: the algorithm is fairer than the enrollment clerk.

Ranking: These new enrollment systems also allow parents to rank their top schools. This is extremely important.

First, a family’s high desire to enroll their child in a school can now  be translated into an increased chance that they actually get into this school.

Previously, high desire meant little unless you were connected, wealthy, or dogged.

Second, ranking allows  parents to publicly signal to government which schools are most and least in demand (which will ideally affect opening, expansion, and closure decisions). It also signals to school operators what attributes make a school in high demand.

By analyzing ranking preferences, researchers in New Orleans were able to correlate school characteristics with parent preference:

Screen Shot 2017-03-17 at 9.44.39 AMRanking transforms family desire into actionable information.

II. Unified Enrollment Systems are Mediocre Platforms

In preparation for writing this blog, I spend an hour on unified enrollment system websites. It was not a great experience.

Here is the school finder homepage from Washington D.C. – I couldn’t even find a way to filter schools by academic performance!

Screen Shot 2017-03-17 at 9.50.14 AM

Here is the search function for Newark’s enrollment system – you have to download a pdf!

Screen Shot 2017-03-17 at 9.55.36 AM

By comparison, here’s the search page from Zillow:

Screen Shot 2017-03-17 at 9.53.10 AM.png

On Zillow, you can easily search for homes based on the most useful search criteria. Yes, Zillow is surely better capitalized than your average enrollment system, but even with modest funds a city should be able to do better than a downloadable pdf.

III. Moving From Equity and Ranking to Matching and Prediction

More sophisticated uniform enrollment could offer two extraordinary improvements: they could better match families with schools, and they could better predict how any given student would do at a school.

Matching: Right now families mostly use enrollment systems for ranking: they know the schools they want and they use enrollment systems to express this desire.

What is not really happening (as far as I can tell) is sophisticated algorithms actually helping families match with schools.

For example, on Camden’s enrollment site (where you can thankfully filter by academic performance!), I found three schools that all met the “on track” performance criteria, and pulled up the comparison page:

Screen Shot 2017-03-17 at 10.11.14 AM

This does very little to help me choose between these schools. My preference variables are limited to very broad categories such as “art classes” and “after school programs.”

After playing around on these websites, I get very little indication that that the platform knows me or the schools very well. Compare this to Netflix, Amazon, or dating websites (previous life) – platforms where I feel like the algorithms know me better than myself.

Unified enrollment systems need to more deeply understand children and schools in order to make better matches.

Prediction: Right now, government accountability systems are a basket case of poor design (generally don’t weight growth enough), brutal politics (what politician wants to tell a bunch of communities they only have “F” schools?), and awful transparency (good luck trying to navigate your average state department of education website).

Most importantly, government accountability systems evaluate schools rather than make predictions.

As a parent, it’s one thing to tell you that a school is a “C+” – it’s another thing to give you a prediction of what will happen to your child if she attends the school.

With current date, we could probably gather basic information on your child’s age, gender, current academic performance, personality type, etc.,  and make a reasonably accurate prediction that if she attends school X she will have a Y% chance of graduating from high schools and a Z% chance of earning a post-secondary degree.

Good enrollment systems, over time, should become better and better predictive agents, and, perhaps, can end up augmenting (displacing?) government accountability systems.

IV. Root Causes and Potential Solutions 

I don’t yet have strong beliefs about the root causes of why these enrollment products aren’t getting better faster. But here’s some guesses:

Non-profits > government operated: Most of the enrollment systems are run by governments, which are not good at running tech products and have bad incentives around giving parents accurate information about schools. Non-profits would likely be better operationally and have better incentives, and avoid the privacy concerns associated with for-profits.

Lack of scale: Matching and predication can better with bigger data sets, and if all these systems are structured as isolated city based data silos, the algorithms will be dumber than they should be.

Weak Customer Demand -> Bad Economics: SchoolMint, from what I understand, is the most successful player in the market. For reasons I don’t underhand, this company has not developed a better product. Perhaps it’s because their government customers don’t actually want it. Or perhaps the economics don’t work (which might suggest philanthropy is needed).

If the above is true, a national non-profit should be backed to scale to enough size to create smart algorithms, and it should be financially structured in a manner that gets it out of the perverse incentives of being beholden to government or individual schools rather than families.

A philanthropic foundation with a great tech backbone could be well situated to support this endeavor.

V. Expectations

Better matching and prediction would probably not make the average student’s education experience 10x better, just as dating websites don’t inevitably lead to great marriages.

But I do think better matching and predication could increase the probabilities that millions of families could find a better fit for their children.

At scale, that’s a better world.

6 Numbers to Watch in 2015

prediction ball

In the blogging world, it’s become obligatory to put out yearly predictions.

While I have thoughts on longer-term trends, I have no idea what will happen in any given year.

That being said, here’s numbers I’ll be watching to better understand where the long-term trends are heading.

1. Math Software Program Effects Sizes

We’re starting to see math software achieve .1-.2 effects. If these effects continue to hold-up, and they do so on rigorous math content, I’ll grow more bullish on tech based math instruction.

2. Average SAT Score of New Teachers

Research is showing that new teachers are increasingly scoring better on aptitude tests, which probably bodes well for long-term teacher effectiveness. I’m curious whether this will hold as the effects of the recession recede.

3. National Charter School Market Share

Over the past couple years, charter market share has grown between 7-10% annually. Given that millions of children attend charter schools, this pace of growth is very significant. I’m interested to see if the growth rate holds as the absolute number of students continues to increase.

4. Number of Urban Charter Markets with +40% Penetration

Urban markets should be viewed as distinct from overall national charter market share, as this is where the best charter work is occurring. In the decade, my hunch is we’ll have over ten urban markets that are majority charter. This evolution will began to call into question the very structure of public education in major cities. So I’m curious to see what progress we make on this mark in 2015 (2014 market share data here).

5. Number of States with Rigorous Assessments 

Note that I didn’t say standards, which are useless. Nor did I say PARC or Smart Balance, which are simply two types of rigorous assessments. The number I care most about is how many states implement some form of rigorous assessments. Given that I do believe assessments impact instruction, I’m eager to understand how much the Common Core push will lead to more rigorous assessments, in some form or another.

6. Teacher Union Membership

Teacher unions have been losing members. Relatedly, the number of teachers covered by collective bargaining agreements continues to drop. If this trend continues for the next decade, will likely impact the politics of education reform.

In Sum

Long-term trends in tech effectiveness, teacher quality, education governance, state assessments, and union membership will all affect the future of education in this country.

It’s worth keeping an eye on these numbers.