The KIPP pre-K study doesn’t tell us if KIPP pre-k works

Mathematica just published a study on KIPP pre-k. The Laura and John Arnold Foundation (where I work) funded the study.

The study was well designed and asked a very important question: does high-quality charter pre-k provide benefits above and beyond simply attending a  high-quality charter elementary school?

The policy implications of this question are important: if high-quality charter pre-k adds to the achievement of students who attend high-performing elementary charter schools, policies that support both early child funding and charter school growth could accelerate achievement gains.

However, if attending a high-quality charter pre-k has no effect beyond the impact of simply attending a high-quality charter elementary school, then our limited public resources may be best spent on expanding high-quality charter elementary schools and not devoting as much resources to pre-k.

In other words, it’s really important to understand if:

KIPP pre-K + KIPP elementary > KIPP elementary

Unfortunately, due to small sample sizes as well as sample attrition, the researchers were not able to answer this question.

While the effects of KIPP pre-K + elementary were larger than the effects of KIPP elementary, these results were not significant.

The researchers write:

In reading, the magnitude of the positive impact was larger for the pre-K cohort than the kindergarten cohort on both the Letter-Word and Passage Comprehension tests administered in grade 2 (by 0.20 and 0.06 standard deviation units, respectively). Neither of these differences is statistically significant; however, the study did not have sufficient power to detect differences of this magnitude. Thus, they may be suggestive of some additional benefit in reading resulting from an offer of admission to KIPP pre-K, above and beyond the impact of an offer to KIPP in kindergarten.

In math, however, the impacts for both samples are identical, suggesting no additional benefit of KIPP pre-K beyond the impact of a KIPP elementary school.

Below you can see the estimated (non-significant) impacts:

 

Screen Shot 2017-08-23 at 12.23.34 PM.png

So, for now, we still don’t know if KIPP pre-k adds any value over KIPP elementary school.

This is no knock on KIPP or the researchers, it’s just the consequence of the limitations of working with small sample sizes that are subject to attrition.

That being said, I do think some of my friends are playing a little fast and loose in over interpreting this study, and I think we all need to be honest about the fact that we don’t yet know if KIPP pre-k “works.”

2 thoughts on “The KIPP pre-K study doesn’t tell us if KIPP pre-k works

  1. Howard Dean

    I’ve always believed that while pre k helps, the real results come from 0-3 efforts with kids and families. Those are the critical years for the brain. Howard Dean

    >

    Like

    Reply
  2. nkingsl Post author

    Hey Howard – good to hear from you. I tend to agree. I sometimes wonder if we should get rid of junior and senior year of high school and take all that $ and drive it into 0-3.

    Like

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s