The case against the Mississippi miracle, part II

Doing my haters' jobs for them

Education skeptics hate her! (Jose Luis Pelaez Inc./Getty Images)

Jerusalem will be answering reader questions on a special “Ask Me Anything” episode of The Argument podcast. If you’re interested in asking a question about building a new publication, liberalism, housing, NIMBYism, politics, or anything else, send a voice memo to podcast@theargumentmag.com.

---

For the sake of thoroughness, I’m going to write the definitive case that there’s nothing happening in Mississippi. I don’t believe this is true, to be clear; I have now written four articles defending it on the following grounds: The gains are steady; the gains happen across every decile of student performance; and the National Assessment of Education Progress (NAEP) is much harder to game than most state and local standardized tests.

However, I share the perspective of many in the education policy world that lots of things that look good on first inspection don’t hold up when you look more closely. I take to heart in particular the criticism that a lot of education results are actually just selection effects.

For that reason, I tried very hard to find any way that Mississippi’s results could just be the result of selection. I looked primarily at the NAEP technical appendices, which have a lot of data on participation rates, accommodations, and students who were exempted from testing.

I looked carefully at the rate of students in Mississippi who, for whatever reason, don’t take the NAEP, and Mississippi doesn’t have more students skipping the test than other states.

I looked at test-taking rates for students with disabilities and special-education students, and whether Mississippi was claiming more students with disabilities and special-education needs than other states. (They are not.)

I looked into whether the pass rates to enter fourth grade varied between years with an NAEP test and years without it.

I also looked at general state data to see if the composition of students might have significantly changed. Some states saw scores improve as educated wealthy families moved in — which is not an education intervention — or saw large shifts between public and private education. So I looked at whether the share of the population in public versus private schools had shifted dramatically.

I looked at whether Mississippi had had an influx of higher-income, more-educated families.

Skepticism without investigation is just nihilism. If you want to do more than just prove you’re smart enough to throw around the words “causal inference,” skepticism should lead you to inspect results carefully, and with education data, most of the information you need for this careful inspection is public.

Having done all that work, I still think the Mississippi Miracle is real. But because I’m an exacting person, I’m also annoyed that the skeptics are doing such a bad job of making their case, and I think I could write a better one. If I’m going to have to constantly interact with arguments that there’s nothing happening in Mississippi, I’d like them to be high-quality arguments.