A recent Wall Street Journal editorial suggests that research proves the benefits of private school choice. Tulane University Professor Douglas N. Harris, a FutureEd research advisor who is studying school choice in New Orleans, examines the Journal’s claims.
School Choice Deniers
Critics hype a pair of studies while ignoring other evidence on education vouchers
The Journal’s headline writer and voucher supporters more generally equate vouchers with school choice. But there are several ways of providing families with school choice that don’t require vouchers, including charter schools, inter-district choice, and intra-district choice. So the headline is problematic; pointing to negative research findings on school vouchers doesn’t necessarily make one a “school choice denier.”
President Trump has made a cause of public and private school choice, and liberals who oppose evaluating teachers based on student achievement are now hyping a few studies that have found vouchers hurt student performance. A closer look still supports the case for giving parents choice.
More than 400,000 students in 30 states and Washington, D.C., participate in private-school choice programs whose designs and funding sources vary. Over the last two decades dozens of studies have sought to measure these programs’ impact on student growth. Those with the most rigorous methodologies have produced positive findings.
If by “rigorous” the Journal means having what researchers call randomization (a comparison of students who receive vouchers with those who sought but didn’t receive vouchers), this is false. The Louisiana example below is randomized and shows the worst effects on student achievement of any study of vouchers—really, some of the worst results ever observed for any program or policy.
A meta-analysis last year by the Friedman Foundation found that 14 of 18 empirical studies analyzing programs in which students were chosen at random by lottery found positive academic outcomes.
This statement is false on several levels. Here are the main problems:
(1) The Friedman report lists 18 studies of academic outcomes. But there are only six separate cities/states that have had randomized voucher programs; this means 12 of the 18 studies are similar analyses of the exact same students. The Friedman Foundation, now known as EdChoice, has double-counted these studies.
(2) EdChoice’s unusual standards for including studies have the effect of excluding two large, rigorous “quasi-experiments” (of voucher programs in Indiana and Ohio) that reflect poorly on vouchers.
(3) The review counts as positive any study that finds a positive effect for any racial or other subgroup of students. However, if you separate students into enough groups, someone is bound to benefit. Researchers call this the “multiple comparisons” problem.
Academic researchers would never carry out an analysis like this. In reality, there are eight programs with rigorous evidence—four positive, three negative, and one with no effect. But if these results are weighted based on the number of students in each study—a standard research practice—the overall effect of vouchers on student achievement in the studies is negative, substantially so. Even if we threw out the three negative results, the four positive ones are limited to a specific student subgroup: African Americans in urban school systems.
Two demonstrated no visible effect, while two recent studies of Louisiana’s voucher program found negative effects. The Louisiana studies are disconcerting since voucher proponents have hailed the program, and the negative effects were large. Math scores declined in one study by 0.4 standard deviations after one year in private schools, representing a 50 percent increase in likelihood of failing the state test.
Correct. It’s nice that the Journal was upfront about this, and about the negative effects in Ohio below.
But Louisiana’s voucher program is unusual in several respects. Fewer than a third of private schools participated in the first year, and they had already experienced significant enrollment declines. This suggests that voucher students had their pick of the worst private schools.
This is true, but it is also true of other forms of choice. This just reinforces the problem with conflating all forms of choice with vouchers.
Some higher performing schools may have been deterred by regulations that prohibit them from setting admissions standards and charging families more than the voucher amount—$5,300 on average in 2012.
Again, this is true, but misses an important point: a primary purpose of vouchers is to provide choice to those who do not have it. The fact that private schools want to maintain their admissions standards means that they are unlikely to serve the students who currently lack access.
Liberals also highlight a Fordham Institute study last year of Ohio’s voucher program that found participants performed “worse on state exams compared to their closely matched peers remaining in public schools.” The study wasn’t included in the Friedman meta-analysis because participating students were not chosen at random, and it excluded students attending the lowest-performing schools who might benefit most from vouchers.
The study did find that vouchers “improved the achievement of the public-school students who were eligible for a voucher but did not use it.”
This is true, but it is also true of other forms of choice. This just reinforces the problem with conflating choice with vouchers.
These students tended to be more economically disadvantaged and lower-achieving than those who used vouchers. It appears vouchers impelled low-performing public schools to improve to avoid losing students.
This conclusion is bolstered by the Friedman meta-analysis, which demonstrated positive effects in 31 of 33 studies evaluating the impact of vouchers on public schools. An analysis of Louisiana’s program last year found that student performance increased “in the public schools exposed to the threat of competition, with effect sizes growing in magnitude as the competitive threat looms larger.”
These studies rebut the union claim that vouchers harm students left behind in public schools.
Again, the other forms of choice have the same effect.
Notably, one of the outlier studies was of Washington, D.C., which compensates schools for funds they lose from voucher students.
One reason public schools in urban areas are so abysmal is that the predominantly low-income students they serve have no other options, while the affluent can flee to private schools. This reduces the political and economic incentives to shape up. Vouchers level the playing field among income strata—which liberals should support—and create competition for the public-school monopoly.
This might be true if: (a) private schools produced better outcomes at scale; and (b) the programs were designed to ensure access of low-income students. Neither condition appears to hold.
Progressives who cherry-pick negative data on vouchers are denying the overwhelming social science that shows private-school choice benefits both participants and public school students. These progressives are thwarting educational progress.
There is nothing overwhelming about this evidence. The only thing we know with any certainty is that most choice programs seem to benefit African American students living in cities. That’s important, but it doesn’t justify voucher programs, and certainly not universal, national programs of the sort the Trump administration is considering.