Cherry-Picking Evidence on Vouchers

As the Indiana spin machine continues, be very skeptical about claims that vouchers will help public schools. The standard M-O here is that groups supporting particular issues release "research reports" that support those issues and then the policy people cite those reports as evidence.  Ingenious, insidious.....unethical.

New Win-Win Report on School Vouchers Still Not a Winner


BOULDER, CO (April 19, 2011) – In 2009, the Friedman Foundation for Educational Choice released a report titled A Win-Win Solution: The Empirical Evidence on How Vouchers Affect Public Schools. The report was reviewed for the Think Twice project by University of Illinois professor Chris Lubienski, who concluded that the 2009 report purports to gather all available empirical evidence on the question of the competitive effects of vouchers, finding a strong consensus that vouchers help public schools. But the report, based on a review of 17 studies, selectively interprets the evidence to support the Foundation’s own conclusions.

The Friedman Foundation has now released an updated version of the report, combining the older discussion of the effects of school competition with a new discussion of various outcomes for voucher participants. The new report, however, is equally flawed.  The earlier Friedman report asserts that “contrary to the widespread claim that vouchers hurt public schools, the empirical evidence consistently supports the conclusion that vouchers improve public schools. No empirical study has ever found that vouchers had a negative impact on public schools” (executive summary).

Lubienski’s review explains that the 2009 report cherry-picks evidence and that the majority of studies cited “were produced by a very small group of people largely associated with school choice advocacy organizations” (p. 6).

Lubienski’s review of the original report explains that the cited “studies include some rigorous work by respected researchers. But issues of methodology, interpretation, and generalizability emerge when the research is marshaled simply to support a narrow agenda, as with the Friedman Foundation’s. Then, the temptation for selectively summarizing research can distort the actual findings” (p. 5).

For instance, Lubienski points to the report’s misuse of Carnoy and his colleagues’ (2007) research. The new report continues to inappropriately use the Carnoy work, as well as others, in order to contend that vouchers have a positive “competition” effect on public school systems. In fact, as Lubienski states, the actual study that Carnoy and colleagues conducted to test the competition effects of voucher systems “found no competition effect” in their results (p. 5).

Lubienski also criticized the earlier Friedman report for championing the expansion of school voucher programs without acknowledging that previous expansions did not increase any positive effects that could be attributed to competition. The new report again sidesteps an explanation for this lack of success, except to present it as evidence of the need to expand the programs more dramatically.

What the new report does add is a discussion of research concerning the effects of voucher programs on the students receiving the vouchers. The report cites eight studies, most of them conducted by avowed voucher advocates.

It is remarkable how unrestrained the report is in pursuing the conclusion of “positive effects” when a fair reading of these studies would conclude there is minimal to no effect. Most disconcerting is the over-reach and distortion employed in the report’s attempt to dismiss the prominent and peer-reviewed study by Princeton’s Alan Krueger and Pei Zhu, which found no effects of a voucher program in New York City.

While the new Win-Win report does improve upon the original report’s arguments in some areas, the overall logic and corresponding evidence still falls short of making the case. The report is not a useful review of the effects of school competition.

Links to Reports/Review:

The original 2009 Win-Win report, produced by The Friedman Foundation, can be found at the following link: http://tinyurl.com/44of9vr

The NEPC Think Twice Review of the 2009 Win-Win report, done by Chris Lubienski, can be found at the following link: http://nepc.colorado.edu/thinktank/review-win-win-solution

The new Win-Win report, released in March 2011 by The Friedman Foundation, can be found at the following link: http://tinyurl.com/3hm9jby

The Think Twice think tank review project (http://thinktankreview.org), a project of the National Education Policy Center, provides the public, policy makers, and the press with timely, academically sound, reviews of selected think tank publications. The project is made possible in part by the generous support of the Great Lakes Center for Education Research and Practice.