The oft-cited pollster Mark Blumenthal, who writes about all matters related to polling and its methodology on his Mystery Pollster blog, has posted the results of his analysis/investigation into what he feels may have gone awry in the Columbus Dispatch polling just prior to the November 8th election in Ohio.
We've previously reported on the final results of the five initiatives on the ballot in the Buckeye State two weeks ago - four of them part of an Election Reform package --- and the "staggeringly impossible" disparities, as we described them, revealed by those results when compared to the results of the Dispatch's final pre-election polling. To catch anybody up who hasn't seen the numbers, here's one set of them:
ISSUE 3 (Revise campaign contribution limits)
PRE-POLLING: 61% Yes, 25% No, 14% Undecided
FINAL RESULT: 33% Yes, 66% No
Blumenthal deserves kudos for his serious and detailed investigation into both the Dispatch survey's "long and remarkable history of success," as he describes it, and where he feels they might have gone so astoundingly wrong this year for the very first time. He points to a few changes in the methodology for the poll; the Dispatch didn't use the precise text of the initiatives in its survey; included an "Undecided" option for the first time; an examination of the data seems to show the poll occurred at a time that the electorate was in the process of changing its opinions on the issues; and finally that many voters found the text of the actual ballot initiatives to be long and confusing leading them to take the safer vote of "No" when they were not clear about what they were actually voting for.
Where he doesn't deserve credit is in his approach which presumes from the start that there must be something wrong with the polling itself --- but not the Election Results --- to explain the unprecedented disparity between the polling and the final results of the actual election.
As a pollster himself, he surely knows that it's quite simple to construct a poll in such a way that one can get back results which match just about any pre-determined outcome an unscrupulous pollster may hope for. Similarly, as with Blumenthal's earlier efforts to examine the disparity of Exit Poll Results vs. Final Election Results in last November's Presidential Election, his approach presumes from the get-go that the polling, not the results, are incorrect...and now we just need to figure out why.
Fair enough, in general. No doubt the articles we've written about the Ohio '05 election results, as well as the one from Fitrakis and Wasserman at Free Press, tend to presume that the election results are suspect, more so than the polling or its methodology.
The difference, however, is that while we welcome the search for reasons why the Dispatch's historically accurate poll might suddenly have failed for the first time in such an unprecedented matter, we have as much or more reason to be skeptical of Ohio's Electoral System which has been documented as full of unquestionable corruption and all-sorts of top-to-bottom malfeasance. Particularly over the last year or so since the Presidential Election.
In other words: Sure, we can, and should, go looking for reasons why the Dispatch poll might have failed so mightily for the first time in its history, as others (including Blumenthal) did with the Exit Polls from November '04 which also --- just a coincidence, we know --- failed so hugely for the first time in their history. We can likely find anomalies to somewhat explain what might justify a pre-determined conclusion that Election Results are right and Pre-Election Polling must be wrong. But why would Blumenthal, or anybody else, have reason to believe that scientifically run polls with methodologies developed and honed over decades, run by pollsters with (theoretically) no dog in the hunt, should be less reliable than an Electoral System being run on newly developed, newly introduced technology employing secret software that nobody inspects for validity or integrity? All of which is run on machines which have a tremendous and documented known failure rate and are run by partisan officials with not just a stake in the outcome, but whose very livelihoods (in the case of the recent Ohio initiatives, as well as last November's election) are at stake in the final results. ...