The oft-cited pollster Mark Blumenthal, who writes about all matters related to polling and its methodology on his Mystery Pollster blog, has posted the results of his analysis/investigation into what he feels may have gone awry in the Columbus Dispatch polling just prior to the November 8th election in Ohio.
We've previously reported on the final results of the five initiatives on the ballot in the Buckeye State two weeks ago - four of them part of an Election Reform package --- and the "staggeringly impossible" disparities, as we described them, revealed by those results when compared to the results of the Dispatch's final pre-election polling. To catch anybody up who hasn't seen the numbers, here's one set of them:
ISSUE 3 (Revise campaign contribution limits)
PRE-POLLING: 61% Yes, 25% No, 14% Undecided
FINAL RESULT: 33% Yes, 66% No
Blumenthal deserves kudos for his serious and detailed investigation into both the Dispatch survey's "long and remarkable history of success," as he describes it, and where he feels they might have gone so astoundingly wrong this year for the very first time. He points to a few changes in the methodology for the poll; the Dispatch didn't use the precise text of the initiatives in its survey; included an "Undecided" option for the first time; an examination of the data seems to show the poll occurred at a time that the electorate was in the process of changing its opinions on the issues; and finally that many voters found the text of the actual ballot initiatives to be long and confusing leading them to take the safer vote of "No" when they were not clear about what they were actually voting for.
Where he doesn't deserve credit is in his approach which presumes from the start that there must be something wrong with the polling itself --- but not the Election Results --- to explain the unprecedented disparity between the polling and the final results of the actual election.
As a pollster himself, he surely knows that it's quite simple to construct a poll in such a way that one can get back results which match just about any pre-determined outcome an unscrupulous pollster may hope for. Similarly, as with Blumenthal's earlier efforts to examine the disparity of Exit Poll Results vs. Final Election Results in last November's Presidential Election, his approach presumes from the get-go that the polling, not the results, are incorrect...and now we just need to figure out why.
Fair enough, in general. No doubt the articles we've written about the Ohio '05 election results, as well as the one from Fitrakis and Wasserman at Free Press, tend to presume that the election results are suspect, more so than the polling or its methodology.
The difference, however, is that while we welcome the search for reasons why the Dispatch's historically accurate poll might suddenly have failed for the first time in such an unprecedented matter, we have as much or more reason to be skeptical of Ohio's Electoral System which has been documented as full of unquestionable corruption and all-sorts of top-to-bottom malfeasance. Particularly over the last year or so since the Presidential Election.
In other words: Sure, we can, and should, go looking for reasons why the Dispatch poll might have failed so mightily for the first time in its history, as others (including Blumenthal) did with the Exit Polls from November '04 which also --- just a coincidence, we know --- failed so hugely for the first time in their history. We can likely find anomalies to somewhat explain what might justify a pre-determined conclusion that Election Results are right and Pre-Election Polling must be wrong. But why would Blumenthal, or anybody else, have reason to believe that scientifically run polls with methodologies developed and honed over decades, run by pollsters with (theoretically) no dog in the hunt, should be less reliable than an Electoral System being run on newly developed, newly introduced technology employing secret software that nobody inspects for validity or integrity? All of which is run on machines which have a tremendous and documented known failure rate and are run by partisan officials with not just a stake in the outcome, but whose very livelihoods (in the case of the recent Ohio initiatives, as well as last November's election) are at stake in the final results. ...
We'll mention again here that 44 of Ohio's 88 counties used all-new Electronic Voting Machines for the very first time in their recent election, and that virtually all of Ohio uses either demonstrably hackable and unsecured Electronic Voting Machines and/or Electronic Central Tabulating Machines to carry out their elections. A branch of the U.S. Dept. of Homeland Security warned the machines are hackable prior to the '04 Presidential Election, and a non-partisan GAO report confirmed as much within the past month.
Blumenthal's analysis of the Dispatch poll, which is very thorough, and we encourage you to read it for yourself, takes pains to examine the minutiae and detailed methodology of the way the Dispatch numbers were gathered. But no such effort is made to look at all into the way that J. Kenneth Blackwell (Ohio's highly partisan Secretary of State, whose job was at stake in the election) gathered his numbers.
That is not Blumenthal's fault, per se. He is, after all, an expert in polling and not election systems, equipment or management. And indeed, he gives a nod at the end of his analysis to the fraud theory that Fitrakis and Wasserman strongly point towards in their article and that we did as well in ours (those less by direct accusation, and more by implication). But the "Fraud" theory is clearly an afterthought for Blumenthal. Or, at least, offered for brief discussion only once he finds whatever possible anomalies he can in the Dispatch's polling methodology.
But Blumenthal himself admits throughout his analysis that even his compelling possible case for the "reasons why the Dispatch poll could have been wrong," [emphasis ours] to use his own words, the case to explain the enormous disparity with the Final Election Results is difficult to justify. Despite the various tweaks in methodology that the Dispatch used this year, Blumenthal acknowledges at various times that "things were much different in this election," and "Unfortunately, we may never know definitively" the reasons for that disparity.
We take only one major exception to the way Blumenthal characterized our article, when he says:
While wouldn't presume to speak for Fitrakis and Wasserman, we don't believe we've ever suggested anything of the kind. Indeed, we discussed margin of error in the poll and its confidence rate. Our point, however, is that when you've got two different suspects for one murder case, one suspect with a long rap-sheet filled with all manner of homicides, theft, and general malfeasance, and the other is a Priest, an Eagle Scout, and a 4-Star General, we'd take a good long hard look at the known Criminal before presuming that the Saint must have done it. Even while either of them could have done it, Blumenthal presumes, without reason, that the Priest was likely the guilty party.
And where he finally throws a reluctant bone towards the possibility that the ex-con with a history of murder just might have killed again (he offers the legal speculation that the "paper 'tab' printout receipts" from the electronic voting machines might be accessible via public records requests after "the official count is complete and certified") he misses the biggest point of all:
If the polling of human beings for accurate information about their opinions has become such a tediously documented "science" over so many years of development, to the point where the smallest changes in methodology can effect the entire outcome, and to the point where an examination of any such changes are actually possible, and also revealing, how is it that he doesn't recognize the enormous sea-change in results that can occur in actual elections when the entire methodology used for decades is thrown out in favor of untested new methods which cannot be verified, audited, or examined because the entire methodology for gathering that data is a "proprietary trade secret" and the data itself is never made available for complete public inspection?!
All the while, as Blumenthal has ample reason to be dubious of any data gathered via Pre-Election Polling, he seems to fail to recognize entirely that with elections in a democracy the mere introduction of myriad new reasons to be dubious about the results is, in and of itself, a threat to the very core of our way of self-selected governance. As a pollster himself, Blumenthal should recognize that fact as much as anybody and join those of us who give a damn about that democracy, by calling for an end to the tsunami of unverifiable, unaccountable, untransparent Electronic Voting Machines run by secret software which is now flooding into our Electoral System in this country with virtually zero oversight or checks and balances.
We suspect Blumenthal wouldn't put much stock in a similarly "faith-based" polling methodology where pollsters tell him "trust us, the methodology we use is very very good, you don't need to actually examine any of it to have complete confidence in our results." So we'd hope that he --- as much as anyone --- would understand that the results of an election where the methodology is allowed to be held as a secret "proprietary trade secret," should be similarly suspect and taken for what it's worth. Which is, as far as we can tell in America 2005, worth very little and decreasingly so every damned day.
|
|
||||
|
Share this article with the world...
                        |
|||||
