A few days ago I mentioned a research paper about errror rates in vote counting that could conceivably shed light on the recount fiasco here in Washington. The paper is called "Using Recounts to Measure the Accuracy of Vote Tabulations: Evidence from New Hampshire Elections 1946-2002". Pro-Gregoire bloggers used this article to argue that there is so much uncertainty in the vote counting that Dino Rossi's two wins were worth nothing more than two coin tosses coming up heads. (Their conclusion was that we might as well flip a coin a third time and let the outcome of the third toss determine the winner). I soundly dismissed those arguments here. I have since also e-mailed the article's co-author, Prof. Stephen Ansolabehere of M.I.T. He has graciously answered my questions to clarify his findings and their relevance to our situation here in WA state. I share his comments in the extended entry.

The punchline is that my original review of the paper and the application of its findings to the WA governor's race were on-target, with only one minor numerical correction. My conclusion holds: The article provides no basis to declare that the first two vote counts were inconclusive as the Gregoire supporters wish to believe.

As I noted in my earlier entry, the following points from the paper were cited:
A) The "residual rate" (meaning ballots that are not counted because they are blank or improperly marked) is 1% - 2%
B) The "tabulation error rate", meaning the discrepancy between an initial vote count and the first recount is 0.56% (for optical scan ballots).

(I suspect that this paper was the source of the Democrats's discredited claim that "voting machines have a 1 - 2% error rate", which was since explained by the Secretary of State's office to be a rate of human error in filling out ballots).

In addition, others who have read this paper have concluded (incorrectly) that:
1) 0.5% of the ballots cast in our governor's race were tabulated incorrectly
therefore
2) there must still be 14,000 erroneous ballots outstanding in the gubernatorial race
therefore
3) Rossi's 42 vote lead is "within the margin of error" and therefore the governor's race is a "statistical tie"

Again, the paper looks at vote counting only in New Hampshire. My own analysis of the WA gubernatorial count, after double-checking my methodology with Prof. Ansolabehere:

* The "residual rate" of unmarked and improperly marked ballots here was well below 1%; In King County it was 0.39%. It's fair to assume that most were intentionally left blank. There is no evidence that more than 0.08% of (King County) ballots in any race were attributable to manually correctable human error; i.e. where the voter marked the ballot in a way that was deemed to have expressed clear intent in a way that was readable by a human, but not by the machine.

* The tabulation error rate statewide was not 0.56%, but 0.09%. (In my original analysis I misunderestood the formula used in the article and incorrectly calculated the error rate to be 0.0040%. This does not materially affect my conclusions)

The paper seems to assume, but does not state explicitly, that a recount is inherently more accurate than an initial count. I asked Prof. Ansolabehere "what evidence is there that a recount is more accurate than an initial count?". His reply:

New Hampshire occasionally has a second recount (a third count). These are almost always identical to the recounts (differing by only 1 or 2 votes). So, I take the recounts as an accurate measure of the actual valid vote.

Not sure what explains the difference. Washington only did a "machine" recount so far, and that might explain some of it. New Hampshire does a complete manual recount.

Another possible difference between NH and WA is the level of aggregation. THe NH data are all town level. All of the WA data that I've seen so far are county level. The ballots swing one way or another. Sometimes a candidate loses a couple of votes in one town and gains a couple in another town. So, at the county level the net effect may be slight.

If you are interested, Minnesota has excellent reports of their recounts, including some details about how specific ballots are resolved. Minnesota is much more like Washington than New Hampshire is (e.g., large cities, mix of punch cards and scanners, sizable absentee vote, etc.)

Some will undoubtedly convince themselves that one thoughtful professor's "Not sure" and speculative comments validate their belief that a manual recount will give us a bullet-proof answer. Others will remind themselves of the following observations that suggest there are some significant, if as yet unknown, differences between vote counts in New Hampshire and vote counts in Washington:

a) the residual rate in Washington was much smaller than what was observed in New Hampshire, and very little of our residual vote has been established to have been caused by voters recording their intent improperly.

b) As we've learned from our Secretary of State, a machine count of optical scan ballots has an error rate of 1 in 1,000,000. (0.0001%). It is not realistic to assume that a hand recount will be more reliable than this. Indeed, we are not told the size of the New Hampshire jurisdictions that had second recounts with "1 or 2" vote discrepancies from the first recount. If those were in, say, 1,000 voter towns, that would be a 0.2% error rate.

c) In the large Washington county that did a partial hand recount to correct for voter error, King County, the tabulation error rate was 0.11% -- 0.08% attributable to acceptance of previously discarded ballots and 0.03% due to the 336 magical mystery ballots that somehow appeared after the first certification and which the superintendent of elections cannot explain.

c) Many Washington counties use technologies that were not used in the scope of the New Hampshire study: punch cards and touch screens. This paper offers no guidance on error rates with those technologies.

The conclusion: Those who sleep better at night believing that there are 14,000 erroneously counted ballots out there (0.5% of all ballots), will continue to believe that there are 14,000 erroneously counted ballots out there, even though the research papers they cite to support their claims are not, in fact, consistent with the results we've seen in WA state. The rest of us will commend Prof. Ansolabehere for making progress towards understanding the mechanics of vote counting and recounting, while recognizing that there are still a large number of open questions remaining to be answered.

In a later post, I will put to rest the canards about "ties" and "statistical ties" and the marginal use of the phrase "margin of error".