October 28, 2006
The Reliability of Polling

This is a modification of a peice that Hugh Hewitt posted on his blog 10/18, and that the WSRP has included in their newsletter:

As someone who has worked his entire adult life, 25 years, processing market research and public opinion surveys, I know there are ways to make subtle changes to affect the outcome of polls. I know enough about surveys to be able to construct one that shows people prefer Pepsi over Coke, 60%-40%, or vice versa, and you would have no idea how I got either result even if I gave you the internals and methodology.

I have increasingly become skeptical that political polls being used in this country are fair and unbiased. You don't have to take it on faith that there is a media conspiracy to misrepresent polling, they admit it in their own poll results. I have yet to see a general population poll that did not show adults 2%-4% more liberal than registered voters and registered voters 2%-4% more liberal than likely voters, and yet the media has no qualms about citing adults or registered voters when they want to give the Democrats an added boost. Likewise, most 7 day tracking studies show a liberal 2%-4% bias when collecting over the weekend, yet the weekend seems to be the favorite time to poll.

It is becoming obvious that the media is driving the results in most of the polls. For example, despite economic conditions that can be described (at worst) as one of the best economies we've ever had, all the polls give Bush bad ratings on the economy. The relentlessly negative coverage the media gives the economy can convince the public of complete untruths (ie. A 2004 survey reported 51% believed the economy had lost jobs, despite the government reporting 1 million jobs had been gained). The media seems to have developed their own version of "push-polls", reporting a storyline, then polling to see how widely their storyline is accepted, then reporting the poll to reinforce the storyline. In my opinion, the entire "Democratic blue wave" media story is based on polls showing people think there will be a "Democratic blue wave", because they've read media stories telling them so. The media coverage drives the polls, which then drives the media coverage, which then drives the polls, eventually becoming a "KnownFact".

But the most important flaw is the fact that every poll seems to over sample Democrats by 5%-10% consistently. This has to do with the identification of "likely" voters, as well as the challenges of getting a representative sample in our age of changing telecommunications. The common error is to assume that their rigorous random sampling methodology truly gives them a representative sample of actual voters. The "sampling error" they give only tells you how big their sample is, it is meaningless if the sample is not representative of voters in November.

The proof of their inaccuracy is in the exit polls. They are taken of a sample of actual voters, and can be verified against the actual voting results. The party affiliation was even in 2004, and +2% for the Republicans in 2002, and this seems pretty correct considering the election results (though given the vote counts it may still undercount Republicans by 1% or 2%). By reviewing the accuracy of their predictions, it is obvious that, for a variety of reasons, the likely voters they talk to in October contain more Democrats than actually cast a vote in November. This was the case in 2004, when three polls taken in late September to late October showed Dino Rossi down by 5,7, and 6 points, only for the race to end in a tie.

The Democrats and their accomplices in the media have made their strategy clear, they are not offering better policies or alternate plans, they are running against President Bush. They believe they will win elections not because more people will vote for them, but because Republicans are dispirited and discouraged and fewer will vote Republican. One method to discourage Republicans is to produce polls that tell us we cannot win. The media spreads the Democrat's message that we cannot win, the polls with faulty samples confirm it, and the media promotes those polls.

In regards to Mike's campaign, there are some disquieting facts that make it difficult to trust any of the polling being done. From 10/13 to 10/19, there were 5 polls done by 5 companies. They had Mike trailing by 8,7,15,9 and 15 points. Websites like RealClearPolitics average them, assuming the outliers will cancel each other out. But two groups of studies can't have a 6-8 point difference in the same week and both be trustworthy. If all the polls were done correctly with the same methodology, statistically you shouldn't see this variance. Since we know the five companies didn't use the same methodology, logically we can assume at most one group is more or less correct, and either Mike is down ~8 points or 15 points, but not by the average of the five polls. Pollsters pretend that these fluctuations reflect changes in public opinion, but it is inconceivable that the public is so fickle that 10% change their mind back and forth routinely with no apparent reason. For instance, Rassmussen (using the same methodology) did three polls 9/6, 9/20 and 10/17 that had Mike down 17, 6 and 15 points. Are we to believe Mike gained 11 points in two weeks, then lost 9 in the next 4 weeks? How much confidence can you have in polls that bounce back and forth?

Look at historical election results, does a state that gave Slade Gorton half the vote against Maria Cantwell in 2000 and gave Dino Rossi half the vote against Christine Gregoire in 2004, vote down Mike McGavick by 15 points? Mike will not lose by 15 points, the polls that put him down that much are only reflections of the liberal media's desire to make it so. Does that mean the rest of the polls showing Mike down by 5%-9% are more accurate? Yes, although history tells us that there is a good chance they have an institutional bias of 5-7 points. It would be comforting to have polls come out showing Mike in the lead, but if he wins in November with 51% of the vote, I would not expect a single poll, not even on election eve, to show him better than 3 or 4 points down.

This biased polling can have a beneficial effect to Mike's campaign, Democrats and Independents assured of Cantwell's victory will be less likely to vote. In addition to the media bias inherent in the polling, they also do not pick up the gains from an ever-improving GOTV ground game. "Unlikely" Republicans convinced to vote are not reflected in polls. Republicans are serious adult citizens, Democrats and Independents are not. We take our citizenship seriously, inform ourselves and vote accordingly. The rest of the population often doesn't even register to vote, or only vote when they are sufficiently ginned up, or when they are herded on busses to the polls. To quote Senator Santorum, "Democrats win polls, Republicans win elections."

Overall, I place less and less trust in polls, because I see them becoming more unbelievable, more inaccurate and more obviously instruments of a liberal media to drive public opinion. I have more confidence in seat-of-your-pants political common sense projections than I do in the polls I have seen, and those projections by-and-large have Mike running close. These polls are designed to discourage Republicans, and it is important that we not let these polls reduce our enthusiasm or deflate our turnout.

Posted by brettmalin at October 28, 2006 03:52 PM | Email This
Comments
1. You can't compare exit polls with standard polling, given the circumstances under which they are taken. Statistical analysis of the 2004 polls showed the major polling companies were pretty accurate, but sometimes overestimated the Democratic vote and sometimes the GOP.

You used Rossi as an example. On 11/1/04, Survey USA had him up by 6 while StrategicVision had him down by three (within the margin of error).

In the 15 Senate races from 2004 listed on SurveyUSA's website, SurveyUSA actually missed by an average of 2% in favor of the GOP. Strategic Vision was a little more tilted to the GOP in the five races listed. I don't think these results bear out your hypothesis, at least in 2004. Of course, that doesn't mean the results will be similar this year, but we will know in about 10 days.

Posted by: wayne on October 29, 2006 12:58 AM
2. Lies, damned lies, and statistics. Polls are essentially worthless and data can be manipulated (analyzed) to the point of outright fraud.

Ignore it all and see what happens on election day.

Posted by: H Moul on October 31, 2006 12:26 PM
3. I've done polling. I've done door-to-door, which I think is the best kind, to phone polling. I've hired reputable pollsters. I've hired crooks. Both are available.

The best polls are never released. They're given to the money guys, but that's all. Mike's raised a lot of money in the last month. You do the inference.

Posted by: OregonGuy on October 31, 2006 09:00 PM
Post a comment
Name:


Email Address:


URL:


Comments:


Remember info?