What’s Up With the Polls?

Seriously. What’s up with the polls? I’ve never seen an election where respected pollsters disagreed so wildly. Gallup today has Barack Obama up 11 over John McCain. But Zogby, on the other hand, has Obama up two. Hotline has Obama up just one. You’re talking about a ten point difference. That’s a little bit out of the margin of error, wouldn’t you say?

I’ve spent a decent portion of my time the last few years diving into the nuts and bolts of polls for campaigns. And I’m wondering how respected pollsters could be seeing this race so differently.

One issue that’s been apparent to me in this cycle is that pollsters are asking questions of far more Democrats than Republicans. That’s not necessarily a bad thing, however. But it is certainly subjective and fluctuating.

Here’s leading pollster Scott Rasmussen on his most recent poll: “For polling data released during the week of October 5-11, 2008, the partisan weighting targets used by Rasmussen Reports will be 39.3% Democratic, 33.3% Republican, and 27.4% unaffiliated.”

That’s six percent more Democrats than Republicans that Rasmussen polled. It’s hardly a surprise then that Rasmussen has Obama up six.

Now, Hotline has a party ID Breakdown of 41%D, 36%R, 18%I. That’s only a difference of five. Not surprisingly, Hotline has a closer race. They have Obama up by only one.

According to Rasmussen, they set partisan affiliation weighting targets based upon survey results obtained during the previous three months. These shift only slightly month-to-month, but they say the change could be significant over a long period of time.

For example, in 2006, Rasmussen had party affiliation set at 36.6% Democrat, 33.5% Republican, and 29.9% unaffiliated. But now, just two years later, the partisan weighting targets used by Rasmussen Reports are 39.3% Democratic, 33.3% Republican, and 27.4% unaffiliated.”

Scott Rasmussen argues, that as party affiliation is somewhat stable, it can and should be used to stabilize polls. He asked recently, “how many people do you know who consider themselves a Republican one day and a Democrat the next?” Very few, I would think.

Polling organizations do adjust their figures for race, income level, gender and other variables so it makes sense for them to adjust for party as well. However, it seems clear to me that this is an art and not a science.

So while weighting seems like a good idea it can also be a dangerous one. For example, according to The Politico, “Democrats picked up 31 seats in 2006 with an 11.5-point lead in generic ballot tests, and the party actually won 53.6% of the general election vote, a 7-point margin.” So to put it plainly there was an 11.5 lead in the generic ballot polls but only a seven point margin on election day.

Now, an oddity is that just last month a USA Today/Gallup survey of the generic ballot had the Democrats leading the Republicans by just 3 percentage points, 48% to 45%, in voters’ “generic ballot” preferences for Congress. Yet, some pollsters seem to still be polling more Democrats than Republicans at a difference of 5% or higher.

And get this, Gallup, in a poll done less than a month ago Republicans were more enthused about the 2008 election and outscoring Democrats in likelihood to vote in November. As a result, Republican candidates led Democratic candidates among likely voters by 5 percentage points, 50% to 45%.

So there were about 3% less Republicans than Democrats but they were more likely to vote by 5%. Seems almost a wash, right? But for some reason pollsters are still giving heavy preference to Democrats in their polls. Why? I’m not sure.

Gallup, on the other hand, does not weight for party. And they’re the ones right now with Obama up 11. So, it seems much of the discrepancy in the polls is based on who they’re speaking with.

Historically, according to Pollster.com, during presidential years over the last five presidential elections, the biggest party ID gap was four points. But right now, we’re looking at larger margins than that in many of the polls. Is this year that anomalous? Maybe? You might argue that this is a very very bad year to be a Republican.

But in 2006, which was also a very very bad year for Republicans, the Democratic advantage in party ID on Election Day was only three points (38-35). But polls leading up to the election showed party ID gaps as big as eleven points. So clearly, the polled party identification was vastly overestimated.

This kind of thing happens frequently. A CBS poll just this August showed an unweighted sample of 317 Republicans and 381 Democrats but after they weighted it, they made it 284 Republicans and 406 Democrats which drastically changed the outcome of the poll, as you could imagine.

A recent LA Times poll during the summer infamously had a 39% Democrats and 22% Republicans weight and unsurprisingly it had Obama way ahead.

Look, I’m sure Obama’s ahead right now. But I really don’t think it’s by 11 as Gallup would have us believe. With no weighting, it would seem that Gallup would be prone to wide fluctuations.

I think one other issue in this race is a possible “Bradley effect.” The media believes its interchangeable with closet racism when, I believe, it occurs when voters don’t want to be viewed as racist so they don’t announce their true preference in a race including an African-American candidate. But many pollsters seem to be ignoring this because they believe that a large turnout in African-American voters will offset any Bradley effect. But they don’t know. It’s not a question you can poll. “Are you just saying you’re for Obama because you fear being called a racist? Come on. Tell me the truth.”

One thing to remember is that Obama built a big early lead in the primaries only to see a majority of “undecided” voters break late for Clinton. Remember New Hampshire polled very well for Obama but broke late for Clinton. I think that’s the Bradley effect there. And I wouldn’t be surprised to see it happen again. The media will say it’s racism but I would say that in this time when Sarah Palin, Bill Clinton, Hillary Clinton and anyone who says a word against Obama is called a racist, people are slightly unwilling to say they’re not voting for Obama.

So my advice to you is to not take the polls at face value. Look into the numbers. There are wide methodological differences between these pollsters and wide differences in polls as you can see. Let’s face it. Only one poll counts. And please make sure you’re counted that day.