Seriously. What’s up with the polls? I’ve never seen an election where respected pollsters disagreed so wildly. Gallup today has Barack Obama up 11 over John McCain. But Zogby, on the other hand, has Obama up two. Hotline has Obama up just one. You’re talking about a ten point difference. That’s a little bit out of the margin of error, wouldn’t you say?
I’ve spent a decent portion of my time the last few years diving into the nuts and bolts of polls for campaigns. And I’m wondering how respected pollsters could be seeing this race so differently.
One issue that’s been apparent to me in this cycle is that pollsters are asking questions of far more Democrats than Republicans. That’s not necessarily a bad thing, however. But it is certainly subjective and fluctuating.
Here’s leading pollster Scott Rasmussen on his most recent poll: “For polling data released during the week of October 5-11, 2008, the partisan weighting targets used by Rasmussen Reports will be 39.3% Democratic, 33.3% Republican, and 27.4% unaffiliated.”
That’s six percent more Democrats than Republicans that Rasmussen polled. It’s hardly a surprise then that Rasmussen has Obama up six.
Now, Hotline has a party ID Breakdown of 41%D, 36%R, 18%I. That’s only a difference of five. Not surprisingly, Hotline has a closer race. They have Obama up by only one.
According to Rasmussen, they set partisan affiliation weighting targets based upon survey results obtained during the previous three months. These shift only slightly month-to-month, but they say the change could be significant over a long period of time.
For example, in 2006, Rasmussen had party affiliation set at 36.6% Democrat, 33.5% Republican, and 29.9% unaffiliated. But now, just two years later, the partisan weighting targets used by Rasmussen Reports are 39.3% Democratic, 33.3% Republican, and 27.4% unaffiliated.”
Scott Rasmussen argues, that as party affiliation is somewhat stable, it can and should be used to stabilize polls. He asked recently, “how many people do you know who consider themselves a Republican one day and a Democrat the next?” Very few, I would think.
Polling organizations do adjust their figures for race, income level, gender and other variables so it makes sense for them to adjust for party as well. However, it seems clear to me that this is an art and not a science.
So while weighting seems like a good idea it can also be a dangerous one. For example, according to The Politico, “Democrats picked up 31 seats in 2006 with an 11.5-point lead in generic ballot tests, and the party actually won 53.6% of the general election vote, a 7-point margin.” So to put it plainly there was an 11.5 lead in the generic ballot polls but only a seven point margin on election day.
Now, an oddity is that just last month a USA Today/Gallup survey of the generic ballot had the Democrats leading the Republicans by just 3 percentage points, 48% to 45%, in voters’ “generic ballot” preferences for Congress. Yet, some pollsters seem to still be polling more Democrats than Republicans at a difference of 5% or higher.
And get this, Gallup, in a poll done less than a month ago Republicans were more enthused about the 2008 election and outscoring Democrats in likelihood to vote in November. As a result, Republican candidates led Democratic candidates among likely voters by 5 percentage points, 50% to 45%.
So there were about 3% less Republicans than Democrats but they were more likely to vote by 5%. Seems almost a wash, right? But for some reason pollsters are still giving heavy preference to Democrats in their polls. Why? I’m not sure.
Gallup, on the other hand, does not weight for party. And they’re the ones right now with Obama up 11. So, it seems much of the discrepancy in the polls is based on who they’re speaking with.
Historically, according to Pollster.com, during presidential years over the last five presidential elections, the biggest party ID gap was four points. But right now, we’re looking at larger margins than that in many of the polls. Is this year that anomalous? Maybe? You might argue that this is a very very bad year to be a Republican.
But in 2006, which was also a very very bad year for Republicans, the Democratic advantage in party ID on Election Day was only three points (38-35). But polls leading up to the election showed party ID gaps as big as eleven points. So clearly, the polled party identification was vastly overestimated.
This kind of thing happens frequently. A CBS poll just this August showed an unweighted sample of 317 Republicans and 381 Democrats but after they weighted it, they made it 284 Republicans and 406 Democrats which drastically changed the outcome of the poll, as you could imagine.
A recent LA Times poll during the summer infamously had a 39% Democrats and 22% Republicans weight and unsurprisingly it had Obama way ahead.
Look, I’m sure Obama’s ahead right now. But I really don’t think it’s by 11 as Gallup would have us believe. With no weighting, it would seem that Gallup would be prone to wide fluctuations.
I think one other issue in this race is a possible “Bradley effect.” The media believes its interchangeable with closet racism when, I believe, it occurs when voters don’t want to be viewed as racist so they don’t announce their true preference in a race including an African-American candidate. But many pollsters seem to be ignoring this because they believe that a large turnout in African-American voters will offset any Bradley effect. But they don’t know. It’s not a question you can poll. “Are you just saying you’re for Obama because you fear being called a racist? Come on. Tell me the truth.”
One thing to remember is that Obama built a big early lead in the primaries only to see a majority of “undecided” voters break late for Clinton. Remember New Hampshire polled very well for Obama but broke late for Clinton. I think that’s the Bradley effect there. And I wouldn’t be surprised to see it happen again. The media will say it’s racism but I would say that in this time when Sarah Palin, Bill Clinton, Hillary Clinton and anyone who says a word against Obama is called a racist, people are slightly unwilling to say they’re not voting for Obama.
So my advice to you is to not take the polls at face value. Look into the numbers. There are wide methodological differences between these pollsters and wide differences in polls as you can see. Let’s face it. Only one poll counts. And please make sure you’re counted that day.
October 9, 2008 at 4:34 am
Good points all. One thing I also noticed was that on Fox News they actually listed in smaller print below how many people had been called, and a lot of times it was under 600. Now, I know bugging thousands of people every day is neither enjoyable nor feasible, but 600 in a nation of what, 270 mil., is hardly a representative sample, and some of the margins are ±4 or even 5%. The polls are misleading… but what’s new about that, eh?
October 9, 2008 at 4:55 am
I just hope the wide margins for Obama don’t dispirit pro-lifers from coming out on E-Day.
October 9, 2008 at 5:28 am
Down here, we have “compulsory voting” – you get fined $20 if you don’t at least rock up and collect a ballot. You can spoil your ballot; write obscenities on it, whatever, but you have to collect it from a polling official, or do a postal vote, or an early vote, or whatever.
What sometimes confuses me with US polls is the whole thing about polling “likely voters” and “intending voters”, or “bums who might vote if the Dems buy them some liquor and drive them to a polling booth”…
What is the most accurate way of factoring in “turnout” in polling? We never bother about it here, because just about everyone turns out rather than pay their $20.00 fine….
I shudder to think of the consequences of Obamanation Kidkill and Joey Hairplugs winning in November….
October 9, 2008 at 5:28 am
Down here, we have “compulsory voting” – you get fined $20 if you don’t at least rock up and collect a ballot. You can spoil your ballot; write obscenities on it, whatever, but you have to collect it from a polling official, or do a postal vote, or an early vote, or whatever.
What sometimes confuses me with US polls is the whole thing about polling “likely voters” and “intending voters”, or “bums who might vote if the Dems buy them some liquor and drive them to a polling booth”…
What is the most accurate way of factoring in “turnout” in polling? We never bother about it here, because just about everyone turns out rather than pay their $20.00 fine….
I shudder to think of the consequences of Obamanation Kidkill and Joey Hairplugs winning in November….
October 9, 2008 at 6:38 am
David, I had never heard of compulsory voting in my life and was actually appalled when I read about it. Forcing someone to vote takes away that person’s right to freedom of boycott or active refusal to participate in the system if he/she thinks it is illigitimate. I’m sincerely at a loss.
Anyway, back to the topic at hand, there is a saying attributed to Benjamin d’Israeli, Winston Churchill and a host of others: “There are lies, damned lies, and then there are statistics.” And that’s all polling is. We saw in 2000 that polls meant nothing. They still don’t. You can lick your finger and put it in the air, but that just won’t tell you when and where the huricane will hit. You’ll know when it gets there. Same with the election results.
I will say this: the mainstream media seems intent on portraying Obama as the clear winner, regardless of the margin. That much is obvious. To what extent that will mobilise or dishearten the opposition remains to be seen.
October 9, 2008 at 11:07 am
I couldn’t agree more – it’s frustrating to watch that the pollsters disagree on who’s ahead in the election campaign.
Luckily, I found a widget that uses poll data from various pollsters.
I think this makes the result more nuanced.
you are more than welcome to check it out – you might like it:-)
It gives a great overview and it is updated as the polls come in!
http://www.youcalc.com/apps/1221747067033
… and its easy to put on your blog and fits in your sidebar!
Make a difference, keep on voting!
October 9, 2008 at 12:09 pm
I think Republicans are by nature a little more respectful of (ahem!) diversity, less willing to wear their thoughts as bumper stickers and slogans and tee-shirts. Republicans are also more “Mind your own da(r)n business” to snoops, including pollsters. And, more alarmingly, perhaps people are afraid of discrimination and even violence if they are open in their fear and disapproval of the Ceaucescu-esque Family Obama.
— Mack, evil, wicked, godless, satanic, blah-blah-blah public-school teacher
October 9, 2008 at 12:09 pm
Another factor to consider is whether the poll is asking “registered voters” vs “likely voters”. Is it safe to say that “likely voters” polls are generally more accurate because these are the folks most likely to go out and vote?
I think there will be a “Bradley effect” this year, and we can’t rule out the “Clinton effect” either. Hillary wants to be President soooooo bad, and if Brocco wins, her next viable chance is 2016. I wouldn’t put it past the Clinonites to try and subvert his election chances just to enhance hers in 2012.
October 9, 2008 at 1:31 pm
Another thing to consider. Many Republicans registed Democrat during the Clinton/Obama battle to try and sway the outcome. I don’t know if they are using official registered votes to determine those percentages but I suspect in the short term the D side might be inflated.
In fact, my wife registered D recently so we could vote in future primaries. The way we figure that gives us one vote in R primaries and one vote in D primaries and since pro-life Democrats DO run around here, we want them to have our support.
October 9, 2008 at 4:12 pm
Many Republicans registed Democrat during the Clinton/Obama battle to try and sway the outcome.
I think it’s fair to wonder whether many Democrats registered Republican during the GOP primaries as well. I simply cannot see how McCain could have won the nomination otherwise.
October 9, 2008 at 6:39 pm
But for some reason pollsters are still giving heavy preference to Democrats in their polls. Why?
It’s psychological warfare. They are trying to demotivate the R’s from turning out on election day. Don’t give up!!
October 9, 2008 at 6:52 pm
Well, I already voted (absentee) so the deed is done.
I voted for Sarah!
October 9, 2008 at 7:22 pm
Polls are just that – polls. We will not know the true outcome until the election and we all know polls are imperfect. Maybe there is a “Bradly effect” and maybe it is the specific sample collected. Yes – two separate polls could come up with very different numbers depending on their sampling methodology.
I’m part of that younger generation with only a cell phone, so no one is polling me. I think polls tell us something, but not as much as the media likes us to believe they tell us. I think they are interesting but it’s not like we decide the winner based on polls – so who cares??
October 9, 2008 at 8:38 pm
Don’t give up!!
Unless you’re in Michigan, in which case the Straight Talk Express has already given up on you. 😉
October 10, 2008 at 2:11 am
National polls are mildly interesting, but don’t matter. The election is won by getting electoral college votes. That means the few states where there is a close enough race and enough electoral votes will determine who wins.
Nothing else.
October 12, 2008 at 3:19 am
The purists say that Australia has compulsory attendance at the voting place, which is almost, but not quite, compulsory voting.
No one is compelled to vote, but they risk a fine if they don’t turn up and have their name marked off.
And in other news, Nate did the numbers at 538 and dismisses the Bradley Effect.