Why you should(n’t) care about polls
Election junkies (and even casual users) have been keeping their eyes on polls for weeks now. As the election approaches, news outlets are coming out with new polls every day, and sites like Real Clear Politics and Five Thirty Eight are spreading the poll fix. Right now, all the numbers are overwhelmingly in Obama’s favor, so based purely on statistics, this election might as well be over. But just how accurate are those polls?
One of the most memorable fiascoes in the history of fiascoes was the “Dewey Defeats Truman” headline after the 1948 election. Since The Hall of Presidents doesn’t have a creepy animatronic Dewey statue, you know that the headline was incorrect. Based on Gallup’s supposedly infallible exit poll numbers, the editors of the “Chicago Daily Tribune” figured that Dewey would win, so they put up the headline and called it a night. And, of course, we all remember what happened the last time the news jumped the gun with some poll numbers.
The numbers we’re seeing may not be as accurate as they seem. Here’s why:
The Cell Phone Problem (aka nobody calls me anymore)
Land lines are going out of style quickly, a fact that nobody realizes more than students. More and more households are becoming cell-phone only, largely among young people. So why does that matter to pollsters? Because many of them don’t call cell phones.
Recent data shows that about 16 percent of homes are cell-only, a number that could rise by the end of the year. And a good number of those were young people — Pew estimates that 46 percent of the cell-only users were ages 18-29 and another 36 percent were 30-49. Additionally, more minorities were cell-only. Those are groups that have been swinging Obama’s way, especially the young people.
Not surprisingly, excluding cell phones is skewing the polls. Calling cells is expensive and time-consuming because pollsters have to dial the numbers by hand. Plus, it’s so much easier to ignore a pollster if their number shows up on your screen. It’s estimated that leaving out cell numbers reduces Obama’s numbers by about 3 percent and inflates McCain’s by 1 percent. Obviously, that doesn’t change a lot in the grand scheme of these polls, but it still shows how skewed the results can be. Ignoring cell-only users means that pollsters are ignoring a lot of the under-30 set, including those first-time voters that are supposed to control this election.
The Bradley Effect (aka nobody wants to look racist)
News flash: there’s a black guy running for president. And that might be tipping the polls.
One of the unspoken problems in this election is that voters might actually be racist. But nobody wants to seem racist, so when talking to a pollster, people might be more inclined to say they’re voting for Obama, even though they agree with everything on that “New Yorker” cover.
It’s called the Bradley Effect, named after Tom Bradley, a black man who ran for California governor in 1982. Even though polls showed him with a significant lead heading into the election and even exit polls projected him winning, he ended up losing the race to his white opponent. Douglas Wilder experienced a similar dip after exit polls and he ended up winning his race for the 1989 Virginia governorship by less than half a percentage point.
Pollsters (and Democrats) are worrying that Obama might fall victim to the Bradley effect. There is little evidence that the effect has been hurting black candidates in the past 15 years, but none of the races have been as big as the presidency. It’s hard to quantify how much the Bradley effect might be skewing the polls, but it might be enough to negate those enormous leads Obama has been racking up.
The Aggregators Take Over (aka fuzzy math)
This election has seen a big digital transformation, whether it’s McCain hitting Facebook or Obama texting his supporters. But it’s also seen the rise of websites that mash poll numbers together. These aggregators, like Five Thirty Eight and Real Clear Politics, take polls from various sources and average them using fancy formulas to give a more accurate poll. They’ve become increasingly popular among people looking for a quick and easy way to see how the race stands. But are they actually accurate?
Five Thirty Eight has gotten props for its system of weighing polls, which supposedly give a more accurate result than just a simple average. And the founder, Nate Silver, knows his stats — he used to work for Baseball Prospectus, the team that predicted a winning season for the Tampa Bay Rays. These poll aggregators often show different numbers than the mainstream media and are more apt to call tossup races.
But no matter how sophisticated the formula, there’s still the issue that these aggregators are performing science on numbers (that might not be all that accurate in the first place) to come up with new numbers. The numbers haven’t been too far off, but it’s still easy to think that statisticians can play with numbers any way they want. And as these aggregators become more and more of a force in polls, it might be worth taking a second look.
This is all to say that polls shouldn’t dominate the coverage. But while they may be accurate and may show how the race is leaning, they don’t show the whole story. The only way to know for sure is to wait for Election Night.
So if the numbers are right, and Obama does win, what will that mean for McCain's legacy? Or you can return home.
Just to let you know, according to your logic, Barack’s numbers would increase due to the inclusion of cell phone users. You say all these people are in his key demographics and yet somehow you claim his polling numbers would decrease.
ummm
October 28, 2008 at 1:13 am
I don’t say that his numbers would decrease; what I said was that ignoring cell users reduces his numbers by 3 percent. In other words, he would gain around 3 percent from cell users and McCain would lose 1.
Sorry for the confusion!
Jason Plautz
October 28, 2008 at 1:34 am
I check RCP about 12 times a day… sure, there’s a potential for inaccuracy, but aggregators follow the basic rule that a larger sample yields a more realistic picture. I’d guess they’ll be closer to the final popular vote than any individual poll.
Vince
October 28, 2008 at 3:45 am
I know that everyone is talking about it, but can we please put this whole Bradley Effect thing to rest? It’s annoyingly cliche and while it is debatable if it ever existed there hasn’t been a single election in recent years that demonstrates such an effect. The last article I read tried to prove black candidates performed better in the polls than in the election in 2006. The writer completely forgot to mention the fact that those candidates were Republican and 2006 turned out to be a bad year for republicans both white and black. One finds the Bradley effect when they strain to find it. There may be a few people who are so paranoid that a complete stranger will read racism into their response that they lie. Maybe, it strains all reason to imagine such an individual, but I guess it’s possible. Enough to throw off the polls to an even appreciable degree? It’s a laughable position. There are plenty of rational, easily defensible reasons to vote for McCain. All one has to say is they like his policies, the fact that he has more experience, they trust him more, etc. See no racism in any of those responses and would any of those responses be interrogated by the pollster? Would an individual be afraid that racism would be read into such a response that is offered to a complete anonymous stranger? It seems unlikely to say the least. There are many reasons why Obama lost NH during the primaries and his loss was not that far off from the polls (most polls had a pretty close race there). Beyond that Obama underperformed in the polls more often than he overperformed in them during the primaries. Of course no one discusses these primaries because it directly refutes the Bradley Effect. For some odd reason the media loves the idea of the Bradley Effect. They need it to exist as some stock topic of ‘oh he’s doing good right now but what about the Bradley Effect.’ Please, for the love of God can we stop this nonsense? It’s a nonissue invented by Pundits. So no, I don’t think Obama will fall victim to a fictitious effect that is absolutely irrational to believe exists.
Jonathan
October 28, 2008 at 2:25 pm
Just because you don’t understand what exactly the statisticians are doing with the numbers doesn’t make them at all suspect or “fuzzy.” Furthermore, it seems pretty irresponsible to call into question the motives of aggregators such as Mr. Silver without any probable cause whatsoever. Yes, statistics can be made deliberately misleading, but I can’t imagine that someone who’s sole measure of job performance is the accuracy of his predictions about this election would give anything but his best effort in attempting to convey those estimations to the public.
Whatever
October 29, 2008 at 2:36 pm
Quote: Bradley effect, So no, I don’t think Obama will fall victim to a fictitious effect that is absolutely irrational to believe exists.
Actually funny you brought that up… I was harrrassed by so many snide follow-up questions after I said I was voting for McCain to a first pollster, that every pollster who called afterwards I said I was voting for Obama and as far as I knew four more family members would be voting for Obama. That ended the phone call quicker, and I could go back to watching FOX News! I suspect that atleast some of the pollsters were Obamanationists by their remarks, reactions and youth!
HearYea
October 31, 2008 at 1:08 pm
Do your research on the Bradley Effect. Great journalistic skills
http://www.nytimes.com/2008/10/20/opinion/20levin.html?n=Top/Opinion/Editorials%20and%20Op-Ed/Op-Ed/Contributors
Bradley
November 3, 2008 at 3:53 pm