|
|
|
Jewish World Review May 18, 2012/ 26 Iyar, 5772 Poll Results Are 'Massaged' Before They Are Released By Roger Simon
Point One: You already know polls suck. Ask journalists about polls, and they will not only tell you the latest numbers, but about margin of error, weighting and callback polling. They will also say things like: "Have you looked at the crosstabs? Hey, you gotta look at the crosstabs." (This is another reason never to talk to journalists.) Ask real people about polling, however, and they always say: "I've never been called by a poll. Have you ever been called by a poll?" And I reply truthfully: No, I have never been called by a poll, and I don't know anybody else who has been called by a poll. This does not mean polls are made up. But we might be better off if they were. Point Two: I know actual pollsters; I have friends who are actual pollsters, and they are trustworthy, loyal, helpful, friendly, courteous, kind, obedient, cheerful, thrifty, brave, clean and reverent. No, wait, that's the Boy Scout Law. But most of the pollsters I know really are honest and recognize both the power and limits of what they do. They don't go around shouting about the limits, but they recognize them. In this week's new and surprising New York Times/CBS News poll, if you bothered to read the sidebar story headlined "How the Poll Was Conducted" and read all the way to the second to last sentence, you would find this admission: "In addition to sampling error, the practical difficulties of conducting any survey of public opinion may introduce other sources of error into the poll." In other words, "the practical difficulties" of polling may mean the poll sucks. Point Three: We have met the enemy, and he is us. We make too much of polls. Way, way too much. Everybody wants to know the future. Everybody wants to know who is going to win the next presidential election, and why wait for November when the polls can tell us right now? I sometimes quote polls. How can I not? Polls drive media coverage, voter enthusiasm, campaign contributions and turnout on Election Day. And polls do this regardless of the "practical difficulties" that "may introduce other sources of error" into the polling. Point Four: Polls tell you one thing and one thing only. Polls tell you how those polled answered the pollsters' questions. Sort of. Pollsters actually "adjust" their results a little. This is called weighting. And here is what the New York Times story I quoted above had to say about its latest poll: "Overall results have been weighted to adjust for variation in the sample relating to geographic area, sex, race, Hispanic origin, age, education, marital status and number of adults in the household." What does that mean? Here is how weighting is explained by PrecisionPolling.com: "It is frequently the case that the people who answered your poll are not fully representative of the region you were polling over. Weighting is a technique to adjust answers to account for over- and under-represented groups." In other words, pollsters "massage" the results of their own polling. Pollsters do this for the best of reasons — they want to make their polls more accurate — but weighting has obvious perils. The National Council on Public Polls has a detailed and technical explanation of weighting, but here is how it introduces the subject: "One of the reasons polls may differ is the way the results are weighted. It is common practice to weight to demographic characteristics. Most times this reduces the sampling error. That is the point of weighting. Sometimes weighting makes things worse." Oh. Is that all. We don't usually get such frankness when journalists talk and write about poll results, however. Weighting and other polling techniques are often complicated and difficult to convey — and, besides, why ruin a good story with the facts? The New York Times/CBS News polls released Monday not only showed Barack Obama trailing Mitt Romney by 3 percentage points overall but "Romney took the lead among women voters, who supported the former Massachusetts governor 46 percent to Mr. Obama's 44 percent, which is within the poll's margin of error. In April, Mr. Obama had an edge among women voters, with 49 percent support to Romney's 43 percent." In other words, in one month — a month in which some commentators had been saying the Republican Party was waging a war against women — Romney went from 6 percentage points down among women to 2 percentage points up among women. That is quite a swing and a real shocker. It led to a lot of analysis and chatter — the Obama campaign denounced the poll — and Ben Smith, writing on Twitter as @BuzzFeedBen, said: "We're approaching the moment when the CW (conventional wisdom) becomes Romney landslide." I assume Ben was being wry. (And I am not going to say hold the mustard.) But on Wednesday, a Fox News poll was released that not only showed Obama ahead of Romney by 7 percentage points overall, but leading Romney by an incredible 22 percentage points among women. So is Obama solidly ahead or narrowly behind? And is he trailing among women or running away with women (figuratively speaking, of course)? Point Five: Right after the New York Times/CBS News poll, but before the release of the Fox News poll, I tweeted this: "When a poll flies in the face of commonsense, the poll is almost always wrong." I think I'll stick with that.
Every weekday JewishWorldReview.com publishes what many in Washington and in the media consider "must reading." Sign up for the daily JWR update. It's free. Just click here.
© 2009, Creators Syndicate | ||||||||||||
