Hunting for “bias,” armed only with complete and total ignorance

Written by Simon Maloy

Published

Working at NewsBusters must be one of the cushier gigs out there. If a NewsBuster spies a bit of news that he or she doesn't like, all they have to do is write the equivalent of “OMG TEH BIAS” stretched out to a couple of hundred words. And they certainly don't need facts to form the basis of an argument, they just start from the premise that the media is liberally biased and let the conjecture flow from there. The entire enterprise is one big logical fallacy -- they start at the conclusion and work backwards.

Take, for instance, the new Washington Post/ABC News poll showing strong support for a health care reform bill that contains a public option. Clearly, a conservative outfit like NewsBusters wouldn't care for such a dataset, so they set out to discredit the poll with -- you guessed it -- accusations of liberal bias. NewsBuster Tim Graham noted that the poll sample was 33 percent Democrat compared to 20 percent Republican, and accused the Washington Post (but not ABC, for some reason) of “stuffing its poll sample with a few extra Democrats” to get the result they wanted. Mind you, he has no actual evidence that the Washington Post did this, he's just using the following logic, if it can be called that:

1) The Washington Post/ABC News poll sampled more liberal Democrats.

2) The Washington Post is part of the liberal media.

3) The liberal Washington Post rigged their liberal poll to get the liberal result they wanted. Liberal.

Accusing a polling outfit of cooking its data to achieve a predetermined outcome is a pretty serious charge. It's also fairly outlandish and can be easily dismissed with just a basic understanding of one of the fundamental aspects of opinion polling -- the random sample. Polling guru Nate Silver gave an excellent rundown of this very topic last fall when liberals complained about a Fox News poll that oversampled Republicans:

As for the FOX poll, I'm a little bit taken aback at the number of people who assume that, just because the poll is from FOX, it must somehow have been cooked. Sixteen times out of 20, an aberrant result (and I'm not sure you can really call this “aberrant”, since a couple of other pollsters show the race at about 3 points right now) is the result of statistical noise. Perhaps 3 times out of 20, it might be the result of a poor sampling procedure. And then there might be that one case in 20 where the pollster feels compelled to put his finger on the scale in some way -- but these cases are extremely rare. And there's no particular reason to accuse FOX News of this behavior. Their polls haven't had much of a partisan lean this cycle, and for that matter, they were among the only pollsters to have John Kerry winning the popular vote in 2004. If there's a problem with FOX News polls, it's not that they're biased, but that they're simply not all that good.

It's true that FOX's sample included a materially higher percentage of Republicans this time around. FOX, however, does not choose its sample; its sample chooses itself. In this case, when they drew their ping-pong balls out of the jar, they came up with a slightly higher percentage of red ones. This kind of thing will happen all the time unless a pollster weights by party ID, which FOX News and many other pollsters do not. The Pew poll that came out the other day, for instance, had a big increase in the number of Democrats in its sample.

ABC News polling director Gary Langer responded yesterday to criticism from Newt Gingrich, who said that the poll was “deliberately rigged,” and pointed out that not only was their sampling legit, but it was also completely in line with their past polls, contemporaneous polls from other outfits, and the general direction of party identification in the country:

Nor is this out of pattern with the long-term trend in political partisanship in this country. After nearly a generation of gradual advance, the Republican Party in 2003 attained parity with the Democrats; on average that year, for the first time in our polling since 1981, equal numbers of Americans identified themselves as Democrats and as Republicans, 31 percent apiece.

But that trend since has been disrupted. In response to the war in Iraq and the increasingly unpopular presidency of George W. Bush, Republican self-identification has been declining since 2003. (It's no coincidence that Republicans in 2008 made up their smallest share of the electorate since 1980.) The dire news for the GOP in party ID since 2004 is nothing new; we've been reporting it steadily the past five years.

In short, there's absolutely no reason to believe that the Washington Post or ABC News had their fingers on the scales. NewsBusters is just smearing the pollster as evilly liberal because they, quite literally, have absolutely nothing else to say.