Media relying on flawed polls: Gallup and CBS/ NYT skewed toward Republicans

Written by Nicole Casta & Katie Barge

Published

Five new national polls give two very different pictures of the presidential race. Three polls show an extremely close race; two show sizable Bush leads. While some members of the media have pointed out potential methodological reasons for this disparity, one clear explanation has gone largely unreported: both polls that show President George W. Bush with a strong lead oversampled Republicans.

Three polls, conducted by The Pew Research Center for People and the Press (September 11-14), Harris Interactive (September 9-13), and Greenberg Quinlan Rosner Research (September 12-14) all found Senator John Kerry and Bush either within one point of each other or tied among likely voters. A Gallup poll conducted September 13-15 found Bush leading Kerry among likely voters by 13 points (55 percent to 42 percent) and by eight points among registered voters (52 percent to 44 percent). A CBS News/New York Times polls (pdf) conducted September 12-16 showed Bush with a nine-point lead among registered voters (50 percent to 41 percent).

The New York Times, the Chicago Tribune, the San Francisco Chronicle, and The Washington Times have cited various differences in the polls' methodology that could account for the disparate results. These differences include: the specific time period during which a poll is conducted; the ways likely voters are identified; and the percentage of undecided voters in a sample.

But the media has largely ignored both Gallup's and the CBS News/New York Times polls' oversampling of Republicans. As author and joint fellow at the Center for American Progress and The Century Foundation Ruy Teixeira explained, these polls include more Republicans in their sample than is representative of the electorate. According to Teixeira, the CBS News/New York Times poll sample included 4 percent more Republicans than Democrats. And Gallup told TheLeftCoaster.com's Steve Soto that it surveyed 7 percent more Republicans than Democrats. Media Matters for America has previously noted that John Zogby, president and CEO of independent polling firm Zogby International, pointed out on September 7 that in the last two presidential elections, Democrats have represented 4 percent to 5 percent more of the electorate than have Republicans:

If we look at the three last Presidential elections, the spread was 34% Democrats, 34% Republicans and 33% Independents (in 1992 with Ross Perot in the race); 39% Democrats, 34% Republicans, and 27% Independents in 1996; and 39% Democrats, 35% Republicans and 26% Independents in 2000.

When, according to Teixeira, the data from the Gallup and the CBS News/New York Times polls are weighted “to conform to the 4 point Democratic party ID lead which we have good reason to believe is the underlying distribution in the voting electorate,” the Gallup result is a tie (48 percent to 48 percent) -- and the CBS News/New York Times result is a one-point Bush lead (47 percent to 46 percent) among likely voters. Notably, the September 20 Wall Street Journal reported that the Gallup and CBS News/New York Times polls surveyed more Republicans than the Pew poll; the article mentioned Teixeira's criticism.

Nevertheless, FOX News Channel hosts Bill O'Reilly and John Gibson and contributor Newt Gingrich boosted the Gallup poll:

  • O'REILLY: Gallup is, you know, we keep very close track of these pollsters. Gallup is pretty reliable. [The O'Reilly Factor, September 17]

  • GIBSON: I would think the Gallup poll's a little worrying [for the Kerry campaign]. That is a very steady poll over the years; it's not given to wild fluctuations. And it's considered pretty reliable. [The Big Story with John Gibson, September 17]

  • GINGRICH: Gallup fits a pattern that we've seen now for two weeks, of a widening Bush gap over Kerry. ... On the other hand, Harris has been, I think, pretty notoriously unreliable. [Your World w/ Neil Cavuto, September 17]

And on CBS's Face the Nation on September 19, anchor Bob Schieffer cited only the CBS News/New York Times poll (pdf) to conclude: “George Bush has now opened a nine-point lead over John Kerry. You don't have to be an expert to figure that out. Voters may be less than enamored with President Bush but they are even more uneasy about John Kerry, whose plans for the country remain a mystery to them, according to this poll.”