CNN anchor Kyra Phillips touted a meaningless Gallup “flash poll” in coverage of reactions to President Bush's June 28 speech on Iraq, failing to inform viewers that the poll questioned far more Republicans than Democrats. Phillips only hinted at this sampling problem by noting that “presidential speech viewers are a much more partisan group than the general public.”

From the June 29 edition of CNN's Live From ...:

PHILLIPS: Well, the reviews are in from the Bush speech, and they're largely positive. A CNN/USA Today/Gallup poll of people who actually watched the speech finds 46 percent report a 'very positive' reaction, 28 percent 'somewhat positive.' And we point out that presidential speech viewers are a much more partisan group than the general public.

But the poll's sample was not so much “more partisan” than the general public as it was much more Republican. Fifty percent of the poll's respondents were Republicans, while only 23 percent were Democrats and 27 percent were independents.

By contrast, the National Annenberg Election Survey found 31.8 percent were self-identified Republicans, compared to 34.6 percent self-identified Democrats, based on a random sample of 67,777 registered voters polled over 13 months from 2003 to 2004. Similarly, Gallup's own study, based on polling data from 37,000 interviews in 2004, found 34 percent of Americans self-identifying as Republicans; 34 percent self-identifying as Democrats; and 31 percent self-identifying as independents.

In a June 28 CNN.com article, CNN polling director Keating Holland accurately noted that the results of the flash poll do not necessarily reflect the opinions of the broader American public. “It's difficult to tell from these poll results how the speech will affect general U.S. public,” he said. “Many Americans did not watch the speech. Those who did were 2-to-1 Republican, so most were arguably already in the president's camp.”

Unfortunately, CNN did not include Holland's comments in its TV coverage. CNN also failed to inform viewers that the poll's sample was not random. While the audience for Bush's speech may have been more Republican than Democratic, Gallup's sample still may not have been not an accurate representation of the speech's full audience. The poll was part of a broader “panel survey” in which respondents were interviewed before and after the speech. Thus, the sample consisted only of people who agreed to take a survey; said they intended to watch the speech; actually watched the speech; and were at home and willing to be interviewed a second time when Gallup called back after the speech on the night of June 28. Any or all of these criteria could have contributed to the poll's relatively large Republican sample.

Though other CNN reports on the flash poll were not as misleading as the Live From ... newscast, several failed to explicitly note that its sample differed dramatically from the percentages of Democrats and Republicans in the general public.

Shortly after the president's speech, for example, CNN's Larry King Live guest host Bob Costas stated:

We've just received a CNN/USA Today flash poll of 323 adult Americans, all of whom watched President Bush's speech tonight. This is significant, and the pollsters have asked us to make note of it. The audience was 50 percent Republican, 23 percent Democratic, 27 percent independent. And the reaction of those 323 adult Americans, very positive reaction to the president's speech tonight -- 46 percent.

On the June 29 edition of CNN Live at Daybreak, anchor Carol Costello similarly omitted the discrepancy between the party identification of those surveyed and the nation at large, telling viewers, “And by the way, most of the people who watched the president's speech are Republican.”

On the June 29 edition of American Morning, anchor Soledad O'Brien pointed out that “folks who already agree with the president and more likely to watch him speak, well, last night's audience was 50 percent Republican, 23 percent Democratic and 27 percent independent.”