In his July 14 Rocky Mountain News column, media critic and Independence Institute research director Dave Kopel dubiously asserted that Fox News “is slanted to the right ... to a lesser degree than CBS, ABC and NBC slant left” and cited a Quarterly Journal of Economics study to substantiate his claim. However, the study Kopel cited used a highly questionable methodology that categorized the American Civil Liberties Union (ACLU) as a “conservative” group and the RAND Corporation as a “liberal” group, and led the authors to conclude that The Wall Street Journal has more “liberal bias” than any other news outlet in America. Media Matters for America has detailed the methodological flaws in that 2005 study.

From Dave Kopel's July 14 Rocky Mountain News column, “Newshounds.us keeps tabs on Fox News”:

If you hate Fox News, you will love Newshounds.us, a citizen activist Web site dedicated to criticizing Fox. I learned about the site a few weeks ago, for its coverage of Bill O'Reilly's numerous sensational untruths about Boulder High School. The site's motto is “We watch Fox so you don't have to” -- an especially useful service for readers such as myself, who don't have cable television.

[...]

Some of the Newshounds write-ups on other Fox topics strike me as too conspiracy-minded or Manichean. I agree with Newshounds that Fox is slanted to the right, but to a lesser degree than CBS, ABC and NBC slant left, as detailed in a 2005 Quarterly Journal of Economics study. Even so, Newhounds plays a helpful role in empowering citizens to check and balance the major media. It would be great if there were comparable citizen activist sites for every one of the major television networks.

As Media Matters noted, the November 2005 study to which Kopel referred -- "A Measure of Media Bias" by political scientist Timothy J. Groseclose of UCLA and economist Jeffrey D. Milyo of the University of Missouri-Columbia -- is a flawed source of evidence that “CBS, ABC and NBC slant left.” In the study, Groseclose and Milyo attempted to “measure media bias by estimating ideological scores for several major media outlets” based on the frequency with which various think tanks and advocacy organizations were cited approvingly by the media and by members of Congress over a 10-year period.

In order to assess media “bias,” Groseclose and Milyo assembled the ideological scores given to members of Congress by the liberal group Americans for Democratic Action; examined the floor speeches of selected members to catalog which think tanks and policy organizations were cited by those members; used those citations as the basis for an ideological score assigned to each think tank (organizations cited by liberal members were scored as more liberal, whereas organizations cited by conservative members were scored as more conservative); then performed a content analysis of newspapers and TV programs to catalog which think tanks and policy organizations were quoted. If a news organization quoted a think tank mentioned by conservative members of Congress, then it was said to have a conservative “bias.” As Groseclose and Milyo put it:

As a simplified example, imagine that there were only two think tanks, and suppose that the New York Times cited the first think tank twice as often as the second. Our method asks: What is the estimated ADA score of a member of Congress who exhibits the same frequency (2:1) in his or her speeches? This is the score that our method would assign the New York Times.

In other words, the study rests on the presumption that if a member of Congress cites a think tank approvingly, and if that think tank is also cited by a news organization, then the news organization has a “bias” making it an ideological mirror of the member of Congress who cited the think tank. This, as Groseclose and Milyo define it, is what constitutes “media bias.” However, Groseclose and Milyo's study relied upon numerous dubious assessments of the political orientation of the think tanks and advocacy groups it incorporated. A score of 50.1 was defined as the “political center”; groups with scores below 50 were labeled “conservative” and those above 50 were described as “liberal.”

• National Rifle Association of America (NRA) scored a 45.9, making it “conservative” but close to the “political center.”
• RAND Corporation, a nonprofit research organization -- whose motto is “Objective analysis. Effective solutions” -- with strong ties to the Defense Department, scored a 60.4, making it a “liberal” group.
• Council on Foreign Relations, whose motto is “A Nonpartisan Resource for Information and Analysis” (its president at the time of the study was and remains former Bush administration official Richard N. Haass; its board includes prominent Democrats and Republicans from the foreign policy establishment) scored a 60.2, making it a “liberal” group.
• American Civil Liberties Union, bête noire of the right, scored a 49.8, putting it just on the “conservative” side of the ledger.
• Center for Responsive Politics, a research group whose primary purpose is the maintenance of databases on political contributions, scored a 66.9, making it highly “liberal.”
• Center for Strategic and Budgetary Assessments, a defense policy think tank whose board of directors at the time of the study was and continues to be chaired by former U.S. Rep. Dave McCurdy (D-OK), scored a 33.9, making it more “conservative” than the American Enterprise Institute (AEI) and the National Taxpayers Union.

The study Kopel cited contained other questionable findings, including the conclusion that The Wall Street Journal has more “liberal bias” than any news outlet they surveyed. Dow Jones & Co., the Journal's publisher, described some of the study's deficiencies in a response published on the Poynter Institute online forum, concluding, “Suffice it to say that 'research' of this variety would be unlikely to warrant a mention at all in any Wall Street Journal story.” According to the Dow Jones statement:

The Wall Street Journal's news coverage is relentlessly neutral. Of that, we are confident.

By contrast, the research technique used in this study hardly inspires confidence. In fact, it is logically suspect and simply baffling in some of its details.

First, its measure of media bias consists entirely of counting the number of mentions of, or quotes from, various think tanks that the researchers determine to be “liberal” or “conservative.” By this logic, a mention of Al Qaeda in a story suggests the newspaper endorses its views, which is obviously not the case. And if a think tank is explicitly labeled “liberal” or “conservative” within a story to provide context to readers, that example doesn't count at all. The researchers simply threw out such mentions.

Second, the universe of think tanks and policy groups in the study hardly covers the universe of institutions with which Wall Street Journal reporters come into contact. What are we to make of the validity of a list of important policy groups that doesn't include, say, the Chamber of Commerce, the National Association of Manufacturers, the AFL-CIO or the Concord Coalition, but that does include People for the Ethical Treatment of Animals? Moreover, the ranking the study gives to some of the groups on the list is simply bizarre. How seriously are we to take a system that ranks the American Civil Liberties Union slightly to the right of center, and that ranks the RAND Corp. as more liberal than Amnesty International? Indeed, the more frequently a media outlet quotes the ACLU in this study, the more conservative its alleged bias.

Third, the reader of this report has to travel all the way Table III on page 57 to discover that the researchers' “study” of the content of The Wall Street Journal covers exactly FOUR MONTHS in 2002, while the period examined for CBS News covers more than 12 years, and National Public Radio's content is examined for more than 11 years. This huge analytical flaw results in an assessment based on comparative citings during vastly differing time periods, when the relative newsworthiness of various institutions could vary widely. Thus, Time magazine is “studied” for about two years, while U.S. News and World Report is examined for eight years. Indeed, the periods of time covered for the Journal, the Washington Post and the Washington Times are so brief that as to suggest that they were simply thrown into the mix as an afterthought. Yet the researchers provide those findings the same weight as all the others, without bothering to explain that in any meaningful way to the study's readers.

Suffice it to say that “research” of this variety would be unlikely to warrant a mention at all in any Wall Street Journal story.