On Monday, Fox & Friends mischaracterized a recent study about Google’s search algorithm and its “Top Stories” search result rankings to push the myth of anti-conservative censorship by tech platforms an opinion piece pushing similar claims ran the same day on FoxNews.com. But the network used the wrong statistic in its segment and misconstrued the methodology of the study.
An audit study from the Northwestern University reviewed over 6,300 unique links from Google’s “Top Stories” box in 200 search results during the month of November 2017, as one of the study’s researchers, Nicholas Diakopoulos, explained in a Columbia Journalism Review report The study found that “just 20 news sources account for more than half of article impressions” (defined by the researchers as the number of times unique news links appear in the Top Stories box). In addition, the top three sources with the most article impressions (CNN, The New York Times, and The Washington Post) accounted for 23% of all article impressions.
fox news covered this story as 'leftist bias at google' even though one could say the stronger suggestion is that 'conservative news is misinformative trash' https://t.co/yTVjJeCmz9 pic.twitter.com/wDDP6swbZ3
— Bobby Lewis (@revrrlewis) May 13, 2019
Fox News misrepresented these findings, and misidentified other labels in the study, to claim that left-leaning media outperformed right-leaning media on Google. Sam Thielman, one of the editors of CJR’s write-up of the study, pointed out these mischaracterizations on Twitter.
Hello, I was the editor on this! First, Fox gets the stat in that image wrong—the 86% is from the top 20 *percent* of stories. Second, the work was performed by Facebook and published in Science, not by the Northwestern Computational Journalism Lab 1/2
— Sam Thielman (@samthielman) May 13, 2019
The Science paper assigned perspectives to news outlets based on the stated ideologies of the people who shared an outlet's stories on Facebook. Ideology is fluid and we thought this was as good a yardstick as any, but it doesn't correlate to anything more complex than that! 2/2
— Sam Thielman (@samthielman) May 13, 2019
On Fox & Friends, the story ran under the headline “Left-Leaning Sources Dominate Google News.” But according to Diakopoulos, ideological labels on news sources, which were borrowed from another study published in Science magazine, “don’t measure the slant of the media outlet per se, but rather reflect the self-reported political affiliation of Facebook users sharing content from those sources.” Fox News host Jillian Mele failed to mention this specific characterization and instead used the label “left-leaning sources” without context or further definition. An opinion piece about the study published on Fox’s site the same day also failed to make that distinction. Thielman said that Diakopoulos and he made the methodology as explicit as possible because they anticipated bad actors would try to manipulate the the results:
god I edited this piece so carefully and Nick did such a great job being explicit about the methodology because we both knew EXACTLY THIS was going to happen
— Sam Thielman (@samthielman) May 13, 2019
Anyway here are the relevant two paragraphs within Nick's excellent article. Kudos to him, the Northwestern Computational Journalism Lab, and the Facebook researchers who published in Science and gave Nick et al this very interesting metric: https://t.co/HFIaFyrUR6 pic.twitter.com/hKnWyShUJM
— Sam Thielman (@samthielman) May 13, 2019
Fox also misstated the key finding it highlighted in the headline. The study found that of 678 sources analyzed, the top 20% of sources (136) accounted for 86% of article impressions. Fox News misclassified that finding, stating that “86% of top stories came from only 20 news sources.”
From the May 13 edition of Fox News’ Fox & Friends: