What makes a good headline? And for that matter, how do we define what “good” even means in such a context? Is a good headline one that informs the reader what to expect -- a sort of one-sentence summary? Or is it one that piques a reader’s curiosity, enticing them to click and share an article across their social media accounts?

There’s no single, agreed-upon answer, but every news organization must -- knowingly or not -- find a balance between the two that works for them. But what happens when the latter comes at the expense of the former? One recent event makes for an interesting case study on the topic.

On March 24, Attorney General William Barr released a four-page letter on special counsel Robert Mueller’s report investigating Russian interference in the 2016 election. In it, Barr wrote that he thought it was “in the public interest to describe the report and to summarize” its main conclusions. Barr went on to state that Mueller’s team had determined that there were “two main Russian efforts to influence the 2016 election,” but that “despite multiple offers from Russian-affiliated individuals to assist the Trump campaign,” he wasn’t able to find that the Trump campaign accepted or acted on any of these offers. On the issue of whether Trump obstructed justice, Barr noted that Mueller did not reach a conclusion on the matter, and so Barr took it upon himself to decline to press charges.

The letter was certainly favorable to Trump and his supporters, but it wasn’t as favorable as many stories' news headlines made it out to be.

In their March 25 front page headlines, none of the five largest daily newspapers in the U.S. (USA Today, The Wall Street Journal, The New York Times, New York Post, and Los Angeles Times) noted that the conclusion was coming from Barr and not Mueller himself. While I personally think that some sort of attribution to Barr belonged in the headline itself -- the Chicago Tribune got it right with a headline reading: “AG: NO RUSSIA CONSPIRACY” -- it’s maybe fair to chalk that up to nitpicking. In one of the few places Barr’s letter did quote directly from Mueller, he wrote, “The investigation did not establish that members of the Trump Campaign conspired or coordinated with the Russian government in its election interference activities.” While perhaps a bit oversimplified, the imprecise wording was not as damaging as some other mistakes outlets made in covering the end of Mueller's investigation.

Other examples of misleading headlines were more troubling, thanks to their use of words like “no proof” or “no evidence.” The Philadelphia Inquirer’s March 25 front page boldly states that there was “NO EVIDENCE OF CONSPIRACY.” This goes beyond anything Barr wrote in his letter. As CNN chief legal analyst Jeffrey Toobin told Washington Post media critic Erik Wemple, “One thing we know with certainty is that Mueller is not bringing a criminal case based on the collusion set of issues. But that doesn’t mean there’s no evidence of collusion. It only means there’s not a prosecutable case. There’s a world of difference between ‘no evidence’ and not enough evidence to bring an actual case.”

The Inquirer wasn’t alone either, as similar narratives were being shared across social media and in online articles. CNN tweeted, “President Trump claims vindication after Mueller finds no evidence of collusion.” (CNN would later edit this headline to more accurately read, “Trump claims vindication after Mueller does not establish collusion.”) Politico reporter Darren Samuelsohn wrote, “Mueller finds no evidence of Trump-Russia conspiracy.” The Wrap, The Wall Street Journal, New York Daily News, Bloomberg, ABC’s The View, and NBC’s Meet the Press shared similar messages. The claim was also included in articles by The Associated Press and The New York Times, among other news organizations.

Both Toobin and Lawfare’s Ben Wittes pointed out the problems with this “no evidence” messaging in Wemple’s piece. Wittes noted that such framing “may turn out to be a lucky guess, but it is not supported by the current record.” Wemple also quoted guidance sent out to journalists on the Post’s national desk about the issue:

It is not accurate to say that Mueller found no evidence of a criminal conspiracy between Trump associates and Russia. Barr’s memo states that Mueller did not “find” or “establish” a criminal conspiracy — meaning whatever evidence the special counsel found, it did not rise to the level of that legal standard.

On April 3, The New York Times reported that some members of Mueller’s team were pushing back on some of conclusions drawn from Barr’s letter, arguing that he glossed over portions of the report that painted Trump in a bad light. Just days earlier, on March 29, Barr clarified that his March 24 letter wasn’t actually intended to be a “summary” of Mueller’s report at all.

While none of this is to suggest that Mueller’s findings differ from Barr’s in any sort of legal sense, the narrative published in the letter’s immediate aftermath was a bit simpler and more exculpatory than reality may dictate. (And this is to say nothing of the numerous outlets that amplified Trump's false claim that the Mueller report was a "complete and total exoneration” of him.) Unfortunately, correcting that narrative may be more difficult than simply publishing an updated article.

Does subtle misinformation in headlines really matter? One study suggests it can have a big effect on a reader’s comprehension of articles themselves.

Responding to the Times’ new reporting, BBC anchor Katty Kay tweeted that Democrats risk looking “like sore losers” if they continue to pursue investigations into Russian interference in the 2016 election. Her tweet was widely panned, but she wasn’t entirely wrong.

A 2014 report published by the Journal of Experimental Psychology explored the role headlines had on what readers take away from articles in both opinion and reported journalism. The researchers behind the report, titled “The effects of subtle misinformation in news headlines,” concluded that “misleading headlines affect readers’ memory, their inferential reasoning and behavioral intentions.”

One of the study’s most interesting findings was just how resilient headline misinformation can be, even when explicitly corrected within the article itself:

Correcting the misinformation conveyed by a misleading headline is a difficult task. Particularly in cases of non-obvious misdirection, readers may not be aware of an inconsistency, and may thus not initiate any corrective updating. By contrast, if a headline is perceived as inappropriate, people may be able to correct its influence on their understanding of the article, although this correctional effort itself may withdraw resources from mnemonic processing of the article and may thus impair memory. In sum, these effects further corroborate the notion that misinformation tends to influence people’s memory and reasoning continuously despite corrections.

This is to say that even in the best case scenario, in which someone reads both the original and updated articles on a story all the way through, bits of the early narrative -- such as claims that there was “no evidence” -- will linger long after it’s been corrected. Kay was right to say that the public narrative around the Mueller report may, indeed, have already been formed. What Kay’s tweet ignored, however, is what role journalists like herself played in creating it.

Responsible publications have an obligation to factor in the importance of a piece of news when calibrating the balance between accuracy and clickability.

The more serious a story is, the more deliberative the headline-writing process should be. Condensing a four-page letter, itself the product of a nearly 400-page report, into a single sentence is an impossible task. Even so, it’s the responsibility of journalism to advance the public understanding of an issue, not muddy it. This lesson goes beyond Trump and Russia, speaking to a need to restore trust in the press. Gray areas -- such as whether it’s fair to say “no evidence,” in this case -- make for difficult editing decisions, but they’ll ultimately benefit both the public and press if made correctly.