Public opinion polling is a complicated process -- and polling on abortion is even more difficult. In particular, polling cannot reflect people’s actual opinions about abortion access when the questions don’t account for the intricacies of the topic or the lived experiences of people who’ve had an abortion.

Right-wing media outlets and anti-abortion groups frequently exploit these weaknesses as a way to argue that the public supports anti-choice views. Sometimes those talking points also seep into other media outlets, such as during a recent interview on CBS This Morning with Planned Parenthood acting President Alexis McGill Johnson about the Supreme Court deciding June Medical Services v. Russo. The case in front of the court is about a Louisiana law that requires abortion providers to have admitting privileges at nearby hospitals and is based on anti-abortion and right-wing misinformation that such privileges are necessary for patients’ safety. However, leading medical organizations in the country say admitting privileges laws are not medically necessary for safe and effective care.

During the interview, CBS’ Tony Dokoupil brought up a Kaiser Family Foundation poll, claiming that it “found a majority of Americans, in fact, 69% support the idea of doctors having admitting privileges if they are going to be doing abortions.” Dokoupil then asked Johnson whether this is “a case where public opinion is out of line with the law” or if Planned Parenthood is “out of line with public opinion.” While right-wing media and anti-abortion outlets also touted this poll as proving Americans support admitting privileges law, a further reading of the poll from Kaiser Family Foundation explains the situation is much more complicated:

In other words, as Johnson answered Dokoupil in the CBS interview, “a lot of that polling … suggests that there has been a campaign of misinformation, disinformation around what it means to access abortion.” Right-wing media’s exploitation of anti-choice polling means that media outlets need to investigate what polls are truly saying about support for abortion access before reporting on them. In particular, media outlets should consider the sources and framing of a poll, the intricacies of the questions asked, and whether the results actually explain people’s nuanced views on abortion.

The source and framing of abortion-related polls

Source of poll / commissioned polls

Before reporting on results, the American Association for Public Opinion Research advises journalists to ask, “Who conducted the poll?” This is because “many different organizations pay for polls to be conducted. It is important to determine who conducted and paid for the poll so you can evaluate credibility and if they have a ‘dog in the fight’ on issues their poll is measuring attributes about.”

The same is true for abortion-related polling.

Polling commissioned by the anti-abortion organization Knights of Columbus (typically conducted by the polling firm Marist) is a prime example of how organizations are able to sway the outcome. For example, the Knights of Columbus released a poll in February 2019 that supposedly showed an increase in the number of individuals identifying as “pro-life” over “pro-choice,” in contrast with historical polling results. As Ed Kilgore of New York magazine noted, “One of the perks that comes with commissioning polls is that you generally get to spin the results,” and the Knights of Columbus “have had a history of finding what they are looking for in polls on abortion policy.” David Byler, a data analyst for The Washington Post, explained his concerns about using this poll as evidence of increasing anti-choice sentiment. As Byler wrote:

Right-wing media's reporting on the Knights of Columbus polls regularly claims to show that a majority of Americans oppose abortion access. Coverage of the organization’s February 2019 poll was no exception, and was often accompanied by attention-getting headlines calling it a “shock poll,” or claiming that the results showed a “huge shift” in public opinion or that it represented “jaw-dropping” support of the anti-abortion movement.

Despite Byler’s caution, outlets outside of right-wing media have similarly relied on the Knights of Columbus’ polling as evidence to claim abortion rights are unpopular -- without disclosing that the poll was commissioned by an organization that is explicitly opposed to abortion rights. In the past, Politico, Axios, and Newsweek have described the Knights of Columbus solely by its religious affiliation. In one article about the group’s polling, The Washington Post did not mention the organization at all, simply disclosing that the poll was done by Marist.

Quantitative vs. qualitative research

Beyond considering who commissions a poll, journalists should also consider the style of research used to gauge abortion-related opinions as it also influences the results.

In examining previous polling data on abortion, the polling firm PerryUndem “found that very little” of the existing data sets contained “a multi-dimensional perspective of opinion on the issue” of abortion, but instead were “surprisingly one-dimensional, outdated, and irrelevant to most policy discussions.” To combat this, PerryUndem suggested that pollsters should use both “qualitative and quantitative research” methods to understand more completely “how voters feel, think, and talk about the right to abortion.” At its most basic, quantitative data focuses on numbers (like polling), while qualitative data examines thoughts, perceptions, and ideas.

In exclusively quantitative research, polling that centers on abortion legality (which seems to be the majority of questions asked by major polling companies) can be strengthened by asking more detailed questions. According to ThinkProgress, even if “many people are morally opposed to abortion,” they may not “necessarily think it should be out of reach for other people who feel differently, and they may struggle with not knowing how to represent both of those views equally.” Such nuanced (and seemingly conflicting points of view) can be better identified if pollsters ask more detailed questions that go beyond the surface level. According to BuzzFeed News, “Asking people questions about what they think of specific abortion laws — common questions in some of the most cited polling by nonpartisan research organizations — often had little bearing on how those people actually felt about abortion itself, or on how they vote”:

As BuzzFeed News further explained, “Though many poll-takers will initially seem confident in their answers, [polling firm] Ipsos found ‘it was clear that people, when discussing [abortion] in focus groups, really changed their attitude when they were forced to think deeply about it and see the nuance.’”

For example, PerryUndem’s research points to quantitative data from a CBS News poll that showed that “35% of adults think abortion should be available under stricter limits than it is now.” Using a qualitative method instead, PerryUndem’s focus group research (which selected people based on their “views on abortion,” their “gender,” and “vote preference” for candidates in the 2016 presidential election) found that “the majority of participants say the experience of having an abortion should be safe, supportive, comfortable, and accessible in communities.” The qualitative research method suggests a different result because it allows participants to describe how they would want the experience of abortion to be for others instead of assuming respondents know the details of current abortion laws and whether those laws should be “stricter.”

The questions in abortion-related polls

Language

The results of abortion-related polling are particularly sensitive to the specific language used in writing questions. In its polling on abortion, Ipsos found “that changing only one word in a question — ‘should abortion be legal’ versus ‘should abortion remain legal’ — could often result in opposite answers.” Although someone of any gender can have an abortion, a poll “could use the words ‘women’ and ‘safe’ in the question, rather than abstractly asking about abortion legality, and the number of people supporting abortion rights will jump by 9 percentage points,” according to a research survey by PerryUndem and Vox.

To demonstrate the sway that certain word choices can have on the results of abortion-related polling, PerryUndem replicated a question asked in the February 2019 Knights of Columbus poll that “asks respondents to choose when ‘abortion should be allowed’ in terms of gestational age or circumstances.” However, PerryUndem “then asked a follow up to clarify responses: Do voters want lawmakers to pass new laws that reflect their responses to when abortion should be allowed or do they think it’s better lawmakers stay out of the issue?” In contrast to the Knights of Columbus’ findings that “the public wants more restrictions on abortion,” PerryUndem found the follow-up question demonstrated “that a majority of voters (68%) think it’s better if lawmakers stay out of the issue.”

Question order

The American Association for Public Opinion Research advises journalists that “the order in which questions are asked can impact the results” and encourages them to consider how “the questions asked prior to the question you’re interested in have influenced how respondents answered the following questions?” For abortion-related polling in particular, the order that certain questions are presented can implicitly bias a respondent toward a certain style of answer.

In just one example, Gallup noted that the results of a question it asked about “Americans' identification as pro-choice or pro-life can vary due to survey context.” The organization found that there is “less of an advantage for the pro-choice position” -- meaning that respondents are less like to express support -- if the identification question comes after “long-term trend questions about the legality of abortion.” In effect, presenting respondents with a series of questions about whether abortion should be allowed (morally or legally) can enable pollsters to unintentionally skew results toward more negative views of abortion.

The answers and results of abortion-related polls

Further complications

Beyond how the questions themselves can affect answers, results of abortion-related polling can be sensitive to the environment surrounding the respondent. As BuzzFeed News outlined, results can be affected by whether a respondent is “answering the questions by phone when someone else is in the house,” whether the pollster “sounds like a young woman,” whether the respondent knows the poll is being conducted by “an interest group like Planned Parenthood or the anti-abortion group Susan B. Anthony List,” and whether the respondents had been influenced by “the pervasive media narrative at the time.”

Difficulties of polling on later abortion

Right-wing media frequently push the idea that the majority of Americans support a ban on abortion at 20 weeks -- often relying on polling as evidence of their claims. Outlets outside of the right-wing media echo chamber adopt this framing frequently, claiming that there is “pretty wide support for restrictions on abortions after 20 weeks.” However, the polling that is used as evidence of this claim typically asks whether people support a 20-week abortion ban (without any context), resulting in misrepresentations of public opinion in a way that unduly bolsters right-wing media and anti-abortion claims.

Other research shows that there’s a drastic drop in support for 20-week abortions bans when people realize that abortions in later stages of pregnancy are often undertaken out of medical necessity or for particular personal circumstances. For example, a Harvard T.H. Chan School of Public Health study on the Zika virus found that when asked in the abstract about later abortion, “less than a quarter of people (23%) believe women should have access to a legal abortion after 24 weeks except in the case of the health or life of the mother.” However, that flipped when people were asked about access to a later abortion when a pregnant person had been infected with the Zika virus -- with results showing “a majority of Americans (59%) believe a woman should have access to a legal abortion after 24 weeks” in that situation.

In other words, as Hart Research Associates found, “Once voters consider the range of circumstances in which abortions would be made illegal under most 20-week abortion ban proposals, a majority of Americans oppose” those restrictions. Polling by PerryUndem has also showed that people believe that the power to decide when to have an abortion should reside with the person who needs an abortion, their doctor, and the larger medical community -- and not be determined by politicians.

As the 2020 elections approach, right-wing media and anti-choice groups are sure to exploit abortion polling to argue that Democrats are extreme and out of line with voters’ beliefs. When reporting on such polling, other media outlets should consider both the challenges of conducting representative polling on the topic, as well as the lived experiences of those who have had an abortion. By moving beyond the sometimes misleading toplines of abortion-related polling to consider their circumstances, structure, and methodologies, outlets can report on this issue with care and insight rather than inattentively repeating right-wing talking points.