A Consumer’s Guide to Interpreting Poll Results

Most of us have been asked to participate in a public opinion poll or read an article publishing the results of a public opinion poll at least once in our lifetimes. Whether the focus is “Who is going to win the White House” or “How do Shifting Demographics Affect Politics,” public opinion polls about current issues are conducted all the time. Public opinion polls can provide valuable insight into the beliefs and attitudes of a target population, but it can be challenging to accurately interpret the results of a poll. In short, it is important to think twice before accepting newspaper headlines about polls at face value. Below are five practical questions to consider in order to more accurately interpret and understand the results of the latest poll.

Who Conducted the Poll?

First, it is important to know which organization or company conducted the poll and who paid for it. Knowing this will help consumers determine how credible the poll results are and if there should be any concerns about bias. For polls conducted or commissioned by political campaigns, it may be helpful to review the wording of the actual questions to make sure the questions present all candidates with equally unbiased language.

What is the Time Frame for the Poll?

For many issues, public opinion constantly shifts on both local and national levels. As a result, poll results are only reflective of the time frame over which the poll was conducted. Knowing when the poll was conducted helps put opinions in context of other events that may have happened before or after the results were collected. Poll results regarding political elections and candidates are usually the most time sensitive.

Who was the Target Population?

It is crucial to know which population the poll represents because different groups can respond very differently to the same survey questions. For example, registered Republicans may respond very differently than all likely voters, and millennials may respond very differently than baby boomers. It is important to understand relevant political, geographic, and demographic associations like the following:

• Political considerations: Is the target population registered voters? Likely voters? Republicans, Democrats, or Independents? Delegates or caucus goers?

• Geographic considerations: Is the target population a national sample, or is it focused on a particular city, state, or region? Urban, suburban, or rural?

• Demographic considerations: Is the target population agnostic to age, gender, income, employment, and/or education level, or does it target a certain demographic?

How was the Poll Conducted?

The survey methodology can make a big difference in the results of a poll. For example, a poll focused on landline telephones is likely to reach an older demographic relative to an online poll in which respondents are younger. Below is a list of common survey contact methods:

• Landline phone survey
• Wireless phone survey
• Interactive voice response (IVR) survey
• Email survey
• Opt-in web survey
• Mail paper survey
• Mail postcard with survey URL
• Live intercept, or in-person survey

What is the Margin of Error?

Finally, it is important to understand the margin of sampling error (MOE) for interpreting poll results. The MOE provides a best-guess range for the likely answer, and it is generally calculated using a 95 percent confidence level. For example, in March 2016, Gallup reported that approval of the U.S. Congress was at 13 percent with a MOE of ±4 percentage points at the 95 percent confidence level.¹ Here’s a translation in plain English: Gallup’s best guess is that 13 percent of Americans approve of Congress, but a reasonable range of guesses would be between 9 percent and 17 percent.

The MOE is important because it is one measure that shows how confident you can be in the poll results. If a poll shows that a political candidate is projected to win 52 percent of the vote, with a MOE of ±5 pts, there is a high likelihood that the candidate could win anywhere between 47 percent and 57 percent of the vote. So don’t necessarily assume an initial 52 percent means the candidate will win.

The sample size and MOE have an inverse relationship: the larger the sample size, the smaller the MOE.

Conclusion

Polls are useful in assessing public opinion about different issues, but there are important considerations to take into account when analyzing them. Knowing the right questions to ask helps consumers recognize the results for what they are and make educated decisions about what they mean.

 

Sources:

1 https://www.gallup.com/poll/189848/no-improvement-congress-approval.aspx?g_source=congress&g_medium=search&g_campaign=tiles

2 https://www.aapor.org/Education-Resources/Election-Polling-Resources/Margin-of-Sampling-Error-Credibility-Interval.aspx