A poll’s margin of error, sample size matters a lot: Here’s why

In almost any discussion of a poll about the very close presidential race between Vice President Kamala Harris and former President Donald Trump, you’ll hear the phrase “within the margin of error of the survey.” These words signal that it is a close race with no clear leadereven if one of them has a slightly higher percentage of support, from 48% to 47%.

The Oregonian/OregonLive October 2024 Local Public Interest Poll.

That director of the Quinnipiac University Pollwhich has been taking the public’s pulse on political issues and elections for the past 30 years, I’ve noticed that people have been paying more attention to this technical term since at least 2016.

That year, some polls in Florida, for example, indicated that Hillary Clinton was only a few percentage points ahead to Trump. Journalists and the public have largely – and incorrectly – understood that the popular vote apparently leads to harm Clinton will probably win.

But those 1 or 2 percentage points were within the margins of error of their polls. And Clinton lost Florida. In a poll about a political race, the margin of error tells readers the likely range of election results.

What is a margin of error?

A survey is one or more questions asked to a small group of people and used to gauge the opinions of a larger group of people. The margin of error is a mathematical calculation of how accurate the survey results are—how closely the answers given by the small group match the opinions held by the larger group.

If everyone in the larger group were surveyed, there would be no margin of error. But it is complicated, difficult and expensive to contact so many people. The US Census Bureau spent $13.7 billion over several years in the latest effort to count every person in the United States every 10 years, and counting it was not able to include exactly everyone.

Pollsters don’t have that kind of time—or money—so they use smaller samples from the population. They seek to identify representative samples where all members of the larger group have a chance to be included in the survey.

Group size is important

Calculating how close the poll is to the views of the larger population is based on the size of the group being surveyed.

For example, a sample of 600 voters will have a larger margin of error—about 4 percentage points—than a sample of 1,000 voters, which has a margin of error of just over 3 percentage points.

The way the sample is chosen also matters: In 1936, Literary Digest magazine polled people about the presidential election by sending surveys to phone owners, car owners and country club members. Everyone in this group was relatively wealthy, so they were not representative of the entire US voting population. Calculating a margin of error would have been meaningless because the sample did not capture all segments of the population.

A concrete example

Let’s use an example of how to understand the margin of error. If a poll shows that 47% of the polled group supports candidate A, and the margin of error is plus or minus 3 percentage points, it means that the percentage of the population that supports candidate A is probably between 44% (47 minus 3). ) and 50% (47 plus 3).

A quick note: most surveys report margins of error alongside another technical term, “confidence interval.” In the most rigorous polling reporting, you might see a sentence near the end that says something like “The margin of error is plus or minus 3 percentage points at a 95% confidence interval. All this means is this: imagine if 100 different random samples of the same size were selected from the larger pool and then asked the same survey questions. The 95% confidence interval means that 95% of the time the responses to the other surveys would be within 3 percentage points of the responses reported in this single survey.

Comparison of support between candidates

The concept of margin of error becomes more complex when looking at differences in support between two candidates. If a margin of error is plus or minus 3 percentage points, the margin of error on the difference between them is roughly double that – or 6 percentage points, in this example.

This is because here the margin of error is a combined one and refers not only to the percentage vote for candidate A, but also to the percentage vote for the other candidate.

Let’s look again at 2016, the final Quinnipiac University of Florida poll ahead of Election Day it showed Clinton with 46% support and Trump with 45% support. The margin of error was 3.9 percentage points, which meant Clinton was likely to get between 42.1% and 49.9% of the vote, and Trump was likely to get between 41.1% and 48.9% from votes.

The actual result was that Trump won Florida with 48.6%compared to Clinton’s 47.4%. Those results were within our poll’s margin of error, meaning we were right to say it was “too close to call” — and wrong to say Clinton was ahead.

2024 will be a close election

In the current election cycle, many media reports about polls are without including information about the margin of error.

Omitting this information or downplaying its significance can help media outlets provide a quick and simple picture of the state of the race. Technology can seem precise in the modern age of the internet and artificial intelligence.

But polls are not as accurate. It’s an inexact science. It is the job of a survey to capture snapshots of the complexity of human nature at a given moment. People’s minds can change and new information can emerge as campaigns unfold.

With the presidential election in its final weeks, our polls have found a pretty tight and steady racethe majority of voters telling us that they have decided. With the margin of error between the presidential candidates in swing states, polls in the fall of 2024 are telling Americans to hold their breath and make sure they vote, because it’s likely going to be a squeaker.