If you read statistical survey results without knowing the margin of error, or MOE, you are only getting part of the story. Survey results themselves (with no MOE) are only a measure of how the sample of selected individuals felt about the issue; they don’t reflect how the entire population may have felt, had they all been asked. The margin of error helps you estimate how close you are to the truth about the population based on your sample data.
Results based on a sample won’t be exactly the same as what you would’ve found for the entire population, because when you take a sample, you don’t get information from everyone in the population. However, if the study is done right, the results from the sample should be close to and representative of the actual values for the entire population, with a high level of confidence.
The MOE doesn’t mean someone made a mistake; all it means is that you didn’t get to sample everybody in the population, so you expect your sample results to vary from that population by a certain amount. In other words, you acknowledge that your results will change with subsequent samples and are only accurate to within a certain range — which can be calculated using the margin of error.
Consider one example of the type of survey conducted by some of the leading polling organizations, such as the Gallup Organization. Suppose its latest poll sampled 1,000 people from the United States, and the results show that 520 people (52%) think the president is doing a good job, compared to 48% who don’t think so. Suppose Gallup reports that this survey had a margin of error of plus or minus 3% with 95% confidence. Now, you know that the majority (more than 50%) of the people in this sample approve of the president, but can you say that the majority of all Americans approve of the president? In this case, you can’t. Why not?
You need to include the margin of error (in this case, 3%) in your results. If 52% of those sampled approve of the president, you can expect that the percent of the population of all Americans who approve of the president will be 52%, plus or minus 3%. Therefore, it’s plausible that between 49% and 55% of all Americans approve of the president. That’s as close as you can get with your sample of 1,000. But notice that 49%, the lower end of this range, represents a minority, because it’s less than 50%. So you really can’t say definitively that a majority of the American people support the president, based on this sample. You can only say you’re 95% confident that between 49% and 55% of all Americans support the president, which may or may not be a majority.
Think about the sample size for a moment. Isn’t it interesting that a sample of only 1,000 Americans out of a population of well over 310,000,000 can lead you to be within plus or minus only 3% on your survey results? That’s incredible! That means for large populations you only need to sample a tiny portion of the total to get close to the true value (assuming, as always, that you have good data and took a sample that is a reasonable representation of the entire population). Statistics is indeed a powerful tool for finding out how people feel about issues, which is probably why so many people conduct surveys and why you’re so often bothered to respond to them as well.
