How to choose a good sample size for your survey
A fitting sample size is key to extrapolating accurate results from your survey. Too small a sample, and your results will almost certainly be unreliable. Too large, and you’ll have wasted precious resources by oversampling your population.
You don’t need to guess what the correct sample size for your survey should be. Use this straightforward yet proven calculation method to work out the ideal sample and get accurate results from your study.
Understand why sample size matters
Your survey sample needs to accurately match the thoughts, opinions, experiences, and backgrounds of the total population you’re trying to study. The appropriate sample size ensures your survey findings are generalizable, meaning they can be usefully applied to the broader population you’re studying.
Say you’re conducting market research on your target audience of millennial dog owners for your brand of gourmet dog food. What is a good sample size for your survey? The answer is one that allows you to draw accurate conclusions about the entire population of millennial dog owners and their opinions and shopping habits around dog food. You want to know the data you’ve collected is a true reflection of the diversity of the population and isn’t negatively affected by having too large or small a sample.
There are risks to choosing the wrong sample size:
- Too few participants: You may find relationships that don’t exist for the larger population, or you may miss important relationships that do exist for the larger population. These are called Type I and Type II errors, which are, respectively, false positive and false negative conclusions.
- Too many participants: You’ll waste resources — namely money and time — by sampling more people than you really need to come to the same conclusions. Say you only need to sample 500 dog owners to be accurate, but you sample 1,000. That’s a big waste of your team’s time. Plus, oversampling can lead to Type I errors if small differences in the survey responses lead to statistically significant effects that don’t exist in the real world.
Use the correct sample size calculation
You can use either a precision-based or power-based calculation to determine your sample size. For a descriptive/survey-based study, you’ll want to go with a precision-based calculation.
- Precision-based calculation: Commonly used in descriptive research, this type of calculation is used to figure out the sample size for studies that seek to estimate something in the population. If you’re trying to estimate the prevalence of an opinion or trend among the target population, usually with a survey, a precision-based calculation works really well.
- Power-based calculation: Power is the ability of your study to detect a statistically significant effect through a significance test. These calculations are primarily used in experimental research when you’re testing different hypotheses, such as the effectiveness of drug treatments vs. placebos. You don’t need to worry about this calculation if you’ll just be running a descriptive survey.
For most types of online surveys, you only need to follow the steps for a precision-based sample size calculation, which we lay out below. If you plan on running statistical tests on your results, you can try this G*Power tool for a power-based calculation and read more about how power-based sample size calculations work.
Work out your population size
The total population size is the first figure you’ll need to know before you can work out your sample size. The sample will be based on this number to ensure your survey is capturing a large enough snapshot of the population.
You might already know the exact number of individuals in your population. For instance, if you work for human resources for a Fortune 500 company and you’re conducting an employee engagement survey, you’ll already know how many people work for your company. Or you’re conducting a market research survey, and your total population is adults in South Dakota. You can get that number easily from the latest census data and then take a random sample.
If you don’t know the exact number in your population, you’ll have to make an estimate. Let’s say you want to survey bike owners in U.S. cities. There is no definitive number or place where that data is tracked, but you can research bike ownership and give an estimate to the best of your abilities.
For the sake of our example, let’s say we want to find out whether a new product is ready for the market. The total market for our product is adults in the U.S., so our population size is about 258.3 million people.
Determine confidence level and margin of error
How confident do you want to be in your results, and what’s an acceptable margin of error? That’s what you’ll determine with these two figures.
- Confidence level, in very basic terms, tells you how confident you can be that your findings reflect the true values that would be seen in the larger population. In statistical terms, it tells you the probability of the results for your sample also matching the true population. Typical confidence levels in studies are 90%, 95%, and 99%.
- Margin of error: The margin of error tells you the degree of uncertainty or error in your findings, i.e., how precise the estimate you have found will be.
For example, let’s say that you run a study and find that 30% of your respondents answer “yes” to a question. Your margin of error is set at 5, and your confidence level is 95%. You can say that there is a 95% probability that between 25% and 35% of the actual population would have answered “yes” to the same question. So, if you ran the same study 100 times, you would expect that in 95% of those studies, between 25% and 35% of participants would answer “yes.”
Estimate standard deviation
Standard deviation is a measure of how much the survey responses vary from the mean. Estimate how much you expect responses to vary from one another to determine the sample size you need.
For example, let’s say you’re looking at whether or not participants like the taste of a new food. In this case, we know that taste is very subjective, so we would expect there to be a higher standard deviation because participants aren’t likely to rate the new food similarly. However, if we were launching a product such as a vacuum cleaner and running a survey to gather customer feedback, we wouldn’t expect responses to vary so much. Objectively, the product either works or doesn’t work, and performance should, hopefully, remain consistent.
If you’re not sure what to estimate, a standard deviation of 0.5 is typical to plug in for an industry survey or other precision-based calculation.
Calculate using the sample size formula
Plug your figures into the sample size formula to calculate the right sample size for your survey.
To use the formula, convert your confidence level to a Z-score. If you’re using a 95% confidence level, you’ll use a 1.96 Z-score.
Look up your Z-score for your confidence level from this table.
The sample size formula you’ll want to use is:
Sample size = (Z-score)2 × standard dev × (1 – standard dev)
(margin of error)2
Using our numbers, our formula would be:
((1.96)2 × 0.5 × (1 – 0.5))
(0.05)2
And we would calculate a sample size of 385 participants.
You can also use an online calculator, like this one from SurveyMonkey, to make it even simpler.
Select your sample from a quality pool
Figuring out how many participants you need is only the first step to running a quality survey. You also need to draw those participants from a quality pool of respondents to get results that are truly generalizable to your population.
