5 questions to ask before creating an online survey for research

Online surveys are one of the best ways to gather data for research. They’re quick and easy for both researchers and participants to use and have a lower margin of error due to direct participant input.

However, before you start to create an online survey for research, you need to be clear on several key details. In this post, we’ve compiled 5 questions that will help you to develop the most effective online survey for your research.

1. What research question are you trying to answer?

It might sound obvious at first, but you need to know what research question you’re trying to answer. Simply saying you want to “research managers” is far too vague. Having a clear and specific question to answer (or at least shed light on) will inform many decisions about your online survey.

For example, your research question might be, “does mindfulness improve employee motivation?” You should think about why this is your research question. Does this question develop an area of empirical research? Is it filling an important gap in public knowledge? Could it inform a company policy? A researcher might decide to explore the mindfulness question because of its potential to provide insights into corporate motivation. 

This information can then guide your approach to the survey. For example, your target demographic will need to be employees. This could be from a particular industry, or you might even look at multiple industries to determine how generalizable the findings are.

2. Are you doing exploratory or confirmatory research?

You’ll also need to decide whether your research will be exploratory or confirmatory. These are both equally reliable options, but you need to be aware of their differences.

In exploratory research, you won't have any real hunch or hypothesis about how the results of your research will turn out. Your goal is to discover the relationships between variables. With confirmatory research, you're trying to “confirm” if a specific hypothesis is accurate or not.

You need to think carefully about which is suitable for your survey, as it will impact the way you approach the survey and how you measure any data. In the mindfulness example, a confirmatory study would use more precise measurements to gather data, such as a self-report test about feelings of motivation. Meanwhile, an exploratory study will investigate on a more general level what motivates employees. It will usually use qualitative data, such as interviews and statements.

3. What is your target population?

When you create an online survey for research, it’s important to know who you want to survey. In the mindfulness example, it’s clear that you’d need to survey people who are employed. However, you might want to be more specific with the demographic. Perhaps you’d like to look at how mindfulness impacts motivation in the male population specifically, or employees in Ghana or Japan.

Pre-screening filters for age, race and socioeconomic status, as an example, can help to ensure the participants of your online survey match your target group.

Alternatively, you might want to use a representative sample. This will ensure that participants are reflective of the general population of a particular country. This is incredibly important for generalizing your data. If the participants are not representative, you can’t determine whether your results will be applicable to the public as a whole.

4. Do you need quantitative or qualitative data?

Another important question to ask when you create an online survey for research is whether you need quantitative or qualitative data. Quantitative data is numerical data which you gather en-masse from a large group of people via methods such as questionnaires or psychometric testing. The results are displayed as statistics, which you can then analyze and use to draw conclusions about relationships between the data.

In contrast, qualitative data is smaller-scale and non-numeric. This data is usually more detailed and focused on the participant’s perspective, involving studies of interviews and diary entries, for example. Researchers generally use qualitative data for exploratory research, pinpointing potential correlations to open up new avenues of research. You can also use qualitative data to explore contradictory or unclear phenomena discovered in quantitative research, revealing participants’ feelings and experiences in more detail. 

Ultimately, the type of data you choose needs to be relevant to your research question and your overall approach. In the mindfulness survey, analyzing diary entries is arguably less relevant than quantitative testing through an online survey for research.

5. How large should your sample size be?

Once you’ve decided whether to use quantitative or qualitative data, it’s time to think about how large your sample size should be. In terms of qualitative data, you’ll need to be sure your sample size is large enough to draw accurate conclusions. Otherwise, your data might not be very valuable.

Meanwhile, for quantitative data, you need to be clear about whether your study will be observational or experimental. Observational studies don’t involve any intervention from the researcher. For these studies, you’ll need to determine a suitable margin of error for your primary variable, as well as the level of confidence you need in your results.

In terms of experimental data, you need to ensure you have the right sample size for a suitable level of statistical power. Put simply, the higher statistical power a study has, the more likely you are to detect a true effect – that is, an effect that is actually there. To determine the required sample size, look at existing studies or run pilot tests to uncover the potential effect size, then use a power calculation to figure out the ideal sample size. There’s no such thing as an “oversampled study”, so the larger the sample size the better.

Research you can rely on

When it comes to creating an online survey for research, your survey is only as good as your participants. However, finding reliable data can be challenging. That’s where Prolific can help.

At Prolific, you can instantly find trusted research participants for your online surveys. Have a specific target group in mind? With over 200 pre-screening filters available, you can tailor your participants to your research requirements. 

Ready to make research simple? Get started with Prolific now.

‍