8 things to keep in mind when designing a research study

Developing a high-quality online research study can be difficult. Gathering the best data for your research hinges on good study design. And there are many different factors to consider.  

From the visuals to the question order, research design affects the quality of responses you'll get from participants. Not sure how to design a research study that will get the results you need? Don’t worry. In this guide, we’ll take you through 8 key things you need to keep in mind.

How to design a research study – 8 key steps

1. Think carefully about your questions

The questions you ask participants can affect the quality of their responses. You need to think carefully about two key things:

Phrasing your questions

How you phrase your questions is important. Certain words have different meanings, and participants may interpret them in different ways. Your questions should also align with the response format. A “yes/no” question won’t work with a Likert scale, for example.

Keep the language simple and remove any generic expressions. You should also avoid double negatives, unless the question is reverse scored.

Ordering your questions

Ask general questions first to ease participants into the survey. You can then move on to more personal questions. Group topics together under specific headings to make them clear for participants.

You should always pilot test questions to ensure that participants can understand them. Test the question order as well to avoid data bias.

2. Minimize bias

All research surveys are prone to bias. However, there are some simple ways to minimize the impact of bias on your findings. Types of bias to consider include:

• Question-order bias – As mentioned above, the order of your questions can bias participants’ answers. To avoid this, begin with positive questions and leave sensitive questions to the end.
• Confirmation bias – This type of bias occurs when you focus on results that support your hypothesis and ignore any results that refute it. You should always check if there are alternative explanations for your data. Pre-registering your research, as well as talking with colleagues, can also help.
• Social desirability bias – Participants may give socially acceptable answers instead of honest ones. To prevent this, make sure participants are aware that your survey is anonymous. Also, always pilot test your questions where possible.

3. Know when to use the Likert scale

The Likert scale is an ordinal scale. It measures participant responses with five to seven fixed options. For example, a Likert scale might have “Strongly Agree” and “Strongly Disagree” at opposite ends of the scale. There is usually a neutral option in between. If “yes/no” questions are too restrictive for the data you want to collect, using a Likert scale is a better option.

4. Collect data securely and responsibly

When you’re working with data, it’s important to be aware of proper data collection procedures. Ensure you’re keeping data safe by:

• Only collecting data that you actually need for your research. Never collect participant data, such as IP addresses, for the sake of it.  
• Always ensuring you have proper online security in place. This is especially important for personal or sensitive data. Encryption, two-factor authentication and pseudo-anonymization are important security measures to put in place.

5. Get consent from participants

Before collecting data from participants, you should always get their consent through a consent form. This form should be clear and easy for participants to understand. It should explicitly state:

• The data you’ll collect
• Whether any data collected will be sensitive or personal
• How you’ll use the data
• How you’ll store the data – and how long you’ll keep it for
• How you’ll ensure responses are kept anonymous
• If other online researchers will be given access to the anonymized data
• How the participant can remove their consent and data
• The legal framework used to hold their data

Always make sure you ask the participant if they understand the consent form. Then check if they are happy to proceed. You should also make them aware that they can remove their consent at any time.

They can then click a check box as evidence that they have consented. Your institution may have their own wording requirements for consent forms, so research this first.

6. Write a study description and debriefing

Participants will want to know the details of your study. They might be curious about the purpose of the study. Or they might have questions about what they need to do.

You can give them this information in the study description. Share details like the study’s aim, what the participants will need to do, and how you’ll use the data gathered. For a full list of what to include, read our detailed guide to writing the study description and debrief.

For the debrief, always thank the participant for taking part - and make sure you inform them of any deception used in the study.

7. Decide on a reward

To ensure your study is ethical, you need to pay your participants fairly for their time. This can also improve the quality of the data they provide. At Prolific, we strongly believe in the importance of ethical research. That’s why we’ve implemented a minimum hourly reward of £6/$8.  

If you want more participants to engage with your study and provide accurate answers, you may need to pay more. We recommend you pay at least £9/$12 per hour. But this will vary depending on different factors, such as the level of effort needed for the study.

8. Consider your visual design

The visual design of your research study can also impact the data quality. For example, if your font is hard to read, participants may misinterpret the question. Choose a simple font, such as Arial, in a readable size.

Pick colours carefully. There should be a clear contrast between the writing and the background. Use bright colours sparingly, especially for sensitive topics. For longer studies, it can also help to include a progress bar.

Design a research study for high quality data

To get the best data for your research, you need to design your study carefully – and find trusted participants to take part. With Prolific, finding the right participants is simple. Our platform features 200,000+ vetted research participants from diverse populations around the globe.

Want to learn more about effective research design? Find out how to design and launch a high-quality online study in The complete best practice guide to online research.