We recently went viral on TikTok - here's what we learned

What happened

On 24th July an unaffiliated TikTok video promoting Prolific as a side hustle went viral, with over 4 million views to date.

The video resulted in around 30,000 new participant signups to Prolific, which skewed heavily towards female participants in their 20s - similar to the creator of the video.

Over the last few weeks, as these new participants have begun to take part in studies on Prolific, we've had a change in the submission data of our participant pool - since the TikTok video, it's leaned more heavily towards young females than previously:

Distribution of age by gender for participant submissions before 24th July
 

Distribution of age by gender for participant submissions after 24th July
 

Up until now, we've never actively balanced the demographic samples of studies run on Prolific, other than for reasons of ensuring high data quality. We've instead erred towards giving researchers control over the demographics of the sample they're collecting using our screening feature, in line with what best suits their research.

However, we've heard from a lot of researchers over the past few weeks about the skewed samples they've collected, particularly when it comes to the gender of participants, and what their expectations have been when it comes to a participant pool with no screening applied.

We've not lived up to the high bar you've come to expect when it comes to the quality of the data you gather from Prolific, and for that we're sorry.

We've learned a lot from the last few weeks, and all your feedback, including that:

• We can make it clearer which participants will take part in studies you launch on Prolific, if you don't apply any of our screeners.
• There are certain demographics, like gender, that we should make it easier to balance across in the product, without the need to run multiple studies.
• In the meantime, we can provide better guidance on how you can setup your study to collect a specific distribution of participants, across selected demographics.

What we're doing

For studies run on Prolific since 24th July, that have received fewer than 25% of participants respond with "Male" to our prescreening questions on Sex and Gender identity, we'll be crediting researchers up to the amount of any platform fees we've charged.

You should see the platform fee applied as credit to your account in the coming days. Please note that studies using a Custom Allowlist, or studies that have been run using these screeners are not eligible for the credit.

Over the coming weeks, we'll be looking into building a feature that makes it easier for researchers to balance studies across certain demographics.

In the meantime, here are some things you can do to make sure you still get the sample you need:

1) Run a representative sample (for no additional cost)

We're temporarily making our representative sample feature free, which can recruit a sample that reflects the demographic distribution of either the UK or US populations. You can take a look at our FAQ page for a more in-depth look at how this feature works. However, representative samples can't currently be used in combination with other screeners, and aren't available for other populations.

2) Use screeners to get the distribution of participants you need

If you do need to apply other screeners, and your study needs specific distribution of participants across selected demographics, such as having an even gender or country of residence split, we'd recommend following our guidance on demographic balancing.

We'd also recommend familiarising yourself with other screening options we have, such as being able to screen on the number of submissions from a participant, which can act as a proxy for how long a participant has been with Prolific. Setting this to 20 submissions will roughly equate to getting signups from our participant pool in mid-July.

We've also temporarily introduced a "Participant Join Date" screener, to provide a way to access our participant pool, prior to the recent signups we've seen from 24th July. However, it's worth bearing in mind that both of these options could affect the naivety of your sample.

If you choose not to apply any screeners, with the aim of getting responses as quickly as possible, we've also made it clearer which participants will take part in your study, if no prescreening is applied.

Next Steps

We're dedicated to empowering you to conduct great research on Prolific, and data quality is our top priority.

As well as the above measures, we'll also be more closely monitoring the pool of participants on Prolific to avoid similar spikes in signups, until we have stronger controls in place for letting you easily balance your studies across certain demographics.

Ultimately, we want to ensure you continue to receive high-quality data from Prolific, while our participants continue to get access to the studies that they love.

And finally, thank you everyone for your feedback. It's been massively appreciated, and will help us build a better product that suits all of your research needs.

Log in or sign up to Prolific today and launch your study to thousands of participants in minutes 

‍