How Columbia ran a complex study to change opinions about the climate crisis

Researchers at Columbia and Northwestern University collaborated on this research project to investigate possible pathways to changing people’s beliefs about climate change. Led by Sandra Matz at Columbia University and Moran Cerf at Northwestern University, the research team aimed to answer the question: what happens when false beliefs come with a tangible cost?

The task 

Attitudes towards climate change and support of climate legislation have often divided America. People’s beliefs around climate change tend to be shaped by their ideological tribe’ - for instance, whether they vote Republican or Democrat. And it can be difficult to shake them. 

This is, in part, because holding false beliefs often doesn’t cost us anything in the here and now. While we might all suffer from the consequences in the future, there are no real repercussions for denying the danger of climate change. On the contrary, some Republicans might be shunned for going against their tribe and saying that climate change is real. 

This research aimed to investigate what would happen when respondents were asked to put their money where their mouth is and bid’ money on their beliefs. Could this not only increase their knowledge about climate change but also alter their opinions about how real and dangerous climate change is? 

Every day for 60 days, respondents were asked if they wanted to bid up to $1 on the likelihood of a climate event happening (for example: This July will be the hottest month on record’, or This September will be the wettest month of the year’). Participants were given scientific sources to help them decide if they wanted to bid, and how much.

The challenge 

The main difficulty with the logistics of this research was that it needed to happen in three very distinct parts.

• Gathering data about attitudes towards climate change to identify eligible respondents.
• Contacting these respondents to gauge their interest in taking part in the study.
• Running the study every day for 60 days to complete the research. 

This was a very involved study. Swift communication between the researcher and participants was critical. The researcher needed to contact large groups of respondents quickly and easily, while still retaining the ability to individually message respondents as well. 

Although respondents only had to spend a minimum of 2 minutes a day on the bidding, the overall length of the study raised concerns that respondents would become bored or unreliable towards the end of the research.

The solution 

With assistance from Prolific, Columbia and Northwestern set up a short screener study, gathering information about the knowledge and opinions of the respondents to make sure groups were evenly balanced in terms of their climate change knowledge and beliefs. They were also asked about their interest in taking part in the longer 60-day study if they were successful, which was integral as they needed participants who would stay engaged through the longer study. They then used Prolific’s mass messaging features to invite eligible participants to the study.

As with any research study, data quality was critical to the research team. Prolific has some of the best quality data compared to other survey facilitators, with respondents being more honest, reliable, and consistent. As a result, Columbia and Northwestern felt confident in the quality of responses that they received from this research. Prolific offered rapid customer support at every step of the study. If researchers encountered any issues or questions from respondents that they couldn’t answer, the support team were always on hand to help.

The results 

1,005 respondents began the research and 667 completed the entire study - a retention rate of 66%. This relatively low drop-off rate for such a long and involved study shows that the respondents were highly engaged and committed throughout the study. 

The full findings of the study, including answers to whether this research was able to alter people’s beliefs about climate change, will be released soon.