What if Prolific could directly implement the scheduling of studies?

As mentioned in another post, I have been trying to run online multiplayer experiments with big groups (at least 8 at a time). However, there are a lot of complications and moving parts involved in doing so. The main aim is to get a lot of people to come in at the same time so that people don’t have to wait for the groups to slowly fill up.

It was recommended to me that I follow a process similar to that of video/audio interview. This involves creating a study, where participants are directed to a scheduling tool to schedule when they will participate. In the meantime, they have to be given a code so that they can submit the study and do other studies between now and then.

I decided to do without the scheduling tool, they can have privacy issues (even Doodle encourages - although does not force - participants to put in some sensitive information such as names and emails). Instead, I do everything in one link with my coding. Participants are sent to a page that tells them that the study will start in X amount of time and to come back here on time before it starts. It informs them about the code to be able to continue and reminds them that they have to actually complete the study to receive payment.

Nevertheless, we are left with quite a few moving parts and complications:

  • Participants might lose the link or not get a reminder to come back to the study in time. (sending a reminder message 15 mins before might not reach them as they won’t get a notification in the browser add-on).
  • Participants might not return their participation if they didn’t complete the study. It’s never nice to have to chase participants to return their study, nor report them if they refuse to do so.
  • This can still seem weird to participants.
  • They might not plan well and start another study just before ours.
  • They might over-plan and wait in front of our study for a whole chunk of time before it starts.

I was wondering if Prolific might consider removing all these moving parts by creating an option do to studies where participants register for a later start. The experimenter would set when to publish the study and when to release the actual link to the study. The participants would register for it. No need to do stuff by providing a code and it wouldn’t block them from doing other studies. They get a reminder a few minutes before the start, where they can click and get to the actual study. This would make things a lot simpler for experimenters and a lot clearer for participants.

(bonus points if you can replace the scheduling process for the video and audio interviews, but I am just interested in this fix for multiplayer experiments, I don’t need people to choose amongst different time slots, just one time to come back to later in the day.)

Thank you for time

This is a really great idea @Samuel_Dupret! We know how fiddly the current video/focus group interview process is. Any ideas on how to improve it are very welcome!

I’m working on a process to ensure that great community ideas get in front of the right team who can work on implementing them. I’ll keep you updated on where this idea could fit into our future product roadmap

I decided to do without the scheduling tool, they can have privacy issues (even Doodle encourages - although does not force - participants to put in some sensitive information such as names and emails).

Just a note on this: our policy on collecting sensitive data when arranging these types of studies

Collection of personal information is not usually permitted on Prolific, so participants would need to be assured that any personal details if at all collected (e.g. Skype usernames) will only be used to contact them for the interview, and deleted as soon as this purpose has been fulfilled.

Let me know if you have any questions!

Hello again,

I have been doing some testing with multiplayer scheduled experiments. I stand by my ask for a system handled by Prolific. Here are a few new things I have discovered (or rediscovered).

One that’s just general to multiplayer studies where we try to create groups (e.g., 10 participants) and we can’t run groups with fewer participants. If we cannot form a group, we send a message saying we are sorry to participants and offering a small bonus in exchange for them returning their participation. Some participants are annoyed by this and demand that they get to participate.

Some of my comments refer to stuff that a Prolific internal system will solve however:

  • allowing participants to enter the code early so that they can do other studies whilst waiting creates weird overall pay/time scores. Some participants only give in the code at the end, so they count as having participated for a lot longer than the actual study (say 30min wait before, and 30 more minutes of the study, instead of just 30 minutes for the study).
  • Despite providing the code for them to enter early, some participants refer to the waiting time as time that counts. Sometimes indirectly, such as “I waiting X amount of time, and didn’t get to participate”, etc. The fact that the countdown/time wait happens on the experimenter’s side instead of through an official Prolific system means that participants might perceive the waiting time as ‘time they worked for’.

Also, the auto approval feature messes up this whole system.

Best regards,

1 Like