I’m running a study that will recruit 4,000 survey respondents and wondering how likely it is that I will get roughly 50:50 male/female-identifying split in responses without creating separate surveys with equal slots for males and females, each screening out the other gender. The survey topic is relatively gender neutral (restaurant menu choice simulation) and other screeners are more likely to eliminate slightly more female responses (e.g. cannot be vegetarian). Any insight would be appreciated! Thank you, Stacy
Hey Stacy, our participant pool is skewed slightly more towards female-identifying participants, and the way our studies are assigned to participants means that a split is not guaranteed. To be absolutely sure you’ll get a 50/50 split, we recommend that you run 2 separate studies
I follow-up on the topic, since for my studies I found samples of roughly 70% females. Although in principle this is not a problem, for the generality of some results it might be. I’m wondering, are there other ways to narrow down a bit this gap? For example, are there nationalities for which the imbalance is smaller, or hours of the day for which there is evidence of parity in participants’ activity by gender? If not, still the suggestion on the 2 separate studies is very insightful!
That’s a great question. Let me investigate, and get back to you
I looked into it, but unfortunately there’s no significant difference between nationalities. They all hover somewhere between 50-55% of our participant pool. So, it’s strange that you’ve been getting a 70% split.
But, we distribute our studies evenly, not randomly. Our primary tool is something called adaptive rate limiting. In essence, when the number of active participants is high relative to the number of study places available, we give priority access to participants who’ve spent less time taking studies recently.
So, it could be the case that the majority of those given priority access, in your case, happened to identify as female
I see, maybe that’s why it happened, it could be.
Anyways, thank you very much for the information!