Here is my proposal on how I think Prolific can allocate funding in a transparent, fair, and effective manner. I highlight in […] why a certain step is important.
Many grant schemes today make it difficult for junior scholars to apply for funding, be it because they require the researcher be a certain number of years past obtaining their PhD (e.g., ERC Starter grants - 2ys), or has a certain level of prior publishing (e.g., la Caixa - at least 1 article in Q1 journal). In many systems, researchers are rewarded for having obtained other grants in the past, creating a snowball effect for some and leaving out others who might have great ideas but have gotten unlucky.
How to submit a proposal:
- Grants follow a pre-specified timeline [transparency]
- Grants exist in three “levels” (and Prolific specifies how often these grant schemes take place and how much money is allocated):
→ amendment 2+3, credit to Magdalena_Kachlicka and Jantsje_Mol
- Small grants, open to everyone: for conducting small-scale pilot tests of studies [ensure research is feasible and makes sense before too much money is spent]
- Junior grants, open to pre-doctoral researchers and PhD students only [those who currently have the fewest opportunities]
- Junior+ grants, open to pre-doctoral researchers, PhD students, post-doctoral scholars, and tenure-track faculty within 7 years of obtaining their PhD. Proposals may be for co-authored projects; in this case the co-author(s) and their contribution to the project need to be disclosed. Preference will be given to single-authored projects or those where all co-authors satisfy the junior eligibility criteria (i.e., all else equal, these projects will receive the funding over projects with senior co-authors). [do not disqualify people just because they co-author; reward good ideas] → amendment 1, credit to bethaniley: make the contribution of each scholar an evaluation criterion in order not to disadvantage PhD students who are required to (for example) add their advisors on all proposals regardless of the advisors’ involvement
- Projects are not restricted time-wise (e.g., given a completion or publication deadline) but realistic timeline is part of project evaluation [prioritize quality over speed]
- Submissions should contain the research question, methods, budget, and scientific contribution, and researchers will have the freedom to choose how these are presented (e.g., charts vs. text). IRB approval, pre-registration of hypotheses/methods incl. a pre-analysis plan and power calculations where applicable, and commitment to making all data and codes public after completing the study are a required part of the proposal. The researcher is free to add any other relevant information about the study (e.g., running pilots, their own expertise using the methods, …) The entire application should aim for approximately 2500 words with a limit of 3000. [commitment to scientific rigor AND open science]
- Prior to the competition, Prolific specifies any legal/technical constraints the proposals need to follow (e.g., budget allocation).
How are grants allocated:
- Every researcher who submits a proposal is required to be a referee for 2 other proposals in his/her discipline (broadly defined). Referees write a public (up to 1500 word) evaluation, which is posted under the proposals (and clearly flagged as such). These evaluations should point out errors (if any), and discuss whether the research is 1) scientifically sound, and 2) innovative. Prolific will provide a standardized and transparent grading scale that the referees will be instructed to use. Referees may make suggestions to the designs, which researchers may but do not have to accept. [focus on scientific rigor]
- The broader Prolific community may vote or comment on proposals or referee evaluations for a period of 3 weeks afterwards. Following this input, and discussion with the researchers, the referees are allowed to change their grade of the proposal. Everybody is allowed to vote or comment, but the person’s Prolific involvement is disclosed: how long have they been active on the forum, how many studies have they run, which institution and discipline are they from. [involvement of the community without giving incentives for voting by family or friends, improving proposals together, fixes any misunderstandings]
- The researcher with the best proposal as evaluated by the referees after the public discussion period will receive the funding; 10 of the most helpful referees (as voted by the community) will receive a smaller prize. [incentives for referees to help others succeed]