Attention checks are an important tool in the fight against bad data quality. But, we also want to make sure that any checks that are used are fair.
We know that you face two key problems in this area:
- It’s not clear what a ‘fair’ attention check looks like in some scenarios
- Some researchers are great at making fair attention checks, but others need more guidance.
- We’ll soon be releasing brand-new guidance that aims to be clearer.
- We’re launching a Community Attention Check Repository
This will be a resource created by the community, for the community! You’ll be able to submit attention checks which can be used by other members. The process is simple:
- Use this template
- Submit your attention checks here
- Our team will review it within the next few weeks, and if it needs to be made more ‘fair’, we’ll contact you with our recommended changes.
Note: This is just an initial test of this idea, so our release of the approved checks may take longer than expected, and we may not be able to fully launch this for some time.
Looking forward to your submissions!