Call for Contribution
Last update: Sep 2, 2026
RSI Bench is built by and for the AI research community. If you're working at the frontier of ML research, we want your hardest, most recent problems, the ones that would genuinely test whether an AI agent can do your job.
What you get
- Co-authorship
- on the RSI Bench research paper. The author position reflects the scope and difficulty of what you contribute.
- $2,000 per accepted task
- (for the initial cohort of 50 tasks).
- Modal compute credits,
- to cover the GPU and compute costs of building and iterating on your task.
- Access to the RSI Bench research community:
- once your initial proposal is selected, we'll invite you to the RSI Bench Community Slack, connecting you with other selected contributors and researchers across our networks.
Task structure
Every task is made up of four components:
- Instructions:
- the agent-facing description of what to attempt.
- Baseline:
- a strong reference solution that sets the bar to beat.
- Environment:
- a container with the agent's workspace, and the read-only baseline artifacts and reference assets.
- Verifier:
- the script that scores a submission after the agent's session ends.
More details coming soon.
How it works
- Submit a short proposal describing the task you'd contribute, your background, the sub-domain, and what the task involves.
- If approved, you'll get computer access to build it (or use your own setup).
- Build and submit the task.
- Every submitted task goes through review, by the community and the Scale team, before being accepted into the benchmark.
Full contribution guidelines, including task format and submission details will be released soon.
Timeline
| Sep 4 | Priority proposal deadline |
| Sep 18 | Target feedback for priority proposals |
| Oct 2 | Task submission deadline |
| Nov 1 | Final 50 tasks selected |
Ten categories
For this first iteration of RSI Bench, we’re looking for 50 high-quality tasks spanning ten categories.
| 01 | Architecture | 06 | Data |
| 02 | Infra & Systems | 07 | Evals |
| 03 | Pre-training | 08 | Alignment |
| 04 | Post-training | 09 | Harness Optimization |
| 05 | Multimodal | 10 | Applied |
But this is only the beginning. We see RSI Bench as an evolving benchmark: one we’ll continue to expand, refine, and make more challenging alongside the research community as the frontier moves.
Current task proposal distribution
We want RSI-Bench to represent a diverse range of research challenges across all of our task categories. The chart below shows how proposals are currently distributed. If you see a category that's underrepresented and have a strong task idea for it, we'd especially encourage you to submit it.
Register here for full requirements.
FAQ
A number of you have reached out with the same questions, so rather than answer them one at a time, here they are for everyone.
Can I build a task with collaborators?
Yes. There's no limit on how many people can work on a task. Every contributor must be named on the submission, sign the contributor terms individually, and meet the eligibility requirements. We don't assess individual contributions within a team; everyone named on a task is treated as having contributed equally.
Can I submit more than one proposal?
Yes. There's no cap on how many proposals you can submit or how many of your tasks can be selected. That said, we care much more about quality than quantity, so please submit carefully considered proposals that you genuinely believe would make strong additions to the benchmark. Category caps still apply to the benchmark as a whole.
How does the $2,000 work for a team?
The $2,000 is per selected task and is split equally among the task's contributors. Two contributors would receive $1,000 each; four would receive $500 each. We pay each contributor directly rather than sending the full amount to one person to distribute.
How does co-authorship work across a team?
Everyone named on a selected task will be included as a co-author on the paper. When multiple people collaborate on a task, we treat their contribution to that task as shared equally.
What does this mean for your proposal?
There's nothing you need to do differently right now. If your proposal is selected, we'll reach out when we invite you to build the task in the GitHub repository, and you can confirm your collaborators then.
Nothing else has changed: the priority proposal deadline is still Friday, September 4, feedback still targets Friday, September 18, and tasks are still due Friday, October 2.