Skip to content

Commit d066f16

Browse files
committed
docs: open up contributing — agents, platform, suite all welcome
1 parent a4bbdcd commit d066f16

2 files changed

Lines changed: 45 additions & 3 deletions

File tree

CONTRIBUTING.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Contributing to CoderCup
2+
3+
Thanks for helping make the benchmark better. All kinds of contributions are welcome — new agents, platform improvements, bug fixes, stronger test plans, task ideas.
4+
5+
## The flow
6+
7+
1. **Fork** the repo and create a branch.
8+
2. Make your change. For site/code changes, make sure the checks pass locally:
9+
10+
```bash
11+
npm install
12+
npm run typecheck # tsc --noEmit
13+
npm test # vitest
14+
npm run build # static export
15+
```
16+
17+
3. **Open a Pull Request** against `main`. CI runs the same checks automatically.
18+
4. A maintainer reviews and merges. Merges to `main` deploy [codercup.ai](https://codercup.ai) automatically, so reviews are careful — please keep PRs focused and small.
19+
20+
## What to contribute
21+
22+
| You want to… | Start here |
23+
|---|---|
24+
| Enter a new coding agent | [`runners/README.md`](./runners/README.md) + a [new-driver issue](https://github.com/TestSprite/CoderCup/issues/new?template=new-driver.md) |
25+
| Improve the site / harness / scoring | A regular PR — for larger changes, open an issue first so we can align before you invest time |
26+
| Strengthen the test suite | Edit the plan JSONs under [`tests/`](./tests); explain in the PR what failure mode the tighter assertion catches |
27+
| Challenge a published verdict | Open an issue linking the test page on codercup.ai and the artifact (video / recorded steps) that contradicts it |
28+
| Suggest a task requirement | [Task-suggestion issue](https://github.com/TestSprite/CoderCup/issues/new?template=task-suggestion.md) |
29+
30+
## Ground rules
31+
32+
- **Neutrality is the product.** Changes that could tilt the arena toward any vendor — in the spec, the suite, or the scoring — get extra scrutiny. If your change affects scoring, say so explicitly in the PR.
33+
- **Never hand-edit published results.** Scores and fixtures are derived from the ledger by scripts; PRs that touch `scores/` or `public/fixtures/` by hand will be asked to go through the pipeline instead.
34+
- Be a normal, decent human in issues and reviews.
35+
36+
## Questions
37+
38+
We're responsive on [Discord](https://discord.gg/W4JDrZfdB), or email [contact@testsprite.com](mailto:contact@testsprite.com).

README.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -104,9 +104,13 @@ CoderCup is open to any AI coding agent that runs headlessly on a Linux host thr
104104

105105
## Contributing
106106

107-
- **Test suite & rubric PRs welcome** — every plan JSON in [`tests/`](./tests) is reviewable; tighten an assertion, propose a new phase surface.
108-
- **Task suggestions** — want a requirement folded into the next iteration? [Open a task-suggestion issue](https://github.com/TestSprite/CoderCup/issues/new?template=task-suggestion.md).
109-
- **Bug reports**[bug template](https://github.com/TestSprite/CoderCup/issues/new?template=bug.md).
107+
Contributions of every kind are welcome — this benchmark gets better the more people poke at it. See [CONTRIBUTING.md](./CONTRIBUTING.md) for the 2-minute guide. Some ideas:
108+
109+
- **Add a coding agent** — the highest-impact contribution. Any agent that runs headlessly on Linux through a CLI can enter; start from [`runners/README.md`](./runners/README.md) or just [open a new-driver issue](https://github.com/TestSprite/CoderCup/issues/new?template=new-driver.md) and we'll help you wire it.
110+
- **Improve the platform** — the leaderboard site, the runner harness, the scoring pipeline: PRs for features, refactors, and bug fixes are all fair game.
111+
- **Tighten the test suite** — every plan JSON in [`tests/`](./tests) is reviewable; strengthen an assertion, challenge a verdict, propose a new phase surface.
112+
- **Shape the next event**[suggest a task requirement](https://github.com/TestSprite/CoderCup/issues/new?template=task-suggestion.md) for the next iteration.
113+
- **Report bugs**[bug template](https://github.com/TestSprite/CoderCup/issues/new?template=bug.md), or just open a plain issue if the templates don't fit.
110114

111115
## Support
112116

0 commit comments

Comments
 (0)