Skip to content

Commit 40f206c

Browse files
committed
Update readme
1 parent 44373a7 commit 40f206c

1 file changed

Lines changed: 24 additions & 0 deletions

File tree

configs/ablations/ladder/README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,27 @@ For instance, for RobotRumble, we created a ladder by doing the following steps:
1111
You can follow these steps to create your own "CC:<arena>" ladder.
1212
The tricky part is typically finding a large collection of human solutions for a particular arena.
1313
We've typically found that googling for online leaderboards or awesome-<arena> repositories (e.g. [BattleSnake](https://github.com/BattlesnakeOfficial/awesome-battlesnake)) is a good strategy.
14+
15+
## Config layout
16+
17+
Each arena has a few kinds of config in this folder:
18+
19+
- `make_<arena>.yaml` — the round-robin used to **build** the ladder (`ladder make`), running PvP tournaments across all pairs of human bots to rank them.
20+
- `<arena>.yaml` — the **run** config (`ladder run`): a climber ascends the ranked ladder rung by rung until it loses.
21+
- `<arena>__<model>.yaml` — per-model run configs (e.g. `battlesnake__opus_4_8.yaml`) that swap in a specific model via `model: !include mini/models/llama_<model>.yaml`.
22+
- `rungs/<arena>.yaml` — the ranked opponent list (worst first, strongest last), shared by both `<arena>.yaml` and every `<arena>__<model>.yaml` through `ladder: !include ablations/ladder/rungs/<arena>.yaml`. Edit the ladder in this one file and every config for that arena picks it up.
23+
24+
## Ladder advancement rule (`ladder_rules`)
25+
26+
Each run config carries a `ladder_rules` block controlling what it takes to clear a rung and continue climbing:
27+
28+
```yaml
29+
ladder_rules:
30+
min_round_win_fraction: 0.5 # must win strictly more than this fraction of rounds
31+
win_last_k: 1 # ...and must win the last K rounds (1 == just the final round)
32+
```
33+
34+
The defaults shown above reproduce the historical behavior (strict majority of rounds **and** win the final round); the block is optional and falls back to these values if omitted. Validation:
35+
36+
- `win_last_k` must be an integer with `1 <= win_last_k <= tournament.rounds`.
37+
- `min_round_win_fraction` must be a number in `[0, 1)`; `0` drops the majority requirement (e.g. `min_round_win_fraction: 0` + `win_last_k: 1` means "just win the final round").

0 commit comments

Comments
 (0)