Skip to content

Commit 89f9d82

Browse files
john-b-yangclaude
andcommitted
Drop flaky npcomplete bot from RoboCode ladder
andr3eee1/npcomplete failed ~half its matchups in the round-robin (60/115 valid pairs; crashes/timeouts vs many opponents), making its Elo unreliable. Remove it from make_robocode.yaml; its logs and the human/andr3eee1/npcomplete branch on CodeClash-ai/RoboCode were also removed. Ladder is now 115 bots. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent fc25764 commit 89f9d82

1 file changed

Lines changed: 7 additions & 6 deletions

File tree

configs/ladder/make_robocode.yaml

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,18 @@
44
# battles (no code edits). N bots -> N*(N-1)/2 pairs; each pair plays sims_per_round rounds (batched
55
# 10 at a time, sim_concurrency parallel). Bots live on CodeClash-ai/RoboCode human/* branches; Docker required.
66
# COST (measured on an 8-core box: a pair is ~11.8s fixed overhead + ~9.7s per 100 sims, so at
7-
# sims_per_round=300 a single pair is ~41s solo and ~25s/pair of throughput under --workers 6).
8-
# 116 bots -> 6670 pairs -> ~46 core-hours -> ~2 days on an 8-core box (-w 6). Resumable: skips
9-
# already-logged pairs, so a killed run can be re-launched. Scale --workers with cores (~cores-2).
7+
# sims_per_round=100 a pair is ~22s solo). 116 bots -> 6670 pairs. Resumable: skips already-logged
8+
# pairs, so a killed run can be re-launched. Scale --workers with cores (~cores-2).
9+
# SIMS: the Elo fit is per-pair win/loss (per_tournament_boolean_drop_draws), and empirically the
10+
# matchups are lopsided (median winner score-share ~89%; only ~7% of pairs within 55/45), so 100
11+
# sims determines the winner as reliably as 300 for all but a handful of genuine toss-ups. 100 is
12+
# plenty for ranking. (Sim count can even be mixed across pairs without affecting a boolean-win Elo.)
1013
# Run: uv run codeclash ladder make configs/ladder/make_robocode.yaml --workers 6
1114
tournament:
1215
rounds: 0
1316
game:
1417
name: RoboCode
15-
sims_per_round: 300
18+
sims_per_round: 100
1619
sim_concurrency: 5
1720
# Ranking only needs per-battle scores (results_*.txt), not replays. Recording every sim writes
1821
# ~8.7MB of compressed replay XML per pair (~59GB over 6670 pairs) and pins a gzip core on every
@@ -154,8 +157,6 @@ players:
154157
branch_init: human/robo_code/myfirstrobot
155158
- agent: dummy
156159
branch_init: human/luke_f_w/nagisphere
157-
- agent: dummy
158-
branch_init: human/andr3eee1/npcomplete
159160
- agent: dummy
160161
branch_init: human/iagomonteiro13579/npcsniper
161162
- agent: dummy

0 commit comments

Comments
 (0)