Skip to content

Commit 4d2b270

Browse files
john-b-yangclaude
andcommitted
Fold elo bootstrap into a single -b/--bootstrap arg with optional N
Replace the separate --bootstrap/--n-bootstrap pair with one optional-value arg: absent = off, -b = run with 1000 samples/type, -b N = run with N. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 6470062 commit 4d2b270

1 file changed

Lines changed: 10 additions & 10 deletions

File tree

  • codeclash/analysis/metrics

codeclash/analysis/metrics/elo.py

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1593,17 +1593,17 @@ def write_latex_table_plain(results: dict[str, dict], output_dir: Path) -> None:
15931593
help="Directory to save plots (default: assets/elo_plots)",
15941594
)
15951595
parser.add_argument(
1596+
"-b",
15961597
"--bootstrap",
1597-
action="store_true",
1598-
help="Run the bootstrap rank-stability analysis (nonparametric + parametric). Off by "
1599-
"default: it refits the whole model per sample, so it is very slow (minutes per sample on "
1600-
"a large ladder). The ranking and per-fit ±1σ uncertainties are produced without it.",
1601-
)
1602-
parser.add_argument(
1603-
"--n-bootstrap",
1598+
nargs="?",
16041599
type=int,
1605-
default=1000,
1606-
help="Bootstrap samples per type when --bootstrap is set (default: 1000).",
1600+
const=1000,
1601+
default=None,
1602+
metavar="N",
1603+
help="Run the bootstrap rank-stability analysis (nonparametric + parametric). Off unless "
1604+
"given: pass -b/--bootstrap to run it with N=1000 samples/type, or -b N to set N. It refits "
1605+
"the whole model per sample, so it is very slow (minutes/sample on a large ladder). The "
1606+
"ranking and per-fit ±1σ uncertainties are produced without it.",
16071607
)
16081608
args = parser.parse_args()
16091609

@@ -1655,7 +1655,7 @@ def write_latex_table_plain(results: dict[str, dict], output_dir: Path) -> None:
16551655
for bootstrap_type in ["nonparametric", "parametric"]:
16561656
bootstrap_results[bootstrap_type] = BootStrapRankStability(
16571657
builder,
1658-
n_bootstrap=args.n_bootstrap,
1658+
n_bootstrap=args.bootstrap,
16591659
game="ALL",
16601660
regularization=args.regularization,
16611661
topks=None,

0 commit comments

Comments
 (0)