Skip to content

Commit 898ab6b

Browse files
committed
docs: refine description and workflow steps for benchmark README sync
1 parent 71c06ae commit 898ab6b

File tree

1 file changed

+29
-15
lines changed
  • .agents/skills/benchmark-readme-sync

1 file changed

+29
-15
lines changed

.agents/skills/benchmark-readme-sync/SKILL.md

Lines changed: 29 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: "benchmark-readme-sync"
3-
description: "Use when a user wants to refresh benchmark results in `README.md` from the latest successful GitHub Actions `Benchmark` workflow run for the current repository. Automatically find the newest successful run, extract the final Markdown tables for each benchmark case from the job logs, and update the README action URL, date, tool versions, and benchmark tables."
3+
description: "Use when a user wants to refresh benchmark results in `README.md` from the latest successful GitHub Actions `Benchmark` workflow run for the current repository. Use a robust `gh` and shell workflow to find the newest successful run, extract final Markdown tables from job logs, and update the README run URL, date, versions, and tables."
44
---
55

66
# Benchmark README Sync
@@ -12,26 +12,23 @@ description: "Use when a user wants to refresh benchmark results in `README.md`
1212

1313
## Workflow
1414
1. Read `README.md` and `.github/workflows/benchmark.yml` to confirm the benchmark cases and the current results layout.
15-
2. Resolve the canonical GitHub repository and default branch with `gh repo view`, then find the latest successful `Benchmark` workflow run on that branch.
16-
3. List the jobs for that run and map each matrix case to its job ID.
17-
4. For each case job, fetch the log for the `Run Benchmark` step, remove ANSI escape codes, and extract the final Markdown tables printed at the end:
18-
- Keep both `Development metrics` and `Build metrics` when present.
19-
- Keep only `Build metrics` for build-only cases.
20-
- Start output at the first `Development metrics:` or `Build metrics:` heading so earlier log noise is excluded.
21-
- Do not recalculate rankings or rewrite the numbers manually.
22-
5. Update `README.md`:
15+
2. Resolve the canonical GitHub repository and default branch with `gh repo view`, then find the latest successful `Benchmark` workflow run on that branch unless the user gave a specific run ID.
16+
3. List the jobs for that run and map each matrix case to its job ID before touching `README.md`.
17+
4. For each case job, extract only the final benchmark tables from the log using the robust shell pipeline below.
18+
5. Update `README.md` carefully:
2319
- Replace the GitHub Actions run URL with the latest run URL.
2420
- Replace the date with today's local date.
2521
- Replace each case table with the extracted table from the matching job.
26-
- If a case section exists without a table, insert the extracted table under that case.
27-
- Preserve surrounding prose unless the benchmark output itself changes the tool names or versions shown in the tables.
28-
6. Validate the result:
22+
- Preserve the case heading, prose, command block, and the final `---` separator before `## Run locally`.
23+
- Do not recalculate rankings or rewrite the numbers manually.
24+
6. Validation is required after the edit:
2925
- Every case in the workflow matrix is represented in `README.md`.
3026
- No duplicated headings, tables, or rows were introduced.
27+
- The separator before `## Run locally` is still present.
3128
- The diff only changes benchmark data, the run URL, and the date.
3229

3330
## Commands
34-
Prefer `gh` because it is authenticated and exposes both run metadata and logs.
31+
Prefer `gh` because it is authenticated and exposes both run metadata and logs. Use these to execute or debug the workflow manually.
3532

3633
Resolve the canonical repository and default branch:
3734

@@ -58,18 +55,35 @@ Fetch jobs for a run:
5855
gh api repos/<owner>/<repo>/actions/runs/<run_id>/jobs --paginate
5956
```
6057

58+
Map case names to job IDs:
59+
60+
```bash
61+
gh api repos/<owner>/<repo>/actions/runs/<run_id>/jobs --paginate \
62+
| jq -r '.jobs[] | [.id, .name] | @tsv'
63+
```
64+
6165
Extract a job's final benchmark tables:
6266

6367
```bash
6468
gh run view <run_id> --job <job_id> --log \
6569
-R <owner>/<repo> \
66-
| awk -F '\t' '$2=="Run Benchmark"{ sub(/^/, "", $3); sub(/^[^ ]* /, "", $3); print $3 }' \
70+
| cut -f3- \
6771
| perl -pe 's/\e\[[0-9;]*[A-Za-z]//g' \
68-
| awk 'BEGIN{capture=0} /^Development metrics:$/ || /^Build metrics:$/ {capture=1} capture {print}'
72+
| sed -E 's/^\xef\xbb\xbf//; s/^[0-9T:.\-]+Z //' \
73+
| awk 'BEGIN{capture=0} /^Development metrics:$/ || /^Build metrics:$/ {capture=1} capture && $0!="Post job cleanup." {print} $0=="Post job cleanup." {exit}'
6974
```
7075

76+
Notes:
77+
- Do not rely on the second log column being `Run Benchmark`. Current `gh run view --log` output may label lines as `UNKNOWN STEP`, while the third column still contains the benchmark output you need.
78+
- Capture starts at the first `Development metrics:` or `Build metrics:` heading so preamble noise is excluded.
79+
- Stop at `Post job cleanup.` so cleanup lines do not leak into the tables.
80+
- Keep both `Development metrics` and `Build metrics` when present, and only `Build metrics` for build-only cases.
81+
- Prefer replacing one case section at a time or using a temporary one-off local command; do not add repository scripts just to complete a single sync.
82+
- The brittle part of the edit is preserving section boundaries, especially the final `---` before `## Run locally`.
83+
7184
## Failure handling
7285
- If no successful `Benchmark` run exists, stop and report that blocker.
7386
- If a job log is missing or truncated, stop and report which case could not be extracted.
7487
- If the workflow matrix and README sections do not match, update only the cases backed by logs and call out the mismatch clearly.
7588
- If `README.md` already points to the latest successful run and the extracted tables match, the expected result is an empty diff.
89+
- If the workflow breaks down, include the failing step in the report: repo resolution, run lookup, job mapping, log extraction, README replacement, or structural validation.

0 commit comments

Comments
 (0)