You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: .agents/skills/benchmark-readme-sync/SKILL.md
+29-15Lines changed: 29 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
name: "benchmark-readme-sync"
3
-
description: "Use when a user wants to refresh benchmark results in `README.md` from the latest successful GitHub Actions `Benchmark` workflow run for the current repository. Automatically find the newest successful run, extract the final Markdown tables for each benchmark case from the job logs, and update the README action URL, date, tool versions, and benchmark tables."
3
+
description: "Use when a user wants to refresh benchmark results in `README.md` from the latest successful GitHub Actions `Benchmark` workflow run for the current repository. Use a robust `gh` and shell workflow to find the newest successful run, extract final Markdown tables from job logs, and update the README run URL, date, versions, and tables."
4
4
---
5
5
6
6
# Benchmark README Sync
@@ -12,26 +12,23 @@ description: "Use when a user wants to refresh benchmark results in `README.md`
12
12
13
13
## Workflow
14
14
1. Read `README.md` and `.github/workflows/benchmark.yml` to confirm the benchmark cases and the current results layout.
15
-
2. Resolve the canonical GitHub repository and default branch with `gh repo view`, then find the latest successful `Benchmark` workflow run on that branch.
16
-
3. List the jobs for that run and map each matrix case to its job ID.
17
-
4. For each case job, fetch the log for the `Run Benchmark` step, remove ANSI escape codes, and extract the final Markdown tables printed at the end:
18
-
- Keep both `Development metrics` and `Build metrics` when present.
19
-
- Keep only `Build metrics` for build-only cases.
20
-
- Start output at the first `Development metrics:` or `Build metrics:` heading so earlier log noise is excluded.
21
-
- Do not recalculate rankings or rewrite the numbers manually.
22
-
5. Update `README.md`:
15
+
2. Resolve the canonical GitHub repository and default branch with `gh repo view`, then find the latest successful `Benchmark` workflow run on that branch unless the user gave a specific run ID.
16
+
3. List the jobs for that run and map each matrix case to its job ID before touching `README.md`.
17
+
4. For each case job, extract only the final benchmark tables from the log using the robust shell pipeline below.
18
+
5. Update `README.md` carefully:
23
19
- Replace the GitHub Actions run URL with the latest run URL.
24
20
- Replace the date with today's local date.
25
21
- Replace each case table with the extracted table from the matching job.
26
-
-If a case section exists without a table, insert the extracted table under that case.
27
-
-Preserve surrounding prose unless the benchmark output itself changes the tool names or versions shown in the tables.
28
-
6.Validate the result:
22
+
-Preserve the case heading, prose, command block, and the final `---` separator before `## Run locally`.
23
+
-Do not recalculate rankings or rewrite the numbers manually.
24
+
6.Validation is required after the edit:
29
25
- Every case in the workflow matrix is represented in `README.md`.
30
26
- No duplicated headings, tables, or rows were introduced.
27
+
- The separator before `## Run locally` is still present.
31
28
- The diff only changes benchmark data, the run URL, and the date.
32
29
33
30
## Commands
34
-
Prefer `gh` because it is authenticated and exposes both run metadata and logs.
31
+
Prefer `gh` because it is authenticated and exposes both run metadata and logs. Use these to execute or debug the workflow manually.
35
32
36
33
Resolve the canonical repository and default branch:
37
34
@@ -58,18 +55,35 @@ Fetch jobs for a run:
58
55
gh api repos/<owner>/<repo>/actions/runs/<run_id>/jobs --paginate
59
56
```
60
57
58
+
Map case names to job IDs:
59
+
60
+
```bash
61
+
gh api repos/<owner>/<repo>/actions/runs/<run_id>/jobs --paginate \
- Do not rely on the second log column being `Run Benchmark`. Current `gh run view --log` output may label lines as `UNKNOWN STEP`, while the third column still contains the benchmark output you need.
78
+
- Capture starts at the first `Development metrics:` or `Build metrics:` heading so preamble noise is excluded.
79
+
- Stop at `Post job cleanup.` so cleanup lines do not leak into the tables.
80
+
- Keep both `Development metrics` and `Build metrics` when present, and only `Build metrics` for build-only cases.
81
+
- Prefer replacing one case section at a time or using a temporary one-off local command; do not add repository scripts just to complete a single sync.
82
+
- The brittle part of the edit is preserving section boundaries, especially the final `---` before `## Run locally`.
83
+
71
84
## Failure handling
72
85
- If no successful `Benchmark` run exists, stop and report that blocker.
73
86
- If a job log is missing or truncated, stop and report which case could not be extracted.
74
87
- If the workflow matrix and README sections do not match, update only the cases backed by logs and call out the mismatch clearly.
75
88
- If `README.md` already points to the latest successful run and the extracted tables match, the expected result is an empty diff.
89
+
- If the workflow breaks down, include the failing step in the report: repo resolution, run lookup, job mapping, log extraction, README replacement, or structural validation.
0 commit comments