Skip to content

Commit 863f728

Browse files
physicsrobclaude
andcommitted
CI: per-push job is a fast compile-and-generate smoke, not the full suite
The full suite on a 2-core runner takes ~an hour — not viable per push. The `test` job becomes `smoke` (scripts/ci_smoke.py: compile binary_increment, generate through a transformers pipeline, assert the output — the whole stack in about a minute). The full CPU suite moves to full-tests.yml, weekly + workflow_dispatch. The require-ci ruleset now requires lint + smoke. Test Suite Status: 1615 passed, 0 failed, 0 skipped (--device cpu). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 2b5dafa commit 863f728

4 files changed

Lines changed: 77 additions & 5 deletions

File tree

.github/workflows/ci.yml

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,13 +30,19 @@ jobs:
3030
- name: Lint
3131
run: make lint
3232

33-
test:
33+
# Fast end-to-end signal on every push: compile an example on CPU,
34+
# generate, assert the output. The full suite is far too slow for
35+
# per-push CI (~1h on a 2-core runner) — it runs on Modal during
36+
# development (make test) and weekly via full-tests.yml.
37+
smoke:
3438
runs-on: ubuntu-latest
35-
timeout-minutes: 60
39+
timeout-minutes: 20
3640
steps:
3741
- uses: actions/checkout@v4
3842
- uses: astral-sh/setup-uv@v6
3943
with:
4044
enable-cache: true
41-
- name: Test (CPU)
42-
run: make test-ci
45+
- name: Sync dev environment
46+
run: uv sync
47+
- name: Compile-and-generate smoke
48+
run: uv run --no-sync python -m scripts.ci_smoke

.github/workflows/full-tests.yml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
name: Full tests
2+
3+
# The full suite on a CPU runner takes on the order of an hour — far too
4+
# slow for per-push CI (that's the smoke job in ci.yml). This runs it
5+
# weekly as a drift backstop, and on demand.
6+
7+
on:
8+
schedule:
9+
- cron: "17 6 * * 1" # Mondays 06:17 UTC
10+
workflow_dispatch:
11+
12+
env:
13+
UV_PYTHON: "3.12"
14+
UV_LOCKED: "1"
15+
16+
jobs:
17+
test:
18+
runs-on: ubuntu-latest
19+
timeout-minutes: 180
20+
steps:
21+
- uses: actions/checkout@v4
22+
- uses: astral-sh/setup-uv@v6
23+
with:
24+
enable-cache: true
25+
- name: Test (CPU, full suite)
26+
run: make test-ci

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ test: check-modal-lock
6060
test-logs:
6161
@tail -f /tmp/torchwright-test.log
6262

63-
# CI entry point (.github/workflows/ci.yml): full suite on the runner's CPU,
63+
# CI entry point (.github/workflows/full-tests.yml): full suite on the runner's CPU,
6464
# synced from the standalone lock like the Modal image. Guarded to CI because
6565
# inside the umbrella workspace `uv sync` operates on the shared venv instead
6666
# (dropping opt-in packages like torchwright_doom's onnxruntime-gpu), and a

scripts/ci_smoke.py

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
"""End-to-end CI smoke: compile an example, generate, check the output.
2+
3+
Compiles ``examples/binary_increment.py`` to a Hugging Face bundle and
4+
verifies that a ``transformers`` pipeline loaded from it increments a
5+
binary string. This exercises the full stack — graph construction,
6+
scheduling, weight writing, bundle export, tokenizer, generation —
7+
in about a minute on a CPU runner, standing in for the full suite on
8+
every push (the full suite runs on Modal locally and on the weekly
9+
full-tests workflow).
10+
"""
11+
12+
import sys
13+
import tempfile
14+
from pathlib import Path
15+
16+
17+
def main() -> int:
18+
from examples.binary_increment import D_HEAD, D_MODEL, create_network_parts
19+
from torchwright import compile_hf_bundle
20+
21+
output_node, embedding = create_network_parts()
22+
with tempfile.TemporaryDirectory() as tmp:
23+
bundle = Path(tmp) / "binary_increment_hf_bundle"
24+
compile_hf_bundle(output_node, embedding, str(bundle), d=D_MODEL, d_head=D_HEAD)
25+
26+
from transformers import pipeline
27+
28+
generate = pipeline("text-generation", model=str(bundle))
29+
out = generate("1011\n", return_full_text=False)[0]["generated_text"]
30+
31+
print(f"generated: {out!r}")
32+
if out.strip() != "1100":
33+
print("FAIL: expected '1100'", file=sys.stderr)
34+
return 1
35+
print("smoke OK")
36+
return 0
37+
38+
39+
if __name__ == "__main__":
40+
sys.exit(main())

0 commit comments

Comments
 (0)