Parakeet multitalker #185
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: rust-ci | |
| # Every-PR gates for the Rust bindings (transcribe-cpp-sys + transcribe-cpp). | |
| # Thin per-binding workflow on the shared rails laid by bindings-shared-infra: | |
| # the binding-agnostic C contracts are certified in native-ci.yml; this file | |
| # adds only what the Rust layer introduces. | |
| # | |
| # - rust-gates: bindgen drift check (pinned to include/transcribe.abihash), | |
| # version-sync, the cargo-package content/size audit, and | |
| # rustfmt. No native build — fast, runs everywhere. | |
| # - rust-build: the real source build (build.rs drives the vendored CMake | |
| # tree, links from lib/transcribe-link.json) across the posture | |
| # matrix on linux + macos + windows, then the -sys smoke tests | |
| # (transcribe_version / abi-struct-size through the FFI), the | |
| # safe-crate conformance suite, and clippy. Postures: STATIC | |
| # (default), SHARED (shared libtranscribe), and DYNAMIC-BACKENDS | |
| # (linux: shared + loadable per-ISA backend modules, exercised | |
| # through transcribe_init_backends from Rust). | |
| # | |
| # Two test tiers (requirements §4): the no-model tests (version/ABI/error | |
| # mapping/device discovery, in tests/no_model.rs) always run, including on | |
| # forks. The model-gated tests (real transcription, streaming, cancel, family | |
| # extensions) un-skip only when the canary GGUFs are fetched — the same | |
| # fetch-canary composite action and HF_TOKEN gate the Python lane uses — and | |
| # `return` early (a clean skip) otherwise. One `cargo test` invocation runs | |
| # both tiers; the model tier self-skips when the canary env is absent. | |
| # | |
| # Posture cost split: the model tier (canary fetch + the model-gated tests + the | |
| # examples) runs ONLY on the representative *-static leg of each OS. The *-shared | |
| # and *-dynamic-backends legs are LINK-posture proofs, not marshalling proofs | |
| # (the marshalling code is byte-identical across postures), so they run build + | |
| # the -sys FFI smoke + the always-on no-model tier (which still covers the | |
| # dynamic-backends module load) and skip the canary fetch — dropping the | |
| # duplicated model-tier cost (chiefly the moonshine streaming conformance) from | |
| # the legs that can't catch a marshalling regression anyway. | |
| # | |
| # Windows joins here: the first cargo-driven MSVC build of this tree (static | |
| # posture). cl.exe comes from msvc-dev-cmd; the build has no external library | |
| # dependency to resolve (miniz is vendored into libtranscribe), so there is no | |
| # vcpkg / CMAKE_PREFIX_PATH setup. | |
| # | |
| # Path filters follow native-ci.yml's shape: the binding's own tree plus the | |
| # native paths it compiles from (binding behavior depends on the C side, so | |
| # do not narrow). Branch protection is OFF (project decision, 2026-06-13) — no | |
| # required-check maintenance comes with this workflow. | |
| on: | |
| push: | |
| branches: [main] | |
| paths: &paths | |
| - "bindings/rust/**" | |
| - "Cargo.toml" | |
| - "Cargo.lock" | |
| - ".cargo/**" | |
| - "src/**" | |
| - "include/**" | |
| - "ggml/**" | |
| - "cmake/**" | |
| - "CMakeLists.txt" | |
| - "CMakePresets.json" | |
| - "scripts/ci/rust_package_audit.py" | |
| - "bindings/python/_generate/check_version_sync.py" | |
| - ".github/workflows/rust-ci.yml" | |
| pull_request: | |
| paths: *paths | |
| workflow_dispatch: | |
| concurrency: | |
| group: ${{ github.workflow }}-${{ github.ref }} | |
| cancel-in-progress: true | |
| jobs: | |
| rust-gates: | |
| # Cheap, native-build-free gates. The drift check needs libclang (bindgen); | |
| # users never do, because the generated FFI is committed. | |
| runs-on: blacksmith-2vcpu-ubuntu-2404 | |
| steps: | |
| - uses: actions/checkout@v6 | |
| - uses: astral-sh/setup-uv@v8.2.0 # for the python gate scripts | |
| - uses: dtolnay/rust-toolchain@stable | |
| with: | |
| components: rustfmt | |
| - name: Install libclang (bindgen drift check) | |
| run: sudo apt-get update && sudo apt-get install -y libclang-dev clang | |
| - name: FFI drift gate (bindgen vs committed, pinned to transcribe.abihash) | |
| run: cargo xtask bindgen --check | |
| - name: rustfmt | |
| run: cargo fmt --all --check | |
| - name: Version sync (header <-> every active manifest) | |
| run: uv run --no-project python bindings/python/_generate/check_version_sync.py | |
| - name: Crate package audit (contents + 10 MB size cap) | |
| run: uv run --no-project python scripts/ci/rust_package_audit.py | |
| rust-build: | |
| name: rust-build (${{ matrix.label }}) | |
| strategy: | |
| fail-fast: false | |
| # Posture matrix: {static, shared} × {linux, macos, windows}, plus a | |
| # linux dynamic-backends leg. `features` is forwarded verbatim to the | |
| # build/test cargo commands ("" = the static default; "--features shared" | |
| # = a shared libtranscribe; "--features dynamic-backends" = shared + | |
| # loadable backend modules). The shared leg is the end-to-end proof of the | |
| # shared-link posture: its tests load libtranscribe at runtime via the | |
| # rpath the safe crate's build.rs emits on Unix, and on Windows (which has | |
| # no rpath) via the -sys build.rs staging transcribe.dll + the ggml DLLs | |
| # next to each cargo artifact. The dynamic-backends leg additionally proves | |
| # the loadable-module path: the build configures GGML_BACKEND_DL + | |
| # (x86) GGML_CPU_ALL_VARIANTS, and the tests/examples call | |
| # init_backends_default() to load the per-ISA CPU module and transcribe. | |
| matrix: | |
| include: | |
| - label: linux-static | |
| runner: blacksmith-2vcpu-ubuntu-2404 | |
| features: "" | |
| model_tier: true # representative leg for linux: full model tier | |
| - label: linux-shared | |
| runner: blacksmith-2vcpu-ubuntu-2404 | |
| features: "--features shared" | |
| model_tier: false # link-posture proof only (build + -sys smoke + no-model) | |
| # Loadable per-ISA backend modules (the multi-ISA CPU provider posture) | |
| # exercised through Rust: build.rs turns on BACKEND_DL + CPU_ALL_VARIANTS | |
| # on x86, the modules install into the -sys prefix, and the safe crate's | |
| # init_backends_default() loads them. CPU-only (no Vulkan SDK needed); | |
| # the C-level Vulkan degradation three-tier stays in native-ci. | |
| - label: linux-dynamic-backends | |
| runner: blacksmith-2vcpu-ubuntu-2404 | |
| features: "--features dynamic-backends" | |
| model_tier: false # no-model tier still covers the DL module load | |
| # macOS arm64 on the bare-metal M4 mini (all macOS arm64 CI on owned | |
| # hardware; the source build turns Metal on, and real hardware is the | |
| # trustworthy place to exercise it). Same posture as native-ci. | |
| - label: macos-arm64-static | |
| runner: [self-hosted, macOS, ARM64] | |
| features: "" | |
| model_tier: true # representative leg for macOS: full model tier | |
| - label: macos-arm64-shared | |
| runner: [self-hosted, macOS, ARM64] | |
| features: "--features shared" | |
| model_tier: false # link-posture proof only | |
| # Windows / MSVC (Blacksmith windows-2025, same image family as the | |
| # wheel lane). First cargo-driven MSVC build of this tree; no ccache | |
| # here (MSVC + ccache is fiddly), so every run is a cold ggml build — | |
| # hence the wider timeout. sccache is a later optimization. Both | |
| # postures: static is self-contained; shared links transcribe.dll (the | |
| # -sys build.rs stages the DLLs next to the test/example binaries). | |
| - label: windows-static | |
| runner: blacksmith-2vcpu-windows-2025 | |
| features: "" | |
| model_tier: true # representative leg for windows: full model tier | |
| - label: windows-shared | |
| runner: blacksmith-2vcpu-windows-2025 | |
| features: "--features shared" | |
| model_tier: false # link-posture proof only | |
| runs-on: ${{ matrix.runner }} | |
| timeout-minutes: 60 | |
| env: | |
| # Gates the model-test tier: present on this repo's branches, absent on | |
| # forks (where fetch-canary skips and the model tests skip cleanly). | |
| HF_TOKEN: ${{ secrets.HF_TOKEN }} | |
| steps: | |
| - uses: actions/checkout@v6 | |
| - uses: dtolnay/rust-toolchain@stable | |
| with: | |
| components: clippy | |
| - uses: astral-sh/setup-uv@v8.2.0 # fetch-canary fetches via uvx | |
| - name: Enable ccache compiler launcher (non-Windows) | |
| # CMake reads these at first configure, so build.rs's cmake-crate build | |
| # picks up ccache with no -D plumbing. NOT set on Windows: ccache isn't | |
| # installed there and an unresolvable launcher fails configure. | |
| if: runner.os != 'Windows' | |
| shell: bash | |
| run: | | |
| echo "CMAKE_C_COMPILER_LAUNCHER=ccache" >> "$GITHUB_ENV" | |
| echo "CMAKE_CXX_COMPILER_LAUNCHER=ccache" >> "$GITHUB_ENV" | |
| - name: Install build deps (Linux) | |
| if: runner.os == 'Linux' | |
| run: sudo apt-get update && sudo apt-get install -y cmake ninja-build ccache | |
| - name: Install build deps (macOS) | |
| if: runner.os == 'macOS' | |
| run: | | |
| brew install ninja | |
| command -v ccache >/dev/null || brew install ccache | |
| - name: Set up MSVC (cl.exe on PATH for the Ninja generator) | |
| # The cmake-crate build uses the Ninja generator (CMAKE_GENERATOR below); | |
| # Ninja needs cl.exe ambient. Without vcvars, CMake silently falls back | |
| # to the image's MinGW gcc (gcc-flavored M_PI errors) — same trap the | |
| # wheel lane documents. | |
| if: runner.os == 'Windows' | |
| uses: ilammy/msvc-dev-cmd@v1 | |
| with: | |
| arch: x64 | |
| - name: Install build deps (Windows) | |
| if: runner.os == 'Windows' | |
| shell: pwsh | |
| run: | | |
| choco install ninja --no-progress -y | |
| # cmake-crate honors CMAKE_GENERATOR from the environment. | |
| Add-Content $env:GITHUB_ENV "CMAKE_GENERATOR=Ninja" | |
| - name: CPU ISA signature (segregates ccache across the heterogeneous fleet) | |
| # ggml builds with -march=native in this source posture; ccache hashes | |
| # the literal flag, not the resolved ISA, so key the cache by the CPU's | |
| # feature flags (matches native-ci.yml). | |
| if: runner.os == 'Linux' | |
| run: echo "CPU_SIG=$(grep -m1 '^flags' /proc/cpuinfo | sha256sum | cut -c1-8)" >> "$GITHUB_ENV" | |
| - name: ccache (compile cache for the native source build) | |
| if: runner.os == 'Linux' # the mini is persistent; its local cache suffices | |
| uses: actions/cache@v5 | |
| with: | |
| path: ~/.cache/ccache | |
| key: ccache-rust-build-${{ matrix.label }}-${{ env.CPU_SIG }}-${{ github.sha }} | |
| restore-keys: ccache-rust-build-${{ matrix.label }}-${{ env.CPU_SIG }}- | |
| # Package-scoped (not --workspace): a single posture's native build, and | |
| # it sidesteps the backend features choking on the xtask member, which has | |
| # none. xtask is compiled/run in rust-gates (the bindgen check). | |
| - name: Build (${{ matrix.label }} posture; build.rs drives CMake) | |
| run: cargo build -p transcribe-cpp ${{ matrix.features }} --verbose | |
| - name: "-sys smoke (transcribe_version / abi size through the FFI)" | |
| run: cargo test -p transcribe-cpp-sys ${{ matrix.features }} | |
| # The canary GGUFs upgrade the safe-crate suite from no-model gates to real | |
| # transcription/streaming/cancel/extension coverage. Guarded on HF_TOKEN | |
| # (forks have none): without it the model tests skip cleanly, leaving the | |
| # always-on no-model tier. jfk.wav ships in-repo, so only the two model | |
| # paths need exporting (fetch-canary handles that). | |
| # Model tier runs only on the representative *-static leg per OS. The | |
| # shared / dynamic-backends legs skip the canary fetch, so the model-gated | |
| # tests in the conformance step below self-skip and only the no-model tier | |
| # runs there (build + link posture + FFI surface proof). | |
| - if: matrix.model_tier | |
| uses: ./.github/actions/fetch-canary | |
| with: | |
| hf-token: ${{ secrets.HF_TOKEN }} | |
| - name: Safe-crate conformance (no-model tier always; model tier when canary present) | |
| run: cargo test -p transcribe-cpp ${{ matrix.features }} | |
| # The 5 canonical examples (requirements §6), run headless under the same | |
| # canary skip rules as the model tests: each resolves its model/audio from | |
| # the TRANSCRIBE_SMOKE_* env fetch-canary exports (or skips cleanly + exits | |
| # 0). shell:bash so the loop is identical on the windows runner (Git Bash). | |
| - name: Examples (CI-executed; §6 Rosetta set) | |
| if: matrix.model_tier # examples need the canary models; only the model-tier legs have them | |
| shell: bash | |
| run: | | |
| for ex in transcribe-file streaming batch backend-select error-handling; do | |
| echo "=== example: $ex ===" | |
| cargo run -p transcribe-cpp --example "$ex" ${{ matrix.features }} | |
| done | |
| - name: Clippy (deny warnings) | |
| # Once is enough; the linux-static leg covers it (clippy on another | |
| # leg would force a second full native build for no extra lint signal). | |
| if: matrix.label == 'linux-static' | |
| run: cargo clippy --workspace --all-targets -- -D warnings | |
| - name: ccache stats | |
| if: runner.os == 'Linux' | |
| run: ccache -s | head -8 |