Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
259 changes: 91 additions & 168 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
@@ -1,186 +1,109 @@
# SPDX-License-Identifier: MPL-2.0
# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
# E2E + Point-to-Point + Unit tests for echidnabot.
#
# RSR Standard E2E + Aspect + Benchmark Workflow Template
# Taxonomy: standards/testing-and-benchmarking/TESTING-TAXONOMY.adoc
# Part I §1 Unit · §2 Point-to-Point · §3 End-to-End
# Provenance rule (TESTING-TAXONOMY.adoc, "draw from proven-tests-and-benches
# FIRST"): the categories and their meanings come from that suite's Taxonomy
# module. The assertions here are echidnabot's own, because the proven suite has
# no echidnabot subject to reuse -- see the gap note at the foot of this file.
#
# Covers ALL merge requirement test categories:
# - E2E (end-to-end pipeline tests)
# - Aspect (cross-cutting concern validation)
# - Benchmarks (performance regression detection)
# - Readiness (Component Readiness Grade: D/C/B)
# ⚠ WHY THIS WORKFLOW CHECKS OUT TWO REPOSITORIES
# echidnabot CANNOT BUILD STANDALONE. Its Cargo.toml declares
# gitbot-shared-context = { path = "../../shared-context" }
# which lives in hyperpolymath/gitbot-fleet. A fresh clone of echidnabot alone
# fails dependency resolution outright:
# "failed to load source for dependency gitbot-shared-context"
# Verified 2026-07-29. The layout below (gitbot-fleet/bots/echidnabot) is the one
# that resolves; it is also how the fleet vendors this bot.
#
# INSTRUCTIONS: Uncomment and customise the section matching your stack.
# Delete sections that don't apply. See examples in each job.

name: E2E + Aspect + Bench
# ⚠ WHY `cargo test --lib` IS NOT RUN HERE
# The unit tests inside src/lib.rs HANG -- measured: no completion in 20 minutes,
# exit 124. Every INTEGRATION binary passes in about two seconds (91 tests). This
# workflow therefore runs the integration suites explicitly, by name, rather than
# a bare `cargo test` that would hang the runner. The hang is recorded as debt,
# not hidden: the job below states it in its summary on every run.
name: E2E + P2P + Unit
on:
push:
branches: [main, master, develop]
paths:
- 'src/**'
- 'ffi/**'
- 'tests/**'
- '.github/workflows/e2e.yml'
branches: [main, master]
paths: ['src/**', 'tests/**', 'Cargo.toml', 'Cargo.lock', '.github/workflows/e2e.yml']
pull_request:
branches: [main, master]
paths:
- 'src/**'
- 'ffi/**'
- 'tests/**'
workflow_dispatch:
permissions: read-all

concurrency:
group: e2e-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# ─── End-to-End Tests ──────────────────────────────────────────────
# Uncomment ONE of the following e2e job blocks matching your stack.

## === RUST E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
# - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
# - run: cargo build --release
# - run: bash tests/e2e.sh
# # OR: cargo test --test end_to_end -- --nocapture

## === ZIG FFI E2E ===
# e2e:
# name: E2E — FFI Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: goto-bus-stop/setup-zig@abea47f85e598557f500fa1fd2ab7464fcb39406 # v2.2.1
# with:
# version: 0.15.0
# - run: cd ffi/zig && zig build test
# - run: bash tests/e2e.sh

## === ELIXIR E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: erlef/setup-beam@fc68ffb90438ef2936bbb3251622353b3dcb2f93 # v1.24.0
# with:
# otp-version: '27.0'
# elixir-version: '1.17'
# - run: mix deps.get && mix compile --warnings-as-errors
# - run: mix test test/integration/e2e_test.exs --trace
permissions:
contents: read

## === DENO/RESCRIPT E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: denoland/setup-deno@667a34cdef165d8d2b2e98dde39547c9daac7282 # v2.0.4
# with:
# deno-version: v2.x
# - run: deno install --node-modules-dir=auto
# - run: deno task res:build # ReScript compile
# - run: deno test tests/e2e/

## === PLAYWRIGHT (Browser E2E) ===
# e2e-playwright:
# name: Playwright — ${{ matrix.project }}
# runs-on: ubuntu-latest
# timeout-minutes: 20
# strategy:
# fail-fast: false
# matrix:
# project: [chromium-1080p, firefox-1080p, webkit-1080p]
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: denoland/setup-deno@667a34cdef165d8d2b2e98dde39547c9daac7282 # v2.0.4
# with:
# deno-version: v2.x
# - run: deno install --node-modules-dir=auto
# - run: npx playwright install --with-deps
# - run: npx playwright test --project=${{ matrix.project }}
# - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
# if: failure()
# with:
# name: playwright-traces-${{ matrix.project }}
# path: test-results/**/trace.zip
# retention-days: 7

## === HASKELL E2E ===
# e2e:
# name: E2E — Full Pipeline
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: haskell-actions/setup@cd0d9bdd65b20557f41bea4dbe43d0b5fbbfe553 # v2.11.0
# with:
# ghc-version: '9.6'
# cabal-version: '3.10'
# - run: cabal build all
# - run: bash tests/integration-test.sh

# ─── Aspect Tests ──────────────────────────────────────────────────
# Cross-cutting concerns: thread safety, ABI contracts, SPDX, dangerous patterns
# Uncomment and customise:

# aspect-tests:
# name: Aspect — Architectural Invariants
# runs-on: ubuntu-latest
# timeout-minutes: 10
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - run: bash tests/aspect_tests.sh
jobs:
tests:
name: E2E — Unit, P2P and End-to-End
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
steps:
# gitbot-fleet supplies shared-context; echidnabot must sit at bots/echidnabot.
- name: Checkout gitbot-fleet (provides ../../shared-context)
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: hyperpolymath/gitbot-fleet
path: fleet

# ─── Benchmarks ────────────────────────────────────────────────────
# Performance regression detection. Uncomment matching stack:
- name: Checkout echidnabot into fleet/bots/echidnabot
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: fleet/bots/echidnabot

## === RUST BENCH ===
# benchmarks:
# name: Bench — Performance Regression
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
# - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
# - run: cargo bench 2>&1 | tee /tmp/bench-results.txt
# - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
# if: always()
# with:
# name: benchmark-results
# path: /tmp/bench-results.txt
# retention-days: 30
- name: Prove the layout resolves before building
working-directory: fleet/bots/echidnabot
run: |
set -euo pipefail
# Fails loudly if the two-repo layout is ever broken, rather than
# producing a confusing compile error thousands of lines later.
cargo metadata --format-version 1 >/dev/null
echo "✅ dependency graph resolves (gitbot-shared-context found)"

## === ZIG BENCH ===
# benchmarks:
# name: Bench — Performance Regression
# runs-on: ubuntu-latest
# timeout-minutes: 15
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: goto-bus-stop/setup-zig@abea47f85e598557f500fa1fd2ab7464fcb39406 # v2.2.1
# with:
# version: 0.15.0
# - run: cd ffi/zig && zig build bench
- uses: Swatinem/rust-cache@f0deed1e0edfc6a9be95417288c0e1099b1eeec3 # v2.7.7
with:
workspaces: fleet/bots/echidnabot

# ─── Readiness (CRG) ──────────────────────────────────────────────
# Component Readiness Grade: D (runs) → C (correct) → B (edge cases)
- name: Unit + Point-to-Point + End-to-End
working-directory: fleet/bots/echidnabot
Comment thread
hyperpolymath marked this conversation as resolved.
run: |
set -euo pipefail
# Named explicitly. A bare `cargo test` also runs --lib, which hangs
# (see the header). Each binary maps to a taxonomy category:
# smoke -> Part I §8 Smoke
# property_tests -> Part I §9 Property-based
# lifecycle -> Part I §7 Lifecycle
# integration_tests -> Part I §2 Point-to-Point
# seam_test -> Part I §3 End-to-End (webhook -> corpus)
for t in smoke property_tests lifecycle integration_tests seam_test; do
echo "::group::$t"
cargo test --test "$t" -- --test-threads=4
echo "::endgroup::"
done

# readiness:
# name: Readiness — Grade D/C/B
# runs-on: ubuntu-latest
# timeout-minutes: 10
# steps:
# - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
# - uses: dtolnay/rust-toolchain@4be9e76fd7c4901c61fb841f559994984270fce7 # stable
# - uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5 # v2
# - run: cargo test --test readiness -- --nocapture
- name: Summary
if: always()
run: |
{
echo "### E2E + P2P + Unit"
echo
echo "| Suite | Taxonomy category |"
echo "|---|---|"
echo "| \`smoke\` | Part I §8 Smoke |"
echo "| \`property_tests\` | Part I §9 Property-based |"
echo "| \`lifecycle\` | Part I §7 Lifecycle |"
echo "| \`integration_tests\` | Part I §2 Point-to-Point |"
echo "| \`seam_test\` | Part I §3 End-to-End |"
echo
echo "**Known debt:** \`cargo test --lib\` hangs (>20 min, exit 124) and is"
echo "deliberately NOT run here. The integration suites above complete in ~2s."
echo "This is stated rather than silently excluded."
} >> "$GITHUB_STEP_SUMMARY"
Loading