Skip to content

CI stability: the test suites don't run in CI — test regressions merge undetected (only CodeQL gates) #452

@MartinCastroAlvarez

Description

@MartinCastroAlvarez

Finding (CI stability / regression-safety audit)

CI does not run the test suites. The only PR checks are CodeQL (Analyze (python), Analyze (javascript-typescript)). pytest and the frontend vitest suite run locally only (scripts/lint.sh / pre-commit), per the deliberate "local-only gate" posture (SECURITY.md §8). So a PR that breaks a test merges green — nothing on the server runs the tests.

This is not hypothetical: #401 (bulk-PATCH list_editable scope guard) merged with passing checks but broke tests/test_logentry.py::test_bulk_patch_emits_one_change_per_row on main (that test's bulk PATCH didn't set list_editable). It went undetected until a later full local run. (Fixed in #451.)

With many agents merging in parallel and only CodeQL gating, test regressions will keep slipping onto main.

Proposed

Add a CI workflow (or extend the existing setup) that runs the gate on every PR and blocks merge on failure:

  • poetry run pytest (backend) — the 488-test suite + coverage threshold already in pyproject/ACCEPTANCE.md.
  • pnpm -r typecheck, pnpm lint, pnpm test, pnpm -r build (frontend).
  • Optionally the Python lint gate (ruff/black/isort/mypy) that scripts/lint.sh runs locally.

Make these required status checks (branch protection) so a red suite can't merge. SHA-pin the actions (consistent with the OIDC release workflow #365).

This is Tier-5 (.github/workflows + branch protection) → human-reviewed. Relates to / likely a concrete sub-task of #331 (CI/release hardening).

Acceptance

  • PRs run pytest + the frontend gate; a failing test blocks merge.
  • Checks are required in branch protection.
  • Actions SHA-pinned.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions