Skip to content

Commit ccbd402

Browse files
harden(ci): require hermetic lifecycle smoke subset on pull-request CI (#2247)
* ci: promote hermetic lifecycle smoke subset to required PR checks Add a new required "Lifecycle Smoke (Linux)" job to ci.yml, wired into merge-gate.yml's EXPECTED_CHECKS, running the smallest stable hermetic subset of the real Consume/Produce/Govern lifecycle contracts at PR time -- a gap since tests/integration/ previously only ran on merge_group. Targets (12 tests, ~13s bare pytest, ~60s projected job wall time, timeout-minutes: 3): - test_install_content_hash_roundtrip.py (component, critical_suite.toml) - test_policy_pinned_constraint_e2e.py (component, critical_suite.toml) - test_virtual_claude_skill_lock_convergence.py (component, critical_suite.toml) - test_virtual_package_lifecycle_matrix.py (direct #2240 regression) - test_architecture_authorities.py::test_ado_lock_coordinates_have_single_owner (direct #2226 AC14 static guard) All five are hermetic: IsolatedApmEnvironment denies all AF_INET/ AF_INET6 socket use (incl. loopback), no requires_apm_binary/ requires_github_token/requires_ado_pat markers, no secrets wired into the new job. tests/quality/test_ci_topology.py gains a positive guard (job name, timeout ceiling, exact target tuple derived dynamically from critical_suite.toml's `component`-marked entries plus the two direct regression targets, no forbidden credential env, EXPECTED_CHECKS membership plus four mutation/negative tests proving the guard actually catches drift (dropped target, swapped AC14 node-id, injected credential env, removed from EXPECTED_CHECKS). #2238 is intentionally not duplicated here: its regression coverage is unit-level and already required via the existing shards. Updates docs/src/content/docs/contributing/integration-testing.md and CHANGELOG.md. No src/apm_cli/** changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75891045-fa50-40b7-9354-b4649195f724 EOF ) * docs(changelog): reference PR #2247 for lifecycle smoke entry Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75891045-fa50-40b7-9354-b4649195f724 * fold: apm-review-panel + apm-strategy recommendations for #2247 apm-strategy (apm-ceo review): tighten the CHANGELOG entry to a single contributor-facing line per changelog.instructions.md's one-line-per-PR convention, and fix a stray non-ASCII em-dash. apm-review-panel (9-persona fan-out, CEO synthesis: ship_with_followups, no blocking findings): fold all 5 curated follow-ups. 1. [test-coverage-expert / python-architect / supply-chain-security-expert, corroborated + empirically proven] Close the hermetic guard's blind spots in tests/quality/test_ci_topology.py: - job-level env: was never checked (GitHub Actions inherits it to every step); add the check + test_lifecycle_smoke_job_level_credential_env_fails. - step with: blocks could smuggle a credential expression past the env/run-only scan; add the check + test_lifecycle_smoke_with_block_credential_fails. - the github.token expression alias resolves to the same automatic token but doesn't match any FORBIDDEN_CREDENTIAL_ENV substring; add FORBIDDEN_CREDENTIAL_EXPRESSIONS + test_lifecycle_smoke_github_token_expression_alias_fails. Full suite: 18/18 passing (was 15/15). 2. [devx-ux-expert / doc-writer] Add the exact copy-paste local-repro command for the 5-target lifecycle-smoke subset to integration-testing.md, so a contributor whose PR fails the new required check has a documented one-liner. 3. [oss-growth-hacker] Update CONTRIBUTING.md's contributor-facing CI-tier summary (the only place a first-time external contributor learns what checks to expect) to mention the new Lifecycle Smoke check, framed as PR-time feedback rather than an unexplained new gate. 4. [doc-writer] Add symmetric "Lifecycle Smoke Verifies" / "Lifecycle Smoke Failures" entries to integration-testing.md (Tiers 1/2 already had both), plus a cross-reference to test_ci_topology.py as the drift guard, and fix the file-list's inconsistent tests/integration/ prefix. All 9 panelists returned no blocking-severity findings; CEO stance is ship_with_followups, all curated follow-ups folded in this commit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75891045-fa50-40b7-9354-b4649195f724 * refactor(ci): select lifecycle smoke via registered lifecycle_smoke marker Supersedes explicit file/node-id targeting with a declarative @pytest.mark.lifecycle_smoke marker so an emptied family fails loudly (pytest exit code 5, 'no tests collected') instead of a path list silently shrinking. - Register lifecycle_smoke in pyproject.toml's pytest markers list. - Mark the four lifecycle modules at module level (test_install_content_hash_roundtrip.py, test_policy_pinned_constraint_e2e.py, test_virtual_claude_skill_lock_convergence.py, test_virtual_package_lifecycle_matrix.py) and, at function level, only test_architecture_authorities.py::test_ado_lock_coordinates_have_single_owner (the file's other 32 tests remain unmarked). - ci.yml's lifecycle-smoke job now runs `pytest --strict-markers -m lifecycle_smoke tests/integration`; rewrote the job's rationale comment to match. - tests/quality/test_ci_topology.py: replaced the exact-target-list assertion and its two target-based mutation tests with semantic contracts -- marker registered (read from pyproject.toml's existing markers list, no new manifest constant), command uses --strict-markers/-m lifecycle_smoke/bounded tests/integration root, job stays required and hermetic, and the marker family collects non-empty right now -- backed by six mutation tests proving each contract is load-bearing (dropped -m expression, wrong marker string, unbounded root, dropped --strict-markers, unregistered marker, empty family). - Docs: integration-testing.md's Tier 3 section and marker-axis table, CONTRIBUTING.md, and CHANGELOG.md updated to describe marker-based selection and the measured ~65-70s job wall time (was ~60s; the full tests/integration/ directory scan for marker deselection adds ~2-8s of collection overhead vs the prior five explicit targets). Verified: marker-based collection selects the identical 12 tests as the prior explicit-target list (parity confirmed via --collect-only diff). Re-ran both failure-injection proofs (pre-#2226 lockfile.py, pre-#2240 update.py) against the new marker-based command -- both still fail exactly as before. Confirmed marked tests are unaffected in ci-integration.yml's merge_group integration job (no -m filter there). Full lint mirror (ruff check/format, pylint R0801, lint-auth-signals.sh, lint-architecture-boundaries.sh, actionlint) clean. tests/quality/test_ci_topology.py + test_test_taxonomy.py + test_architecture_authorities.py: 67 passed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75891045-fa50-40b7-9354-b4649195f724 * feat(ci): promote prune-hook reconciliation [claude] to lifecycle_smoke Adds pytest.mark.lifecycle_smoke to only the `claude` parametrization of test_prune_removes_merged_hook_entries_and_sidecar (deliberately excluding the `cursor` sibling), now that #2249's prune-hook fix has merged. The existing marker-driven `-m lifecycle_smoke tests/integration` command in ci.yml auto-picks up the newly-marked test with zero workflow edit -- confirmed via `git diff` showing no change to the lifecycle-smoke job section. Family grows from 12 to 13 tests, all passing (13 passed, ~16.5-17.9s local across 3 repeated runs -- comparable to the prior 12-test baseline of ~17.7-18.4s). Failure-injection re-proof: reverting #2249's production fix (prune.py, hook_integrator.py, uninstall/engine.py, apm_package.py) to pre-fix content causes exactly `[claude]` to fail under `-m lifecycle_smoke` selection (12 passed, 1 failed), confirming the gate is load-bearing for this regression; restored cleanly afterward (working tree verified clean pre/post). Also merges origin/main (additive merge commit, no rebase/force-push) to bring in #2249 plus intervening #2237 (Windows CI fix) and #2246 (lockfile-property tests). Resolved one real conflict in merge-gate.yml's EXPECTED_CHECKS where main's #2237 follow-on independently added a new required "Windows Compatibility Gate" check to the same ternary our branch extended with "Lifecycle Smoke (Linux)" -- both required-check names now coexist in both event branches. Docs (integration-testing.md, CHANGELOG.md) updated to enumerate the 6th target and prune's Consume/Produce/Govern coverage. No change needed to test_ci_topology.py's semantic assertions (count-agnostic by design) or ci.yml (marker-driven auto-pickup, zero edit). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 75891045-fa50-40b7-9354-b4649195f724 --------- Co-authored-by: danielmeppiel <danielmeppiel@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 3fbb624 commit ccbd402

13 files changed

Lines changed: 560 additions & 13 deletions

.github/workflows/ci.yml

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -401,3 +401,129 @@ jobs:
401401
# .agents/skills/ content could silently diverge from source (#1716).
402402
- name: apm audit --ci
403403
run: uv run apm audit --ci
404+
405+
# Promotes the smallest stable hermetic slice of the real Consume/Produce/
406+
# Govern lifecycle contracts to a PR-time required check. Tier 1.
407+
#
408+
# Why this job exists
409+
# --------------------
410+
# Every module under tests/integration/ -- including the curated
411+
# producer/consumer/govern contracts in tests/quality/critical_suite.toml
412+
# and the regression coverage added by #2226 (ADO lock coordinate
413+
# preservation) and #2240 (virtual/manifestless lifecycle convergence) --
414+
# only ran on merge_group (ci-integration.yml), never as a required PR
415+
# check. A revert of either fix could sit un-caught until a PR reached the
416+
# merge queue. This job closes that gap for the subset that is both
417+
# hermetic and fast enough for the PR-time critical path.
418+
#
419+
# Declarative selection via a registered pytest marker
420+
# -----------------------------------------------------
421+
# The family is selected by `-m lifecycle_smoke`, not by an explicit
422+
# file/node-id list. `@pytest.mark.lifecycle_smoke` (registered in
423+
# pyproject.toml's [tool.pytest.ini_options] markers, enforced by
424+
# --strict-markers) is applied at the module level to
425+
# test_install_content_hash_roundtrip.py, test_policy_pinned_constraint_e2e.py,
426+
# test_virtual_claude_skill_lock_convergence.py, and
427+
# test_virtual_package_lifecycle_matrix.py, and at the function level to
428+
# ONLY test_architecture_authorities.py::test_ado_lock_coordinates_have_single_owner
429+
# (the exact #2226 AC14 static guard pinning that ADO coordinates are
430+
# derived by DependencyReference and never persisted in the lockfile --
431+
# the file's other 32 tests are deliberately unmarked; four of them shell
432+
# out to a subprocess pytest run (~30s each) and would blow the time
433+
# budget without adding coverage for this PR's regressions).
434+
#
435+
# This is the load-bearing reason for marker-based selection over an
436+
# explicit-path list: if every `lifecycle_smoke` mark were ever stripped
437+
# (accidentally or via a bad merge), `-m lifecycle_smoke` collects zero
438+
# tests and pytest exits 5 ("no tests collected") -- a hard, structural
439+
# CI failure, not a silent no-op. An explicit path list degrades instead:
440+
# a dropped/renamed target just quietly shrinks what runs.
441+
#
442+
# Why these four modules + one function, Consume/Produce/Govern coverage
443+
# ------------------------------------------------------------------------
444+
# - test_install_content_hash_roundtrip.py, test_policy_pinned_constraint_e2e.py:
445+
# already-curated `component`-tier Consume/Govern contracts from
446+
# critical_suite.toml; no built binary, no network.
447+
# - test_virtual_claude_skill_lock_convergence.py: `component`-tier
448+
# Produce/Consume lock-convergence contract from critical_suite.toml,
449+
# directly adjacent to the #2226 ADO lock-coordinate fix.
450+
# - test_virtual_package_lifecycle_matrix.py: the direct #2240
451+
# regression test (install/lock/frozen-install/update/audit matrix for
452+
# virtual + manifestless packages). Not in critical_suite.toml (it
453+
# carries only `pytest.mark.integration`, not a taxonomy behavioral
454+
# marker), so no manifest change is needed to mark it.
455+
# - test_architecture_authorities.py::test_ado_lock_coordinates_have_single_owner:
456+
# see above.
457+
#
458+
# #2238 (bounded best-effort ref-lookup retries) is intentionally NOT
459+
# duplicated here: its regression coverage
460+
# (tests/unit/deps/test_download_strategies_phase3.py,
461+
# tests/unit/deps/test_git_reference_resolver.py) is unit-level and
462+
# already required today via build-and-test-shard.
463+
#
464+
# Why this stays hermetic, network-free, and credential-free
465+
# ------------------------------------------------------------
466+
# None of the twelve tests this marker currently collects carry
467+
# requires_apm_binary, requires_github_token, requires_ado_pat, or any
468+
# other network/credential precondition marker.
469+
# tests/utils/isolated_apm_environment.py's socket guard denies ALL
470+
# AF_INET/AF_INET6 use (including loopback) inside the subprocess these
471+
# tests drive; git remotes are rewritten to local bare repos via
472+
# `git config url.<path>.insteadOf`. This job declares no `secrets:` and no
473+
# GITHUB_APM_PAT/ADO_APM_PAT env, so it is structurally unable to depend on
474+
# credentials.
475+
#
476+
# Why `tests/integration` as the collection root, not `tests/`
477+
# ----------------------------------------------------------------
478+
# Bounding collection to tests/integration keeps `-m lifecycle_smoke` from
479+
# ever silently picking up a same-named marker use added under
480+
# tests/unit or tests/quality in the future; it also keeps collection
481+
# cost proportional to the directory that actually owns this family.
482+
#
483+
# Why timeout-minutes: 3
484+
# ------------------------
485+
# Before this refactor (explicit 5-target list), measured cold wall time
486+
# was ~16-19s bare pytest. After moving to marker-based selection over
487+
# the full tests/integration/ directory (299 files), measured cold wall
488+
# time is ~18-27s bare pytest -- the marker scan adds a real but bounded
489+
# +2 to +8s of collection overhead versus enumerating five paths
490+
# directly. With checkout + setup-python + setup-uv + `uv sync --extra
491+
# dev` overhead (~45-50s, per the comparable Test Architecture Ratchets
492+
# job), total job wall time is projected at ~63-70s -- still within the
493+
# 60-90s target with adequate headroom. 3 minutes gives ample
494+
# runner-variance headroom while still failing fast and loud if this
495+
# suite ever regresses toward materially slower (e.g. an accidental real
496+
# network call retrying under the hood, or the marker family silently
497+
# growing unbounded).
498+
#
499+
# tests/quality/test_ci_topology.py guards this job's name, the marker's
500+
# registration, the command's use of --strict-markers/-m/bounded root,
501+
# required-check membership, and that the marker family is non-empty
502+
# right now -- against silent drift.
503+
lifecycle-smoke:
504+
name: Lifecycle Smoke (Linux)
505+
runs-on: ubuntu-24.04
506+
timeout-minutes: 3
507+
permissions:
508+
contents: read
509+
510+
steps:
511+
- uses: actions/checkout@v4
512+
513+
- name: Set up Python
514+
uses: actions/setup-python@v5
515+
with:
516+
python-version: ${{ env.PYTHON_VERSION }}
517+
518+
- name: Install uv
519+
uses: astral-sh/setup-uv@v6
520+
with:
521+
enable-cache: true
522+
523+
- name: Install dependencies
524+
run: uv sync --extra dev
525+
526+
- name: Run required lifecycle smoke subset
527+
run: |
528+
uv run --extra dev pytest -p no:cacheprovider -q --strict-markers \
529+
-m lifecycle_smoke tests/integration

.github/workflows/merge-gate.yml

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -126,13 +126,22 @@ jobs:
126126
# allocation latency on the critical path.
127127
# merge_group context: Coverage Combine IS required, so the
128128
# global 80% floor blocks the actual merge.
129-
# "Windows Compatibility Gate" is required at BOTH PR-time and
130-
# merge-queue-time: it is the only Windows signal that runs
131-
# before merge (build-release.yml's full Windows matrix only
132-
# runs post-merge on push to main), so it must gate the class
133-
# of Windows-only regressions fixed in microsoft/apm#2233
134-
# rather than surfacing only after landing on main.
135-
EXPECTED_CHECKS: ${{ github.event_name == 'merge_group' && 'Build & Test Shard 1 (Linux),Build & Test Shard 2 (Linux),Coverage Combine (Linux),Windows Compatibility Gate,APM Self-Check,NOTICE Drift Check,Build (Linux),Smoke Test (Linux),Integration Tests (Linux),Release Validation (Linux)' || 'Build & Test Shard 1 (Linux),Build & Test Shard 2 (Linux),Windows Compatibility Gate,APM Self-Check,NOTICE Drift Check' }}
129+
# Lifecycle Smoke (Linux) is required in BOTH contexts: it's a
130+
# hermetic, ~65-70s ci.yml job (runs on pull_request AND
131+
# merge_group, same as APM Self-Check / NOTICE Drift Check),
132+
# promoting the smallest stable subset of the real
133+
# Consume/Produce/Govern lifecycle contracts -- selected
134+
# declaratively via the `lifecycle_smoke` pytest marker,
135+
# including the direct #2226/#2240 regression tests -- onto
136+
# the PR-time critical path.
137+
# "Windows Compatibility Gate" is also required at BOTH
138+
# PR-time and merge-queue-time: it is the only Windows signal
139+
# that runs before merge (build-release.yml's full Windows
140+
# matrix only runs post-merge on push to main), so it must
141+
# gate the class of Windows-only regressions fixed in
142+
# microsoft/apm#2233 rather than surfacing only after landing
143+
# on main.
144+
EXPECTED_CHECKS: ${{ github.event_name == 'merge_group' && 'Build & Test Shard 1 (Linux),Build & Test Shard 2 (Linux),Coverage Combine (Linux),Windows Compatibility Gate,APM Self-Check,NOTICE Drift Check,Lifecycle Smoke (Linux),Build (Linux),Smoke Test (Linux),Integration Tests (Linux),Release Validation (Linux)' || 'Build & Test Shard 1 (Linux),Build & Test Shard 2 (Linux),Windows Compatibility Gate,APM Self-Check,NOTICE Drift Check,Lifecycle Smoke (Linux)' }}
136145
# Poll budget: ci-integration.yml chains Build -> Smoke ->
137146
# Integration (timeout 20m) -> Release Validation (timeout 20m).
138147
# Theoretical worst case ~50m; observed today ~5m end-to-end.

CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Changed
1111

12+
- Pull requests now run a ~65-70s hermetic `Lifecycle Smoke (Linux)` check,
13+
selected declaratively via a registered `lifecycle_smoke` pytest marker,
14+
that validates core install, lock-convergence, policy-enforcement, and
15+
prune-time hook-reconciliation contracts at PR time instead of only in
16+
the merge queue -- no network, credentials, or built binary required
17+
(#2247).
1218
- Object-form Git dependencies now reject unsupported keys instead of silently
1319
ignoring them; use `ref` rather than `version`, and omit inert `name` fields
1420
from `git: parent`. CI audit also rejects full-SHA pins whose

CONTRIBUTING.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,11 @@ What this means for contributors:
150150

151151
- You don't need to keep your branch up to date with `main` manually.
152152
- The fast unit + build checks (Tier 1) run on every push to your PR.
153+
- A hermetic ~65-70s Lifecycle Smoke check also runs on every push, validating
154+
core install/lock-convergence/policy-enforcement contracts via a declarative
155+
`lifecycle_smoke` pytest marker -- no network, credentials, or built binary
156+
required, so you get real regression feedback before the queue instead of
157+
only after.
153158
- The full integration suite (Tier 2) only runs once your PR is in the queue,
154159
not on every WIP push.
155160

docs/src/content/docs/contributing/integration-testing.md

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,20 @@ APM uses a tiered approach to integration testing:
3333
- **Duration**: ~10-15 minutes per platform (with 20-minute timeout)
3434
- **Trigger**: merge queue integration workflow, plus tag, schedule, and manual promotion runs
3535

36+
### 3. **Lifecycle Smoke** (PR-time required check)
37+
- **Location**: selected declaratively via the registered `lifecycle_smoke` pytest marker, applied to all of `tests/integration/test_install_content_hash_roundtrip.py`, `test_policy_pinned_constraint_e2e.py`, `test_virtual_claude_skill_lock_convergence.py`, `test_virtual_package_lifecycle_matrix.py` (module-level), to only `test_architecture_authorities.py::test_ado_lock_coordinates_have_single_owner` (function-level -- the file's other tests are not part of this family), and to only the `[claude]` parametrization of `test_prune_hook_reconciliation_e2e.py::test_prune_removes_merged_hook_entries_and_sidecar` (the sibling `[cursor]` parametrization is deliberately excluded)
38+
- **Purpose**: Promote the smallest stable, hermetic slice of the real Consume/Produce/Govern lifecycle contracts onto the PR-time critical path, so a regression in install/lock/audit convergence, ADO lock-coordinate handling, or prune-time hook reconciliation fails the PR instead of only surfacing once a change reaches the merge queue
39+
- **Scope**: install content-hash roundtrip, policy pinned-constraint enforcement, virtual-skill lock convergence, the virtual/manifestless lifecycle matrix, the ADO lock-coordinate ownership guard, and prune's merged-hook/sidecar reconciliation -- no built binary, no network, no credentials
40+
- **Duration**: ~65-70s job wall time (hard 3-minute timeout)
41+
- **Trigger**: every pull request and merge queue run (`ci.yml`'s `lifecycle-smoke` job, required via `merge-gate.yml`)
42+
- **Selection mechanism**: `pytest --strict-markers -m lifecycle_smoke tests/integration` -- declarative, not an explicit file/node-id list. If every `@pytest.mark.lifecycle_smoke` mark were ever removed, the marker family collects zero tests and pytest exits code 5 ("no tests collected"), failing the job loudly rather than silently shrinking. The marker is registered in `pyproject.toml`'s `[tool.pytest.ini_options]` markers list; `--strict-markers` rejects any typo'd or unregistered mark used as a decorator.
43+
- **Drift guard**: `tests/quality/test_ci_topology.py` pins the marker's registration, the command's use of `--strict-markers`/`-m lifecycle_smoke`/the bounded `tests/integration` root, hermeticity (no credentials at job- or step-level `env`, no credential expressions in `run:`/`with:`), required-check membership, and that the marker family collects a non-empty set of tests right now -- editing the job without updating that guard fails CI
44+
- **Run it locally** (the exact command CI runs):
45+
```bash
46+
uv run --extra dev pytest -p no:cacheprovider -q --strict-markers \
47+
-m lifecycle_smoke tests/integration
48+
```
49+
3650
## Running Tests Locally
3751

3852
Integration tests live under `tests/integration/` and run via `pytest`
@@ -73,9 +87,14 @@ Pytest markers compose across independent axes:
7387
| Behavioral | What boundary does the test cross? | `unit`, `component`, `e2e` |
7488
| Scheduling | When is the test selected? | `integration`, `slow`, `benchmark`, `live` |
7589
| Prerequisite | What environment must exist? | `requires_*` |
90+
| CI-selection | Is this test part of a named required CI gate? | `lifecycle_smoke` |
7691

7792
`live` is both an opt-in scheduling marker and an external-service
7893
prerequisite. Behavioral markers do not replace prerequisite markers.
94+
`lifecycle_smoke` is orthogonal to all three: it does not describe a
95+
test's boundary, scheduling, or precondition, only that
96+
`ci.yml`'s required `lifecycle-smoke` job selects it via
97+
`-m lifecycle_smoke` (see Tier 3 above for the full rationale).
7998

8099
The behavioral definitions are:
81100

@@ -248,7 +267,7 @@ environment end-to-end; for local iteration prefer the direct
248267
### GitHub Actions Workflow
249268

250269
**On PR and merge queue:**
251-
1. PR-time unit checks run first; merge queue adds Linux smoke, integration, and release-validation gates.
270+
1. PR-time unit checks and the hermetic Lifecycle Smoke gate run first; merge queue adds Linux smoke, integration, and release-validation gates.
252271

253272
**On version tag releases:**
254273
1. Unit tests + Smoke tests
@@ -325,6 +344,15 @@ Promotion integration tests run on:
325344
- ✅ Binary artifacts work across platforms
326345
- ✅ Release pipeline integrity (GitHub Release → PyPI)
327346

347+
### Lifecycle Smoke Verifies:
348+
- Install content-hash roundtrip (Consume contract)
349+
- Virtual-skill lock convergence (Produce contract, adjacent to the #2226 ADO lock-coordinate fix)
350+
- Policy pinned-constraint enforcement (Govern contract)
351+
- The virtual/manifestless lifecycle matrix: install, lock, frozen-install, update, and audit stay consistent (the direct #2240 regression)
352+
- The ADO lock-coordinate single-owner guard (the direct #2226 regression)
353+
- Prune's merged-hook and ownership-sidecar reconciliation for the `claude` target (the direct #2249 regression -- an orphaned package's merged hook entries and sidecar markers must be cleaned up, not left pointing at deleted scripts)
354+
- No network, no credentials, no built binary required for any of the above
355+
328356
## Benefits
329357

330358
### **Speed vs Confidence Balance**
@@ -361,6 +389,11 @@ Promotion integration tests run on:
361389
- Check GitHub Models API availability
362390
- Review actual vs expected output
363391
- Test locally with same environment
392+
393+
### Lifecycle Smoke Failures
394+
- These tests are hermetic -- no credentials, no built binary, no network (a real socket attempt raises `OSError`, it does not hang or retry). A failure is a genuine regression, not an environment issue.
395+
- Run the exact CI command from the "Run it locally" block under Tier 3 above to reproduce.
396+
- If the failure is about the CI job's shape (marker not registered, wrong `-m`/`--strict-markers` invocation, unbounded root, timeout, empty marker family, or required-check wiring) rather than test logic, check `tests/quality/test_ci_topology.py` -- that guard pins the job's contract and its own failure message will point at what drifted.
364397
- For hanging issues: Check command transformation in script runner (codex expects prompt content, not file paths)
365398

366399
## Adding New Tests

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -215,4 +215,5 @@ markers = [
215215
"requires_runtime_llm: requires llm runtime installed",
216216
"requires_inference: requires APM_RUN_INFERENCE_TESTS=1 + model API access",
217217
"req: binds a test to one or more OpenAPM req-XXX ids (spec-conformance suite)",
218+
"lifecycle_smoke: promoted to the required PR-time Lifecycle Smoke (Linux) job -- see ci.yml's lifecycle-smoke job and docs/src/content/docs/contributing/integration-testing.md",
218219
]

tests/integration/test_architecture_authorities.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ def test_object_git_dependency_fields_have_single_owner() -> None:
5252
assert "Object-form Git dependency fields must come from the product parser" in guard
5353

5454

55+
@pytest.mark.lifecycle_smoke
5556
def test_ado_lock_coordinates_have_single_owner() -> None:
5657
"""AC14 derives ADO coordinates without provider-specific lock fields."""
5758
import inspect

tests/integration/test_install_content_hash_roundtrip.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
from apm_cli.models.dependency.reference import DependencyReference
3030
from apm_cli.utils.content_hash import compute_package_hash
3131

32-
pytestmark = pytest.mark.component
32+
pytestmark = [pytest.mark.component, pytest.mark.lifecycle_smoke]
3333

3434
_PATCH_UPDATES = "apm_cli.commands._helpers.check_for_updates"
3535
_VIRTUAL_COMMIT = "a" * 40

tests/integration/test_policy_pinned_constraint_e2e.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
from apm_cli.policy.discovery import PolicyFetchResult
1818
from apm_cli.policy.schema import ApmPolicy, DependencyPolicy
1919

20-
pytestmark = pytest.mark.component
20+
pytestmark = [pytest.mark.component, pytest.mark.lifecycle_smoke]
2121

2222
_PATCH_DISCOVER_GATE = "apm_cli.policy.discovery.discover_policy_with_chain"
2323
_PATCH_DISCOVER_PREFLIGHT = "apm_cli.policy.install_preflight.discover_policy_with_chain"

tests/integration/test_prune_hook_reconciliation_e2e.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -220,7 +220,11 @@ def _pre_tool_use_commands(settings_path: Path) -> list[str]:
220220
return commands
221221

222222

223-
@pytest.mark.parametrize("target", ["claude", "cursor"], ids=["claude", "cursor"])
223+
@pytest.mark.parametrize(
224+
"target",
225+
[pytest.param("claude", marks=pytest.mark.lifecycle_smoke), "cursor"],
226+
ids=["claude", "cursor"],
227+
)
224228
def test_prune_removes_merged_hook_entries_and_sidecar(
225229
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, target: str
226230
) -> None:

0 commit comments

Comments
 (0)