Skip to content

fix(test): permit Stage 3 executor SYSTEM_ACCESSTOKEN env mapping#887

Merged
jamesadevine merged 1 commit into
mainfrom
copilot/fix-exec-context-trust-boundary-test
Jun 7, 2026
Merged

fix(test): permit Stage 3 executor SYSTEM_ACCESSTOKEN env mapping#887
jamesadevine merged 1 commit into
mainfrom
copilot/fix-exec-context-trust-boundary-test

Conversation

@jamesadevine

Copy link
Copy Markdown
Collaborator

Summary

CI on main is broken after the v0.32.0 release (run 27087796410) — the trust-boundary test added in #865 is failing because of a semantic conflict with #873 ("default executor to System.AccessToken"), which landed in between.

---- test_execution_context_pr_does_not_leak_system_accesstoken stdout ----
thread '...' panicked at tests/compiler_tests.rs:5335:22:
SYSTEM_ACCESSTOKEN was found in a step env block that is NOT the exec-context
PR prepare step. displayName = Some("Execute safe outputs (Stage 3)").
This indicates a credential leak into another step.

Root cause

The sibling test test_agent_job_steps_do_not_map_system_access_token already documents the right allow-list ("Only Setup-job filter-gate and Stage 3 executor are allowed to map it") and stays scoped to the Agent job. This PR brings the new test in line with that policy.

What this PR does

Replaces the single-displayName whitelist with an explicit ALLOWED_DISPLAY_NAMES list:

  1. Stage PR execution context (aw-context/pr/*) — owned by the exec-context extension.
  2. Execute safe outputs (Stage 3) — Stage 3 SafeOutputs executor (per feat(compile): default executor to System.AccessToken and add always-on Azure CLI #873).

Also keeps an explicit saw_exec_context_step assertion so the test cannot silently regress to passing when the PR contributor fails to activate at all.

The cross-stage trust boundary (no SYSTEM_ACCESSTOKEN inside the AWF-sandboxed Agent step) is unchanged — that's enforced by the agent-job-scoped sibling test.

Verification

$ cargo test --test compiler_tests -- test_execution_context_pr_ test_agent_job_steps_do_not_map_system_access_token
test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 119 filtered out

$ cargo test          # full suite
1765 + 127 + ... all green

$ cargo clippy --all-targets --all-features
Finished `dev` profile [unoptimized + debuginfo] target(s)   # no warnings

Follow-up: re-publishing v0.32.0 assets

The v0.32.0 release tag was created, but no asset binaries were uploaded because the Build matrix failed on this test. After this PR merges, the Release workflow can be re-triggered against the existing tag via workflow_dispatch:

gh workflow run Release --ref main -f tag_name=v0.32.0

The trust-boundary test test_execution_context_pr_does_not_leak_system_accesstoken
(added in #865) walks the compiled YAML and asserts that every step env: block
declaring SYSTEM_ACCESSTOKEN is the exec-context PR prepare step. That
was correct on #865's PR branch, but #873 ("default executor to
System.AccessToken") landed first and added a legitimate
SYSTEM_ACCESSTOKEN: $(System.AccessToken) mapping to the Stage 3
SafeOutputs executor step. The two compose into a semantic conflict on
main: each PR passes its own CI, but the combined tree fails.

Stage 3 runs in its own non-agent job and needs the token to apply safe
outputs (PRs, work items, etc.) — this is the same allow-list documented
by the sibling test test_agent_job_steps_do_not_map_system_access_token,
which scopes its check to the Agent job. The cross-stage trust boundary
(no SYSTEM_ACCESSTOKEN inside the AWF-sandboxed Agent step) is unaffected.

Replace the single-allowed-displayName whitelist with an explicit, audited
ALLOWED_DISPLAY_NAMES list covering:
  1. "Stage PR execution context (aw-context/pr/*)" — owned here.
  2. "Execute safe outputs (Stage 3)" — per #873.

Also keep an explicit assertion that the exec-context prepare step is
present so we don't silently regress to a state where the PR contributor
fails to activate but the test still passes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jamesadevine jamesadevine merged commit e3beb0f into main Jun 7, 2026
12 checks passed
@jamesadevine jamesadevine deleted the copilot/fix-exec-context-trust-boundary-test branch June 7, 2026 09:13
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

🔍 Rust PR Review

Summary: Looks good — correct fix for the CI break, logic is sound.

Findings

⚠️ Suggestions

  • tests/compiler_tests.rs:5263-5272 (docstring vs. allow-list mismatch): The doc comment lists three sanctioned locations, but ALLOWED_DISPLAY_NAMES only contains two entries. The third item — the filter-gate step — emits SYSTEM_ACCESSTOKEN with displayName of "Evaluate PR filters" or "Evaluate pipeline filters" (see filter_ir.rs:334-335). It's absent from the list because execution-context-agent.md doesn't have filters configured, so it never shows up in this fixture's compiled output.

    This is not a bug today, but if the fixture is ever updated to include filter config, the test will start failing even though the filter-gate is explicitly documented as sanctioned. Either:

    • Add both filter-gate display names to ALLOWED_DISPLAY_NAMES now (with a comment noting they appear only when filters are active), or
    • Add a comment on the const clarifying it is scoped to the current fixture's output (not the full system allow-list).

✅ What Looks Good

  • saw_exec_context_step sentinel is an excellent addition — it closes the silent-pass hole where the test could green-light a compiled output that never contained the exec-context step at all.
  • Panic message improvement: printing ALLOWED_DISPLAY_NAMES alongside the offending displayName makes future failures immediately actionable.
  • Trust boundary preserved: the agent-job-scoped sibling test (test_agent_job_steps_do_not_map_system_access_token) is untouched, so the Stage 1 / AWF sandbox guarantee remains independently enforced.
  • Narrow allow-list with audit comment: the // Keep this list narrow and audited note sets the right cultural expectation for anyone expanding the list later.

Generated by Rust PR Reviewer for issue #887 · sonnet46 896.7K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant