Skip to content

Commit 6ad1816

Browse files
feat(compile): runtime prompt loading via {{#runtime-import}} markers (#625)
* feat(compile): runtime prompt loading via {{#runtime-import}} markers Adopts gh-aw's {{#runtime-import path}} marker model and the inlined-imports front-matter toggle. Agent prompt bodies (and the Stage-2 threat-analysis prompt) are now loaded at pipeline runtime by default; body edits no longer require ado-aw compile. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(compile): introduce System phase so ado-script installs before user runtimes ADO's NodeTool@0 prepends to PATH, so when both AdoScriptExtension (gate+import bundle, requires Node 20.x) and NodeExtension (user-pinned version) emit NodeTool@0 into the same Agent job, the LAST install wins on PATH. Previously AdoScript ran in the Tool phase, AFTER Runtime — so its hardcoded 20.x silently overrode the user's pinned version. Introduces a new ExtensionPhase::System variant that sorts before Runtime. AdoScript moves to System: its NodeTool@0 install runs first, the resolver step uses it during a brief 20.x window, then the user's NodeExtension's NodeTool@0 runs and the user's version wins on PATH for everything after. Adds test_node_runtime_install_orders_after_ado_script_so_user_version_wins pinning this ordering. Updates extension-count tests for the new phase. Refreshes ExtensionPhase docs and the collect_extensions ordering policy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): guard runtime-import marker paths against whitespace and ##vso injection Addresses three findings from the 2026-05-19 Rust PR review on #625: 1. Reject whitespace in agent source paths at compile time when inlined-imports is false. The runtime resolver (scripts/ado-script/src/import/index.ts) matches marker bodies with [^\s}]+, so a space silently truncated the path and produced either a misleading runtime `file not found` or, for optional markers, an unexpanded marker visible to the LLM. Mirrors the existing resolve_imports_inline guard. 2. Aggregate all import errors instead of reporting only the first. Previously hadError ??= captured one failure and silently swallowed subsequent ones. Now every missing/unreadable required marker emits its own ##vso[task.logissue type=error] line before exit(1). 3. Sanitize rawPath in ##vso error output. Strips `]`, CR, and LF from path strings embedded in diagnostic lines so an unusual path can no longer break the ##vso command framing. Tests: * tests/compiler_tests.rs::test_runtime_imports_default_rejects_source_path_with_whitespace * scripts/ado-script/src/import/__tests__/error-reporting.test.ts (2 cases) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): block path-traversal in runtime-import resolver and guard fixture helper Two follow-up findings from the 2026-05-19 review on #625: 1. Reject `..` path components in both resolvers. `resolve_imports_inline` (compile-time, inlined-imports: true) and `import.js` (runtime, inlined-imports: false) both accepted `../`-style paths without restriction. A malicious markdown body on an untrusted PR branch could therefore embed host files (e.g. `{{#runtime-import ../../../../etc/passwd}}`) into the compiled YAML or, at runtime, into the agent prompt. The new guard rejects any path whose `/` or `\\`-split segments include `..`, regardless of whether the path is absolute or relative. Literal `..` characters inside a filename (e.g. `name..md`) are still allowed because they are not segments. 2. `compile_fixture_with_inlined_imports` now refuses fixtures that already declare `inlined-imports:`. The helper used to inject `inlined-imports: true` by raw string substitution before the closing `---`. If a future fixture hard-coded `inlined-imports: false`, the rewritten front matter would have two `inlined-imports:` keys; serde_yaml silently uses the last one so the test would still pass, but the duplicate-key fixture is confusing and the helper would silently flip the author's intent. The guard panics with an actionable message. Tests: * src/compile/extensions/ado_script.rs: 5 new unit tests covering relative/embedded/absolute/backslash `..` rejection and the literal `name..md` allow case. * scripts/ado-script/src/import/__tests__/path-traversal.test.ts: 4 new vitest cases. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(compile): drop unused ADO_AW_IMPORT_BASE env var `import.js` consults `ADO_AW_IMPORT_BASE` only when resolving a relative marker path (`isAbsolute(rawPath) ? rawPath : resolve(base, rawPath)`). In the pipeline the only marker `import.js` ever sees is the compiler-generated top-level body marker, which embeds an absolute `$(Build.SourcesDirectory)/...` path. The resolver is also single-pass by design, so author-written nested relative markers inside the inlined body are never re-expanded at runtime. The env var was therefore dead code at runtime. Changes: - Remove the `env:` block from `resolver_step()` in `src/compile/extensions/ado_script.rs` (and update its unit test to assert the variable is absent). - Drop the `process.env.ADO_AW_IMPORT_BASE ??` fallback in `scripts/ado-script/src/import/index.ts`; `import.js` now always uses `dirname(argv[2])` for relative-path resolution (irrelevant in pipeline use, useful for local invocations). - Replace the `ADO_AW_IMPORT_BASE`-override vitest with a `dirname(target)` default-base test that pins the fallback for standalone callers. - Update `docs/ado-script.md` and `docs/runtime-imports.md` to drop the env-var contract and explain why the runtime never sees a relative marker. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): correct threat-prompt doc claim and gate required_hosts on actual download Two findings from the latest PR review: 1. docs/template-markers.md falsely claimed that the `{{ threat_analysis_prompt }}` marker is emitted as a `{{#runtime-import ...}}` when `inlined-imports: false`. The threat-analysis prompt is tooling-shipped (compiled into the `ado-aw` binary via `include_str!`) and unconditionally inlined at step 11 of `compile_shared`. The marker is for the agent body, not the threat prompt. Rewrote the paragraph to reflect this and to cross-reference the rationale in `src/compile/common.rs`. 2. `AdoScriptExtension::required_hosts()` always requested `github.com`, even when `inlined-imports: true` AND no filters were configured (so neither `setup_steps()` nor `prepare_steps()` emits the NodeTool@0 + curl pair, and github.com is therefore unreachable from the pipeline). For a security-sensitive project, the allowlist should match the actual network reach of the compiled pipeline. Now returns `vec![]` unless `has_gate()` or `runtime_imports_active()`. Added three unit tests covering all three branches (no-consumer, gate-active, imports-active). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): reject absolute paths in compile-time runtime-import resolver Three findings from the latest PR review: 1. **Security**: `resolve_imports_inline` accepted absolute paths without restriction. When `inlined-imports: true`, an untrusted PR branch could embed arbitrary host files into the compiled YAML via `{{#runtime-import /home/runner/.ssh/id_rsa}}`, `{{#runtime-import C:\Users\runner\secret.txt}}`, or `{{#runtime-import \\server\share\file}}`. The PR description already called out the `..`-traversal threat for "untrusted PR branches" — the same threat applies here. Compile-time resolution now requires a **relative** path rooted under `base_dir` (the source `.md` file's parent directory, which is inside the repo and therefore part of the same trust domain). `std::path::Path::is_absolute` is platform-dependent, so the guard also explicitly checks `/`-prefixed and `\\`-prefixed shapes. `import.js` (runtime resolver) keeps its absolute-path support unchanged — the agent VM is a lower-risk environment and the pipeline-generated body marker is always absolute. 2. **Cleanup**: `AdoScriptExtension::has_gate()` and `setup_steps()` both called `lower_pr_filters()` / `lower_pipeline_filters()`, doing the same lowering twice. Factored into a single `lowered_checks()` helper that both call sites reuse. 3. **Comment**: `sanitizeForVsoMessage` in `import.js` now explains why `[` is intentionally NOT stripped (without a closing `]` and a fresh `##vso` prefix it cannot open a new logging command). Tests: * `rejects_absolute_posix_path_at_compile_time`, `rejects_absolute_windows_drive_path_at_compile_time`, `rejects_unc_path_at_compile_time` in `src/compile/extensions/ado_script.rs`. * `supports_relative_and_absolute_paths` test removed (no longer reachable behaviour); replaced with `supports_relative_path_resolution`. * `docs/runtime-imports.md` updated to document the new restriction and call out the security rationale. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): reject `}` in runtime-import paths to match the runtime regex `import.js`'s marker regex `[^\s}]+` excludes `}` from the path capture so the regex terminates cleanly at the closing `}}`. The compile-time resolver (`resolve_imports_inline`) terminated only at `}}` and would therefore silently accept a path like `foo}bar.md` that the runtime resolver would either truncate or leave unexpanded — a real compile-vs-runtime divergence. Reject `}` in paths up front at compile time so the failure mode is one clear compile error rather than two different runtime behaviours depending on `inlined-imports`. Added a comment to `import.js`'s regex documenting that the compile-time side enforces the same restriction. Test: `rejects_path_containing_closing_brace` in `src/compile/extensions/ado_script.rs`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): make absolute-path check platform-independent for runtime-import `Path::is_absolute` is platform-dependent — on Linux it doesn't recognize Windows drive-letter paths like `C:\Users\...` as absolute, so `rejects_absolute_windows_drive_path_at_compile_time` failed on Linux CI with `file not found` instead of the expected absolute-path rejection. Added an explicit drive-letter detector that matches `[A-Za-z]:[/\\]` via pure string inspection, so the guard fires on every host where `ado-aw compile` may run regardless of the local platform's path semantics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): reject absolute paths in runtime resolver and pass --base from pipeline The compile-time `resolve_imports_inline` already rejected absolute paths, but `import.js` allowed them at runtime with a misleading "Mirrors `resolve_imports_inline`" comment. Although the documented single-pass design currently prevents author-written nested markers from reaching `import.js`, the asymmetry was a defence-in-depth gap: - the resolver explicitly says it handles "arbitrary author-written markers in the agent body"; - if the design ever shifts to multi-pass, an author marker like `{{#runtime-import /tmp/awf-tools/staging/mcpg-config.json}}` could silently embed credentials into the agent's system prompt; - the runtime and compile-time paths should enforce the same policy so reviewers don't have to mentally track which guards apply where. Changes: * `scripts/ado-script/src/import/index.ts` - Add `--base <path>` CLI arg (replaces the implicit `dirname(target)` fallback for pipeline use; the fallback remains for standalone invocation). - Reject absolute paths the same way Rust does: POSIX `/foo`, Windows drive-letter `C:\foo`/`C:/foo` (detected via a platform-independent character scan because `path.isAbsolute` is OS-dependent on this), and UNC `\\server\share`. * `src/compile/common.rs` - Emit a trigger-repo-relative marker (stripping `$(Build.SourcesDirectory)/` from the resolved source path) so the new absolute-path reject in `import.js` doesn't fire on the compiler-generated body marker. The relative-form marker is the only form `import.js` ever needs to resolve at runtime. * `src/compile/extensions/ado_script.rs` - Resolver step passes `--base "$(Build.SourcesDirectory)"` so the relative marker resolves against the trigger-repo checkout root. Tests: * New vitest cases for the three absolute-path shapes (POSIX, drive-letter, UNC) and for `--base` resolution. * Replaced the "uses absolute snippet paths as-is" test with a rejection assertion. * Unit test updated to assert the resolver step now includes `--base "$(Build.SourcesDirectory)"`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compile): reject `}` in runtime-import source path and fix stale extension refs Two findings from the latest review: 1. **Bug**: `agent_marker_path` in `compile_shared` was only guarded against whitespace, not `}`. An agent file at `agents/fo}o.md` (valid on Linux/macOS/NTFS) would emit a malformed marker that the runtime regex `[^\s}]+` truncates at the `}` and then fails to match because the next two chars aren't `}}`. The marker would survive as literal text in the agent's prompt with no error surfaced. Added an explicit `}` check that mirrors the same guard in `resolve_imports_inline`, with a clear error suggesting either renaming the path or setting `inlined-imports: true`. 2. **Stale refs**: `TriggerFiltersExtension` and `src/compile/extensions/trigger_filters.rs` were referenced in: - `src/compile/pr_filters.rs` (3 comments) - `.github/workflows/ado-script.yml` (path triggers) - `tests/bash_lint_tests.rs` (annotation comment) - `scripts/ado-script/README.md` (Bundles + Layout + See also) - `site/src/content/docs/reference/filter-ir.mdx` (Integration Points section + Gate Step Injection) All updated to point at `AdoScriptExtension` / `src/compile/extensions/ado_script.rs` (the consolidated extension introduced earlier in this PR). The site doc was also extended to note the additional `prepare_steps()` hook the extension owns for the runtime-import resolver. Tests: * `test_runtime_imports_default_rejects_source_path_with_closing_brace` in `tests/compiler_tests.rs` — exercises a source path with `}` and asserts the compiler errors with the new guard. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 1130ba4 commit 6ad1816

54 files changed

Lines changed: 3834 additions & 969 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ado-script.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ on:
55
paths:
66
- "scripts/ado-script/**"
77
- "src/compile/filter_ir.rs"
8-
- "src/compile/extensions/trigger_filters.rs"
8+
- "src/compile/extensions/ado_script.rs"
99
- "Cargo.toml"
1010
- "Cargo.lock"
1111
- ".github/workflows/ado-script.yml"
@@ -19,7 +19,7 @@ on:
1919
paths:
2020
- "scripts/ado-script/**"
2121
- "src/compile/filter_ir.rs"
22-
- "src/compile/extensions/trigger_filters.rs"
22+
- "src/compile/extensions/ado_script.rs"
2323
- "Cargo.toml"
2424
- "Cargo.lock"
2525
- ".github/workflows/ado-script.yml"
@@ -69,13 +69,13 @@ jobs:
6969
working-directory: scripts/ado-script
7070
run: npm run typecheck
7171

72-
- name: Build bundle (gate.js)
72+
- name: Build bundles
7373
working-directory: scripts/ado-script
7474
run: npm run build
7575

76-
- name: Smoke-test bundle
76+
- name: Smoke-test bundles
7777
working-directory: scripts/ado-script
78-
run: npx vitest run -c vitest.config.smoke.ts
78+
run: npm run test:smoke
7979

8080
- name: E2E gate test
8181
run: cargo test --test gate_e2e -- --ignored --nocapture

AGENTS.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ Every compiled pipeline runs as three sequential jobs:
6262
│ │ │ ├── ado_aw_marker.rs # Always-on metadata marker extension (emits # ado-aw-metadata JSON)
6363
│ │ │ ├── github.rs # Always-on GitHub MCP extension
6464
│ │ │ ├── safe_outputs.rs # Always-on SafeOutputs MCP extension
65-
│ │ │ ├── trigger_filters.rs # Trigger filter extension (gate evaluator delivery)
65+
│ │ │ ├── ado_script.rs # Always-on ado-script extension (gate evaluator + runtime-import resolver, per-job downloads)
6666
│ │ │ └── tests.rs # Extension integration tests
6767
│ │ ├── codemods/ # Front-matter codemods (one file per transformation)
6868
│ │ │ ├── mod.rs # Codemod struct, CODEMODS registry, runner
@@ -159,7 +159,9 @@ Every compiled pipeline runs as three sequential jobs:
159159
│ ├── update-ado-agentic-workflow.md # Guide for modifying an existing agentic pipeline
160160
│ └── debug-ado-agentic-workflow.md # Guide for troubleshooting a failing agentic pipeline
161161
├── scripts/ # Supporting scripts shipped as release artifacts
162-
│ ├── ado-script/ # TypeScript workspace for bundled gate.js (and future bundles)
162+
│ ├── ado-script/ # TypeScript workspace for bundled gate.js, import.js, and future bundles
163+
│ │ └── src/
164+
│ │ └── import/ # Runtime prompt resolver bundle
163165
│ └── gate.js # Bundled gate evaluator (built from scripts/ado-script/, see docs/ado-script.md)
164166
├── tests/ # Integration tests and fixtures
165167
├── docs/ # Per-concept reference documentation (see index below)
@@ -172,7 +174,7 @@ Every compiled pipeline runs as three sequential jobs:
172174
- **Language**: Rust (2024 edition) - Note: Rust 2024 edition exists and is the edition used by this project
173175
- **CLI Framework**: clap v4 with derive macros
174176
- **Error Handling**: anyhow for ergonomic error propagation
175-
- **Bundled scripts**: TypeScript + ncc (`scripts/ado-script/`) — compiled gate evaluator and future internal helpers; see [`docs/ado-script.md`](docs/ado-script.md).
177+
- **Bundled scripts**: TypeScript + ncc (`scripts/ado-script/`) — compiled gate evaluator, runtime import resolver, and future internal helpers; see [`docs/ado-script.md`](docs/ado-script.md).
176178
- **Async Runtime**: tokio with full features
177179
- **YAML Parsing**: serde_yaml
178180
- **MCP Server**: rmcp with server and transport-io features
@@ -196,6 +198,8 @@ index to jump to the right page.
196198

197199
- [`docs/front-matter.md`](docs/front-matter.md) — full agent file format
198200
(markdown body + YAML front matter grammar) with every supported field.
201+
- [`docs/runtime-imports.md`](docs/runtime-imports.md) — runtime prompt import
202+
markers, path resolution, and `inlined-imports:` behavior.
199203
- [`docs/schedule-syntax.md`](docs/schedule-syntax.md) — fuzzy schedule time
200204
syntax (`daily around 14:00`, `weekly on monday`, timezones, scattering).
201205
- [`docs/engine.md`](docs/engine.md)`engine:` configuration (model,
@@ -242,7 +246,7 @@ index to jump to the right page.
242246
adding codemods.
243247
- [`docs/ado-script.md`](docs/ado-script.md)`ado-script` workspace
244248
(`scripts/ado-script/`): the bundled TypeScript runtime helpers (today:
245-
`gate.js`), schemars-driven type codegen, and the A2 design decision.
249+
`gate.js` and `import.js`), schemars-driven type codegen, and the A2 design decision.
246250
- [`docs/local-development.md`](docs/local-development.md) — local development
247251
setup notes.
248252

docs/ado-script.md

Lines changed: 91 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,9 @@
33
`ado-script` is the umbrella name for the TypeScript workspace at
44
[`scripts/ado-script/`](../scripts/ado-script/). It produces small,
55
ncc-bundled Node programs that the **compiler injects into every emitted
6-
pipeline** as runtime helpers. The first (and currently only) bundle is
7-
`gate.js`, the trigger-filter gate evaluator.
6+
pipeline** as runtime helpers. Today it produces `gate.js`, the
7+
trigger-filter gate evaluator, and `import.js`, the runtime prompt
8+
resolver described in [`runtime-imports.md`](runtime-imports.md).
89

910
> **Internal-only.** `ado-script` is not a user-facing front-matter
1011
> feature. Authors never write an `ado-script:` block in their agent
@@ -33,6 +34,35 @@ discriminated union. There is no `eval`, no `Function`, no `vm` — a
3334
compromised compiler cannot use the spec to run arbitrary code on the
3435
pipeline runner.
3536

37+
## What `import.js` does
38+
39+
`import.js` is a single-shot Node program. It reads the prompt file path
40+
from `argv[2]` and resolves `{{#runtime-import path}}` markers in place.
41+
The compiler runs it as a post-prepare-prompt step when
42+
[`inlined-imports: false`](front-matter.md#inlined-imports). See
43+
[`runtime-imports.md`](runtime-imports.md) for the author-facing marker
44+
syntax.
45+
46+
### Env-var contract
47+
48+
`import.js` takes no environment variables. Relative-path markers
49+
resolve against `dirname(argv[2])`; in pipeline use this is irrelevant
50+
because the compiler always embeds an absolute marker path and
51+
`import.js` is single-pass (nested markers inside the inlined body are
52+
not re-expanded).
53+
54+
The bundle lives at `dist/import/index.js` and ships in the same
55+
`ado-script.zip` release asset as `gate.js`, so pipelines download it
56+
through the same Setup-job asset flow. `import.js` uses only the Node
57+
standard library, so the ncc bundle is small (~1.5 KB) and carries no
58+
SDK dependency.
59+
60+
The Stage-2 threat-analysis prompt is **not** runtime-imported.
61+
`src/data/threat-analysis.md` is `include_str!`'d into the `ado-aw`
62+
binary and inlined into the emitted YAML at compile time, matching
63+
gh-aw's pattern (their `threat_detection.md` ships with the setup
64+
action and is read directly from disk — no marker, no resolver).
65+
3666
## End-to-end data flow
3767

3868
```
@@ -147,14 +177,19 @@ scripts/ado-script/
147177
│ │ ├── env-facts.ts # Pipeline-variable readers + ENV_BY_FACT + BRANCH_FACTS + ref-prefix stripping
148178
│ │ ├── policy.ts # PolicyTracker state machine
149179
│ │ └── vso-logger.ts # ##vso[…] emitters with property/message escaping; complete() is idempotent
150-
│ └── gate/ # gate.js entry point + per-concern modules
151-
│ ├── index.ts # main(): decode → preflight → bypass → facts → eval → emit
152-
│ ├── bypass.ts # build-reason auto-pass
153-
│ ├── facts.ts # fact acquisition (env + REST)
154-
│ ├── predicates.ts # 11 predicate evaluators + validatePredicateTree + glob ReDoS hardening
155-
│ └── selfcancel.ts # best-effort build cancellation
180+
│ ├── gate/ # gate.js entry point + per-concern modules
181+
│ │ ├── index.ts # main(): decode → preflight → bypass → facts → eval → emit
182+
│ │ ├── bypass.ts # build-reason auto-pass
183+
│ │ ├── facts.ts # fact acquisition (env + REST)
184+
│ │ ├── predicates.ts # 11 predicate evaluators + validatePredicateTree + glob ReDoS hardening
185+
│ │ └── selfcancel.ts # best-effort build cancellation
186+
│ └── import/ # import.js entry point + runtime prompt resolver
187+
│ ├── index.ts # main(): expand runtime-import markers in place
188+
│ └── __tests__/ # marker, path-resolution, and single-pass coverage
156189
├── test/ # End-to-end smoke tests
157-
└── dist/gate/index.js # ncc bundle output (gitignored)
190+
└── dist/ # ncc bundle output (gitignored)
191+
├── gate/index.js
192+
└── import/index.js
158193
```
159194

160195
The release workflow (`.github/workflows/release.yml`) runs
@@ -195,11 +230,18 @@ The Rust subcommand that emits the schema is intentionally hidden:
195230
cargo run -- export-gate-schema --output schema/gate-spec.schema.json
196231
```
197232

198-
## How the gate bundle is wired into emitted pipelines
233+
## How the bundles are wired into emitted pipelines
234+
235+
`AdoScriptExtension`
236+
(`src/compile/extensions/ado_script.rs`) is the always-on single
237+
extension that owns all `ado-script` wiring. It has two independent
238+
features, each emitted **into the job that actually consumes the
239+
bundle**:
199240

200-
`TriggerFiltersExtension`
201-
(`src/compile/extensions/trigger_filters.rs`) injects three Setup-job
202-
steps when any `filters:` block is active:
241+
### Setup job (gate evaluator)
242+
243+
When `filters:` lowers to non-empty checks, `setup_steps()` returns
244+
three step strings into the Setup job:
203245

204246
1. **`NodeTool@0`** — installs Node 20.x LTS, capped at
205247
`timeoutInMinutes: 5`.
@@ -209,9 +251,42 @@ steps when any `filters:` block is active:
209251
`unzip -o /tmp/ado-aw-scripts/ado-script.zip -d /tmp/ado-aw-scripts/`.
210252
Also capped at `timeoutInMinutes: 5`.
211253
3. **`bash: node '/tmp/ado-aw-scripts/ado-script/dist/gate/index.js'`**
212-
runs the gate with `GATE_SPEC` and the env-var contract above.
213-
214-
The IR-to-bash codegen that produces these steps is
254+
runs the gate with `GATE_SPEC` and the env-var contract documented
255+
above.
256+
257+
### Agent job (runtime-import resolver)
258+
259+
When `inlined-imports: false` (the default), `prepare_steps()` returns
260+
the same install + download pair plus the resolver invocation, into
261+
the Agent job's existing `{{ prepare_steps }}` block:
262+
263+
1. **`NodeTool@0`** — same shape as above.
264+
2. **`curl` download + verify + extract** — same artefact, same
265+
verification.
266+
3. **`bash: node '/tmp/ado-aw-scripts/ado-script/dist/import/index.js'`**
267+
expands `{{#runtime-import …}}` markers in
268+
`/tmp/awf-tools/agent-prompt.md` in place. See
269+
[`runtime-imports.md`](runtime-imports.md) for marker syntax.
270+
271+
### Per-job download (NOT a duplication bug)
272+
273+
ADO jobs use **isolated VMs**`/tmp` is not shared between jobs.
274+
The `ado-script.zip` bundle therefore has to be downloaded once per
275+
job that consumes it. When both features are active (a pipeline with
276+
both `filters:` and `inlined-imports: false`), install + download
277+
steps appear in **both** Setup and Agent. That's correct architecture
278+
given ADO's topology, not waste.
279+
280+
### What gets emitted, by case
281+
282+
| `filters:` | `inlined-imports` | Setup-job steps | Agent-job extra steps |
283+
|---|---|---|---|
284+
| inactive | `true` | (none) | (none) |
285+
| inactive | `false` | (no Setup job) | install + download + resolver |
286+
| active | `true` | install + download + gate | (none) |
287+
| active | `false` | install + download + gate | install + download + resolver |
288+
289+
The IR-to-bash codegen that produces the gate step is
215290
`compile_gate_step_external` in `src/compile/filter_ir.rs`.
216291

217292
## Modifying `ado-script`

docs/filter-ir.md

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -350,24 +350,30 @@ The bash shim exports only the ADO macros needed by the spec's facts:
350350

351351
## Integration Points
352352

353-
### TriggerFiltersExtension
353+
### AdoScriptExtension
354354

355-
When Tier 2/3 filters are configured, the `TriggerFiltersExtension`
356-
(`src/compile/extensions/trigger_filters.rs`) activates via
357-
`collect_extensions()`. It implements `CompilerExtension` and controls:
355+
When `filters:` is configured (and lowers to non-empty checks), the
356+
always-on `AdoScriptExtension`
357+
(`src/compile/extensions/ado_script.rs`) emits the gate-side steps via
358+
the `setup_steps()` trait hook. The extension also owns the unrelated
359+
runtime-import resolver — see [`runtime-imports.md`](runtime-imports.md).
360+
361+
For the gate path it controls:
358362

359363
1. **Node install step** — emits a `NodeTool@0` step pinned to Node 20.x
360-
LTS so `gate.js` has a runtime
364+
LTS so `gate.js` has a runtime.
361365
2. **Download step** — fetches `ado-script.zip` from the ado-aw release
362366
artifacts, verifies its SHA256 checksum via `checksums.txt`, then
363-
extracts `gate.js` to `/tmp/ado-aw-scripts/ado-script/dist/gate/index.js`
367+
extracts `gate.js` to `/tmp/ado-aw-scripts/ado-script/dist/gate/index.js`.
364368
3. **Gate step** — calls `compile_gate_step_external()` to generate a step
365-
that runs `node /tmp/ado-aw-scripts/ado-script/dist/gate/index.js` (no inline heredoc)
369+
that runs `node /tmp/ado-aw-scripts/ado-script/dist/gate/index.js` (no inline heredoc).
366370
4. **Validation** — runs `validate_pr_filters()` / `validate_pipeline_filters()`
367-
during compilation via the `validate()` trait method
371+
during compilation via the `validate()` trait method.
368372

369-
The extension uses the `setup_steps()` trait method (not `prepare_steps()`)
370-
because the gate must run in the **Setup job** (before the Agent job).
373+
The gate-side steps use `setup_steps()` (not `prepare_steps()`)
374+
because the gate must run in the **Setup job**, before the Agent job.
375+
Runtime-import resolver steps for the agent body use `prepare_steps()` and
376+
land in the Agent job — see [`runtime-imports.md`](runtime-imports.md).
371377

372378
### Tier 1 Inline Path
373379

@@ -379,7 +385,7 @@ no Node evaluator and no download step.
379385
### Gate Step Injection
380386

381387
Gate steps are injected into the Setup job by `generate_setup_job()` in
382-
`common.rs`. When the `TriggerFiltersExtension` is active, its
388+
`common.rs`. When the `AdoScriptExtension` is active, its
383389
`setup_steps()` are collected and injected first (download + gate). When
384390
only Tier 1 filters are present, the inline gate step is injected directly.
385391

docs/front-matter.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,27 @@ becomes a `repos:` entry, with `checkout: false` added for entries
234234
that weren't listed under `checkout:`. Mixing the legacy fields with
235235
an existing `repos:` block is rejected; pick one shape.
236236

237+
## Inlined Imports
238+
239+
The `inlined-imports:` field controls when `{{#runtime-import ...}}`
240+
markers in the markdown body are resolved. It defaults to `false`.
241+
See [`runtime-imports.md`](runtime-imports.md) for the full marker
242+
syntax, path resolution rules, and runtime behavior.
243+
244+
When `inlined-imports: false`, the compiler leaves runtime-import
245+
markers to be resolved on the pipeline runner. This is the default
246+
behavior, and it means prompt-body edits do not require recompiling the
247+
generated YAML.
248+
249+
When `inlined-imports: true`, the compiler resolves all runtime-import
250+
markers at compile time, including the implicit top-level marker that
251+
normally reloads the body itself. The emitted YAML contains the fully
252+
expanded prompt body, so the pipeline file is self-contained.
253+
254+
The trade-off is that the generated YAML is larger, and prompt-body
255+
edits require `ado-aw compile` plus committing the updated pipeline
256+
file.
257+
237258
## Filter Validation
238259

239260
The compiler validates filter configurations at compile time and will emit

docs/runtime-imports.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Runtime Imports
2+
3+
_Part of the [ado-aw documentation](../AGENTS.md)._
4+
5+
Runtime imports let agent prompts pull in snippet files with gh-aw-compatible
6+
`{{#runtime-import ...}}` markers. They are available in the markdown body and
7+
are controlled by the [`inlined-imports:` field](front-matter.md#inlined-imports).
8+
The runtime bundle that expands them is documented in [`ado-script.md`](ado-script.md).
9+
10+
## Marker syntax
11+
12+
Use `{{#runtime-import path}}` for a required import. If the target file is
13+
missing, resolution fails.
14+
15+
```markdown
16+
## Repository policy
17+
{{#runtime-import docs/policy.md}}
18+
```
19+
20+
Use `{{#runtime-import? path}}` for an optional import. If the target file is
21+
missing, the marker is replaced with an empty string.
22+
23+
```markdown
24+
## Local notes
25+
{{#runtime-import? docs/local-notes.md}}
26+
```
27+
28+
## Where markers can appear
29+
30+
Authors can place runtime-import markers anywhere in the agent markdown body.
31+
When `inlined-imports: false` (the default), the compiler also injects an
32+
implicit top-level runtime-import marker that reloads the body itself at
33+
pipeline runtime instead of embedding it into the generated YAML. When
34+
`inlined-imports: true`, that implicit body marker is resolved at compile time
35+
along with any author-written markers.
36+
37+
## Path resolution
38+
39+
- **Author-written markers** must use **relative paths** rooted at the agent
40+
`.md` file's directory. Absolute paths and `..` segments are rejected. This
41+
protects the compile host (`ado-aw compile`, which may run on a CI agent
42+
carrying privileged material like SSH keys and service-connection tokens)
43+
from untrusted PR branches embedding host files into the compiled YAML —
44+
e.g. `{{#runtime-import /home/runner/.ssh/id_rsa}}` or
45+
`{{#runtime-import ../../../../etc/passwd}}` are both compile-time errors.
46+
- **Compiler-generated marker for the agent body** uses an absolute path
47+
(`$(Build.SourcesDirectory)/…`) built from the trigger-repo checkout root,
48+
so the runtime resolver never has to resolve a relative path. The
49+
compile-time restriction does not apply here because the path is
50+
tooling-generated, not author-supplied.
51+
52+
## Single-pass behavior
53+
54+
Runtime imports are expanded in a single pass. Imported snippets are inserted
55+
verbatim, and any nested `{{#runtime-import ...}}` or
56+
`{{#runtime-import? ...}}` markers inside those snippets are **not** expanded.
57+
This matches gh-aw's runtime-import behavior.
58+
59+
## Resolver ordering
60+
61+
The runtime-import resolver runs first. Any extension supplements that are
62+
appended later with `cat >>` — including SafeOutputs guidance, GitHub MCP
63+
guidance, runtime guidance, and cache-memory guidance — are added after import
64+
resolution and are left untouched.
65+
66+
## Failure modes
67+
68+
| Marker kind | Missing file behavior |
69+
|---|---|
70+
| `{{#runtime-import path}}` | Resolver exits with status 1 and the pipeline fails. |
71+
| `{{#runtime-import? path}}` | Marker is silently replaced with an empty string. |
72+
73+
When `inlined-imports: true`, the same required/optional rules are applied at
74+
compile time instead of on the pipeline runner.
75+
76+
## Implementation notes
77+
78+
- **Runtime**: `dist/import/index.js` is ncc-bundled into `ado-script.zip`.
79+
The always-on `AdoScriptExtension`'s `prepare_steps()` injects three
80+
steps into the Agent job's existing `{{ prepare_steps }}` block:
81+
`NodeTool@0` install, the `ado-script.zip` download/verify/extract,
82+
and the `node import.js` resolver invocation. All three run on the
83+
same VM as the agent — ADO jobs are VM-isolated, so the bundle must
84+
be downloaded inside whichever job consumes it.
85+
- **Compile time**: `resolve_imports_inline()` in
86+
`src/compile/extensions/ado_script.rs` performs the inline expansion
87+
when `inlined-imports: true`.
88+

0 commit comments

Comments
 (0)