Skip to content

Commit ceb32cd

Browse files
jamesadevineCopilot
andcommitted
feat(compile): formalize trigger filter IR with compile-time validation
Introduces a typed intermediate representation (IR) for trigger filter expressions, replacing the manual bash string construction in pr_filters.rs. The IR separates data acquisition (Fact) from boolean predicates (Predicate), enabling: - Compile-time conflict detection (impossible/redundant filter combinations) - Dependency-ordered fact acquisition (pipeline vars → API → computed) - A single codegen pass from IR → bash gate step - Shared infrastructure for both PR and pipeline completion triggers Filter compilation now follows a three-pass architecture: 1. Lower: PrFilters/PipelineFilters → Vec<FilterCheck> 2. Validate: detect conflicts (min>max, include/exclude overlap, zero-width time windows, label set contradictions) 3. Codegen: GateContext + Vec<FilterCheck> → bash gate step Pipeline completion triggers now support runtime filters via gate steps (GateContext::PipelineCompletion), using the same IR and codegen as PR filters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 35f08e6 commit ceb32cd

7 files changed

Lines changed: 2033 additions & 601 deletions

File tree

docs/extending.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,3 +45,30 @@ pub trait CompilerExtension: Send {
4545
```
4646

4747
To add a new runtime or tool: (1) create a directory under `src/tools/` or `src/runtimes/`, (2) implement `CompilerExtension` in `extension.rs`, (3) add a variant to the `Extension` enum and a collection check in `collect_extensions()` in `src/compile/extensions/mod.rs`.
48+
49+
### Filter IR (`src/compile/filter_ir.rs`)
50+
51+
Trigger filter expressions (PR filters, pipeline filters) are compiled to bash
52+
gate steps via a three-pass IR pipeline:
53+
54+
1. **Lower**`PrFilters` / `PipelineFilters``Vec<FilterCheck>` (typed
55+
predicates over typed facts)
56+
2. **Validate** — detect conflicts at compile time (impossible combinations,
57+
redundant checks)
58+
3. **Codegen** — dependency-ordered fact acquisition + predicate evaluation →
59+
bash gate step
60+
61+
To add a new filter type:
62+
63+
1. **Add a `Fact` variant** (if the filter needs a new data source) — implement
64+
`dependencies()`, `shell_var()`, `acquisition_bash()`, and
65+
`failure_policy()` on the new variant
66+
2. **Add a `Predicate` variant** (if the filter needs a new test shape) —
67+
implement the codegen match arm in `emit_predicate_check()`
68+
3. **Extend lowering** — add the filter field to `PrFilters` or
69+
`PipelineFilters` in `types.rs`, then add the lowering logic in
70+
`lower_pr_filters()` or `lower_pipeline_filters()` in `filter_ir.rs`
71+
4. **Add validation rules** — check for conflicts with other filters in
72+
`validate_pr_filters()` or `validate_pipeline_filters()`
73+
5. **Write tests** — lowering test, validation test, and codegen test in
74+
`filter_ir.rs`

docs/front-matter.md

Lines changed: 55 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,13 +71,49 @@ safe-outputs: # optional per-tool configuration for safe output
7171
artifact-link: # optional: link work item to repository branch
7272
enabled: true
7373
branch: main
74-
triggers: # optional pipeline triggers
74+
on: # trigger configuration (unified under on: key)
75+
schedule: daily around 14:00 # fuzzy schedule - see docs/schedule-syntax.md
7576
pipeline:
7677
name: "Build Pipeline" # source pipeline name
7778
project: "OtherProject" # optional: project name if different
7879
branches: # optional: branches to trigger on
7980
- main
8081
- release/*
82+
filters: # optional runtime filters (compiled to gate step)
83+
source-pipeline:
84+
match: "Build.*"
85+
time-window:
86+
start: "09:00"
87+
end: "17:00"
88+
pr: # PR trigger
89+
branches:
90+
include: [main]
91+
paths:
92+
include: [src/*]
93+
filters: # runtime PR filters (compiled to gate step)
94+
title:
95+
match: "\\[review\\]"
96+
author:
97+
include: ["alice@corp.com"]
98+
draft: false
99+
labels:
100+
any-of: ["run-agent"]
101+
source-branch:
102+
match: "^feature/.*"
103+
target-branch:
104+
match: "^main$"
105+
commit-message:
106+
match: "^(?!.*\\[skip-agent\\])"
107+
changed-files:
108+
include: ["src/**/*.rs"]
109+
min-changes: 5
110+
max-changes: 100
111+
time-window:
112+
start: "09:00"
113+
end: "17:00"
114+
build-reason:
115+
include: [PullRequest]
116+
expression: "eq(variables['Custom.Flag'], 'true')" # raw ADO condition
81117
steps: # inline steps before agent runs (same job, generate context)
82118
- bash: echo "Preparing context for agent"
83119
displayName: "Prepare context"
@@ -127,3 +163,21 @@ list:
127163

128164
Set `workspace:` explicitly to `root`, `repo` (alias `self`), or a specific
129165
checked-out repository alias to override this behavior.
166+
167+
## Filter Validation
168+
169+
The compiler validates filter configurations at compile time and will emit
170+
errors for impossible or conflicting combinations:
171+
172+
| Condition | Severity | Message |
173+
|-----------|----------|---------|
174+
| `min-changes` > `max-changes` | Error | No PR can satisfy both constraints |
175+
| `time-window.start` = `time-window.end` | Error | Zero-width window never matches |
176+
| Same value in `author.include` and `author.exclude` | Error | Conflicting include/exclude |
177+
| Same value in `build-reason.include` and `build-reason.exclude` | Error | Conflicting include/exclude |
178+
| Label in both `labels.any-of` and `labels.none-of` | Error | Label both required and blocked |
179+
| Label in both `labels.all-of` and `labels.none-of` | Error | Label both required and blocked |
180+
| Empty `labels` filter (no any-of/all-of/none-of) | Warning | No label checks applied |
181+
182+
Errors cause compilation to fail. Fix the conflicting filter configuration
183+
before recompiling.

src/compile/common.rs

Lines changed: 81 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1182,28 +1182,45 @@ pub fn validate_resolve_pr_thread_statuses(front_matter: &FrontMatter) -> Result
11821182
/// Generate the setup job YAML.
11831183
///
11841184
/// When `pr_filters` is `Some`, injects a pre-activation gate step that evaluates
1185-
/// PR filters and self-cancels the build if they don't match. The Setup job is
1186-
/// created even if `setup_steps` is empty (solely for the gate).
1185+
/// PR filters and self-cancels the build if they don't match. When `pipeline_filters`
1186+
/// is `Some`, injects a similar gate step for pipeline completion triggers.
1187+
/// The Setup job is created even if `setup_steps` is empty (solely for the gate).
11871188
pub fn generate_setup_job(
11881189
setup_steps: &[serde_yaml::Value],
11891190
pool: &str,
11901191
pr_filters: Option<&super::types::PrFilters>,
1192+
pipeline_filters: Option<&super::types::PipelineFilters>,
11911193
) -> String {
1192-
if setup_steps.is_empty() && pr_filters.is_none() {
1194+
if setup_steps.is_empty() && pr_filters.is_none() && pipeline_filters.is_none() {
11931195
return String::new();
11941196
}
11951197

1198+
let has_gate = pr_filters.is_some() || pipeline_filters.is_some();
11961199
let mut steps_parts = Vec::new();
11971200

1198-
// Gate step (if PR filters are configured)
1201+
// PR gate step
11991202
if let Some(filters) = pr_filters {
12001203
steps_parts.push(super::pr_filters::generate_pr_gate_step(filters));
12011204
}
12021205

1203-
// User setup steps (conditioned on gate passing when PR filters are active)
1206+
// Pipeline gate step
1207+
if let Some(filters) = pipeline_filters {
1208+
steps_parts.push(generate_pipeline_gate_step(filters));
1209+
}
1210+
1211+
// User setup steps (conditioned on gate passing when filters are active)
12041212
if !setup_steps.is_empty() {
1205-
if pr_filters.is_some() {
1206-
let conditioned = super::pr_filters::add_condition_to_steps(setup_steps, "eq(variables['prGate.SHOULD_RUN'], 'true')");
1213+
if has_gate {
1214+
// Determine which gate step name to reference
1215+
let gate_var = if pr_filters.is_some() {
1216+
"prGate.SHOULD_RUN"
1217+
} else {
1218+
"pipelineGate.SHOULD_RUN"
1219+
};
1220+
let conditioned = super::pr_filters::add_condition_to_steps(
1221+
setup_steps,
1222+
&format!("eq(variables['{gate_var}'], 'true')"),
1223+
);
12071224
steps_parts.push(format_steps_yaml_indented(&conditioned, 4));
12081225
} else {
12091226
steps_parts.push(format_steps_yaml_indented(setup_steps, 4));
@@ -1225,6 +1242,34 @@ pub fn generate_setup_job(
12251242
)
12261243
}
12271244

1245+
/// Generate a pipeline gate step using the filter IR.
1246+
fn generate_pipeline_gate_step(filters: &super::types::PipelineFilters) -> String {
1247+
use super::filter_ir::{
1248+
compile_gate_step, lower_pipeline_filters, validate_pipeline_filters, GateContext,
1249+
Severity,
1250+
};
1251+
1252+
let diags = validate_pipeline_filters(filters);
1253+
for diag in &diags {
1254+
match diag.severity {
1255+
Severity::Error => eprintln!("error: {}", diag),
1256+
Severity::Warning => eprintln!("warning: {}", diag),
1257+
Severity::Info => eprintln!("info: {}", diag),
1258+
}
1259+
}
1260+
if diags.iter().any(|d| d.severity == Severity::Error) {
1261+
let errors: Vec<String> = diags
1262+
.iter()
1263+
.filter(|d| d.severity == Severity::Error)
1264+
.map(|d| format!("# FILTER ERROR: {}", d))
1265+
.collect();
1266+
return errors.join("\n");
1267+
}
1268+
1269+
let checks = lower_pipeline_filters(filters);
1270+
compile_gate_step(GateContext::PipelineCompletion, &checks)
1271+
}
1272+
12281273
/// Generate the teardown job YAML
12291274
pub fn generate_teardown_job(
12301275
teardown_steps: &[serde_yaml::Value],
@@ -1285,15 +1330,18 @@ pub fn generate_finalize_steps(finalize_steps: &[serde_yaml::Value]) -> String {
12851330

12861331
/// Generate dependsOn clause and condition for setup/gate dependencies.
12871332
///
1288-
/// When PR filters are active, adds a condition that allows non-PR builds to
1289-
/// proceed unconditionally, while PR builds require the gate to pass.
1333+
/// When PR or pipeline filters are active, adds a condition that allows
1334+
/// non-matching trigger types to proceed unconditionally, while matching
1335+
/// builds require the gate to pass.
12901336
/// When `expression` is provided, it's ANDed into the condition as an escape hatch.
12911337
pub fn generate_agentic_depends_on(
12921338
setup_steps: &[serde_yaml::Value],
12931339
has_pr_filters: bool,
1340+
has_pipeline_filters: bool,
12941341
expression: Option<&str>,
12951342
) -> String {
1296-
let has_setup = !setup_steps.is_empty() || has_pr_filters;
1343+
let has_gate = has_pr_filters || has_pipeline_filters;
1344+
let has_setup = !setup_steps.is_empty() || has_gate;
12971345

12981346
if !has_setup && expression.is_none() {
12991347
return String::new();
@@ -1305,7 +1353,7 @@ pub fn generate_agentic_depends_on(
13051353
""
13061354
};
13071355

1308-
if has_pr_filters || expression.is_some() {
1356+
if has_gate || expression.is_some() {
13091357
let mut parts = Vec::new();
13101358
parts.push("succeeded()".to_string());
13111359

@@ -1319,6 +1367,16 @@ pub fn generate_agentic_depends_on(
13191367
);
13201368
}
13211369

1370+
if has_pipeline_filters {
1371+
parts.push(
1372+
"or(\n\
1373+
\x20 ne(variables['Build.Reason'], 'ResourceTrigger'),\n\
1374+
\x20 eq(dependencies.Setup.outputs['pipelineGate.SHOULD_RUN'], 'true')\n\
1375+
\x20 )"
1376+
.to_string(),
1377+
);
1378+
}
1379+
13221380
if let Some(expr) = expression {
13231381
parts.push(expr.to_string());
13241382
}
@@ -1954,7 +2012,9 @@ pub async fn compile_shared(
19542012
// 8. Setup/teardown jobs, parameters, prepare/finalize steps
19552013
let pr_filters = front_matter.pr_filters();
19562014
let has_pr_filters = pr_filters.is_some();
1957-
let setup_job = generate_setup_job(&front_matter.setup, &pool, pr_filters);
2015+
let pipeline_filters = front_matter.pipeline_filters();
2016+
let has_pipeline_filters = pipeline_filters.is_some();
2017+
let setup_job = generate_setup_job(&front_matter.setup, &pool, pr_filters, pipeline_filters);
19582018
let teardown_job = generate_teardown_job(&front_matter.teardown, &pool);
19592019
let has_memory = front_matter
19602020
.tools
@@ -1965,8 +2025,15 @@ pub async fn compile_shared(
19652025
let parameters_yaml = generate_parameters(&parameters)?;
19662026
let prepare_steps = generate_prepare_steps(&front_matter.steps, extensions)?;
19672027
let finalize_steps = generate_finalize_steps(&front_matter.post_steps);
1968-
let expression = pr_filters.and_then(|f| f.expression.as_deref());
1969-
let agentic_depends_on = generate_agentic_depends_on(&front_matter.setup, has_pr_filters, expression);
2028+
let pr_expression = pr_filters.and_then(|f| f.expression.as_deref());
2029+
let pipeline_expression = pipeline_filters.and_then(|f| f.expression.as_deref());
2030+
let expression = pr_expression.or(pipeline_expression);
2031+
let agentic_depends_on = generate_agentic_depends_on(
2032+
&front_matter.setup,
2033+
has_pr_filters,
2034+
has_pipeline_filters,
2035+
expression,
2036+
);
19702037
let job_timeout = generate_job_timeout(front_matter);
19712038

19722039
// 9. Token acquisition and env vars

0 commit comments

Comments
 (0)