Skip to content

Commit 5434f37

Browse files
jamesadevineCopilot
andcommitted
fix(compile): quote pipeline name in generated YAML to handle colons
Front-matter agent names like 'Daily safe-output smoke: noop' produced invalid YAML when substituted bare into the top-level 'name:' line: ADO YAML parsers (and yaml.safe_load) saw the second colon as a mapping indicator and rejected the file with 'Mapping values are not allowed in this context.' Fix by introducing a new {{ pipeline_display_name }} marker that emits the full '<name>-$(BuildID)' value as a YAML double-quoted scalar with proper escaping for backslashes, embedded double quotes, and ASCII control characters. The marker replaces {{ agent_name }}-$(BuildID) in src/data/base.yml and src/data/1es-base.yml. Other usages of {{ agent_name }} (inside already-quoted displayName fields and markdown bodies) are unchanged. Includes 8 new unit tests for yaml_double_quoted covering plain strings, the original colon bug, backslash/quote/control-char escaping, ADO macro passthrough, and unicode. Test_compiled_yaml_structure updated to require the new marker. docs/template-markers.md documents the new marker and warns against using {{ agent_name }} in unquoted YAML positions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 29b221e commit 5434f37

31 files changed

Lines changed: 343 additions & 211 deletions

docs/template-markers.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,34 @@ This distinction allows resources (like templates) to be available as pipeline r
9999

100100
Should be replaced with the human-readable name from the front matter (e.g., "Daily Code Review"). This is used for display purposes like stage names.
101101

102+
> ⚠️ This marker is only safe inside an **already-quoted YAML scalar**
103+
> (e.g. `displayName: "{{ agent_name }}"`) or a markdown body. For the
104+
> top-level pipeline `name:` line, use
105+
> [`{{ pipeline_display_name }}`](#-pipeline_display_name-) instead — it
106+
> emits the value as a quoted scalar so colons and other plain-scalar-
107+
> unsafe characters in the agent name don't break YAML parsing.
108+
109+
## {{ pipeline_display_name }}
110+
111+
Should be replaced with the top-level pipeline `name:` value, including
112+
the `-$(BuildID)` suffix that ADO appends to every run, always emitted
113+
as a **YAML double-quoted scalar**. For an agent named
114+
`Daily safe-output smoke: noop`, this expands to:
115+
116+
```yaml
117+
name: "Daily safe-output smoke: noop-$(BuildID)"
118+
```
119+
120+
The quoting is unconditional because front-matter `name` values are
121+
free-form and frequently contain colons (which would otherwise be
122+
parsed as YAML mapping indicators). `$(...)` ADO macros pass through
123+
untouched — `$` has no special meaning inside a YAML double-quoted
124+
scalar and ADO expands the macro at queue time after YAML parsing.
125+
126+
This marker is intended only for the top-level `name:` line in
127+
`src/data/base.yml` and `src/data/1es-base.yml`; the job- and
128+
stage-level templates don't have a top-level pipeline name field.
129+
102130
## {{ engine_install_steps }}
103131

104132
Should be replaced with engine-specific pipeline steps to install the engine binary. Generated by `Engine::install_steps()`. For Copilot, this produces:

src/compile/common.rs

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -996,6 +996,40 @@ pub fn sanitize_filename(name: &str) -> String {
996996
.join("-")
997997
}
998998

999+
/// Emit `s` as a YAML double-quoted scalar (always quoted, never plain).
1000+
///
1001+
/// We always quote because the value is substituted into YAML positions
1002+
/// where colons and other plain-scalar-unsafe characters are common in
1003+
/// agent names (e.g. `"Daily safe-output smoke: noop"`). A bare scalar
1004+
/// like `name: Daily safe-output smoke: noop-$(BuildID)` is invalid YAML
1005+
/// because the second colon is interpreted as a mapping indicator.
1006+
///
1007+
/// `$(...)` ADO macros pass through untouched — `$` has no special meaning
1008+
/// inside a YAML double-quoted scalar and ADO expands the macro at queue
1009+
/// time after YAML parsing.
1010+
///
1011+
/// `reject_pipeline_injection` already strips newlines and template /
1012+
/// pipeline-command sequences from front-matter `name` values, so the
1013+
/// escape table only has to cover `\` and `"`. Tabs and ASCII control
1014+
/// characters are escaped too as a belt-and-braces measure.
1015+
pub fn yaml_double_quoted(s: &str) -> String {
1016+
let mut out = String::with_capacity(s.len() + 2);
1017+
out.push('"');
1018+
for ch in s.chars() {
1019+
match ch {
1020+
'\\' => out.push_str("\\\\"),
1021+
'"' => out.push_str("\\\""),
1022+
'\n' => out.push_str("\\n"),
1023+
'\r' => out.push_str("\\r"),
1024+
'\t' => out.push_str("\\t"),
1025+
c if (c as u32) < 0x20 => out.push_str(&format!("\\x{:02x}", c as u32)),
1026+
c => out.push(c),
1027+
}
1028+
}
1029+
out.push('"');
1030+
out
1031+
}
1032+
9991033
/// Default self-hosted pool for 1ES templates.
10001034
pub const DEFAULT_ONEES_POOL: &str = "AZS-1ES-L-MMS-ubuntu-22.04";
10011035
/// Default Microsoft-hosted VM image for non-1ES templates.
@@ -2856,6 +2890,12 @@ pub async fn compile_shared(
28562890
let checkout_steps = generate_checkout_steps(&front_matter.checkout);
28572891
let checkout_self = generate_checkout_self();
28582892
let agent_name = sanitize_filename(&front_matter.name);
2893+
// Top-level pipeline `name:` value in the generated YAML. Always emitted
2894+
// as a YAML double-quoted scalar because the front-matter `name` is
2895+
// free-form and frequently contains colons. The `-$(BuildID)` suffix is
2896+
// embedded so ADO appends the build ID to every run.
2897+
let pipeline_display_name =
2898+
yaml_double_quoted(&format!("{}-$(BuildID)", front_matter.name));
28592899

28602900
// 3. Run extension validations
28612901
for ext in extensions {
@@ -3069,6 +3109,7 @@ pub async fn compile_shared(
30693109
("{{ checkout_repositories }}", &checkout_steps),
30703110
("{{ agent }}", &agent_name),
30713111
("{{ agent_name }}", &front_matter.name),
3112+
("{{ pipeline_display_name }}", &pipeline_display_name),
30723113
("{{ agent_description }}", &front_matter.description),
30733114
("{{ engine_run }}", &engine_run),
30743115
("{{ engine_run_detection }}", &engine_run_detection),
@@ -3996,6 +4037,69 @@ mod tests {
39964037
assert_eq!(sanitize_filename("test_case"), "test-case");
39974038
}
39984039

4040+
// ─── yaml_double_quoted ──────────────────────────────────────────────────
4041+
4042+
#[test]
4043+
fn test_yaml_double_quoted_plain_string() {
4044+
assert_eq!(yaml_double_quoted("hello"), r#""hello""#);
4045+
}
4046+
4047+
#[test]
4048+
fn test_yaml_double_quoted_string_with_colon_is_safe() {
4049+
// The bug this helper exists to fix: an agent name like
4050+
// "Daily safe-output smoke: noop" must not be emitted bare in the
4051+
// top-level pipeline `name:` line, where the second colon would
4052+
// be parsed as a YAML mapping indicator.
4053+
assert_eq!(
4054+
yaml_double_quoted("Daily safe-output smoke: noop-$(BuildID)"),
4055+
r#""Daily safe-output smoke: noop-$(BuildID)""#
4056+
);
4057+
}
4058+
4059+
#[test]
4060+
fn test_yaml_double_quoted_escapes_backslash() {
4061+
assert_eq!(yaml_double_quoted(r"a\b"), r#""a\\b""#);
4062+
}
4063+
4064+
#[test]
4065+
fn test_yaml_double_quoted_escapes_double_quote() {
4066+
assert_eq!(yaml_double_quoted(r#"say "hi""#), r#""say \"hi\"""#);
4067+
}
4068+
4069+
#[test]
4070+
fn test_yaml_double_quoted_escapes_whitespace_controls() {
4071+
assert_eq!(yaml_double_quoted("a\nb"), r#""a\nb""#);
4072+
assert_eq!(yaml_double_quoted("a\rb"), r#""a\rb""#);
4073+
assert_eq!(yaml_double_quoted("a\tb"), r#""a\tb""#);
4074+
}
4075+
4076+
#[test]
4077+
fn test_yaml_double_quoted_escapes_other_control_chars() {
4078+
// Bell (0x07) is a low ASCII control char — should escape as \x07.
4079+
assert_eq!(yaml_double_quoted("a\u{0007}b"), r#""a\x07b""#);
4080+
}
4081+
4082+
#[test]
4083+
fn test_yaml_double_quoted_passes_through_ado_macros() {
4084+
// $(BuildID), $(Build.SourcesDirectory) etc. have no special meaning
4085+
// inside a YAML double-quoted scalar; ADO expands them at queue time
4086+
// after YAML parsing.
4087+
assert_eq!(
4088+
yaml_double_quoted("$(Build.BuildId)/$(System.JobId)"),
4089+
r#""$(Build.BuildId)/$(System.JobId)""#
4090+
);
4091+
}
4092+
4093+
#[test]
4094+
fn test_yaml_double_quoted_passes_through_unicode() {
4095+
// Non-ASCII characters pass through as-is — YAML 1.2 supports UTF-8
4096+
// in double-quoted scalars natively.
4097+
assert_eq!(
4098+
yaml_double_quoted("résumé — 你好"),
4099+
r#""résumé — 你好""#
4100+
);
4101+
}
4102+
39994103
// ─── generate_pr_trigger ─────────────────────────────────────────────────
40004104

40014105
#[test]

src/data/1es-base.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
# This template extends the 1ES Unofficial Pipeline Template with Copilot CLI,
33
# AWF network isolation, and MCP Gateway — matching the standalone pipeline model.
44

5-
name: {{ agent_name }}-$(BuildID)
5+
name: {{ pipeline_display_name }}
66
{{ parameters }}
77
{{ schedule }}
88
{{ pr_trigger }}

src/data/base.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11

2-
name: {{ agent_name }}-$(BuildID)
2+
name: {{ pipeline_display_name }}
33
{{ parameters }}
44
resources:
55
repositories:

tests/compiler_tests.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ fn assert_required_markers(content: &str) {
6969
"{{ checkout_repositories }}",
7070
"{{ allowed_domains }}",
7171
"{{ source_path }}",
72-
"{{ agent_name }}",
72+
"{{ pipeline_display_name }}",
7373
"{{ engine_run }}",
7474
"{{ compiler_version }}",
7575
"{{ integrity_check }}",

tests/safe-outputs/add-build-tag.lock.yml

Lines changed: 8 additions & 8 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/safe-outputs/add-pr-comment.lock.yml

Lines changed: 8 additions & 8 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/safe-outputs/comment-on-work-item.lock.yml

Lines changed: 8 additions & 8 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)