Skip to content

Commit 8478453

Browse files
jamesadevineCopilot
jamesadevine
and
Copilot
authored
feat: Download releases from GitHub. (#17)
* feat: migrate AWF download to GitHub Releases and align checksum model Replace the legacy DownloadPipelineArtifact@2 task (pipeline 2450, project 4x4) for the AWF binary with a curl-based download from GitHub Releases at github.com/github/gh-aw-firewall. Add a pinned AWF_VERSION constant in common.rs with a {{ firewall_version }} template marker. Also migrate the ado-aw compiler checksum verification from per-binary .sha256 files to checksums.txt with --ignore-missing, matching the gh-aw-firewall publishing convention. Both standalone and 1ES templates are updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: add assertions for AWF GitHub Releases download and checksums.txt Verify the standalone template no longer references ADO pipeline 2450 or DownloadPipelineArtifact, and instead downloads AWF from GitHub Releases with firewall_version marker and checksums.txt verification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: update release workflow and AGENTS.md for checksums.txt Update release.yml to publish checksums.txt instead of per-binary .sha256 files. Add {{ firewall_version }} marker documentation to AGENTS.md and update the Network Isolation section to reflect GitHub Releases for AWF. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: harden checksum verification and address review feedback Add grep confirmation after sha256sum to ensure at least one file was verified, preventing silent success on naming mismatches. Add compiled output test asserting AWF GitHub Releases URL appears in final YAML. Add clarifying comment for no-op firewall_version in 1ES compiler. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 87bb243 commit 8478453

8 files changed

Lines changed: 104 additions & 57 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,13 +48,13 @@ jobs:
4848
set -euo pipefail
4949
cd target/release
5050
cp ado-aw ado-aw-linux-x64
51-
sha256sum ado-aw-linux-x64 > ado-aw-linux-x64.sha256
51+
sha256sum ado-aw-linux-x64 > checksums.txt
5252
5353
- name: Upload release assets
5454
env:
5555
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
5656
run: |
5757
gh release upload ${{ needs.release-please.outputs.tag_name }} \
5858
target/release/ado-aw-linux-x64 \
59-
target/release/ado-aw-linux-x64.sha256 \
59+
target/release/checksums.txt \
6060
--clobber

‎AGENTS.md‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -592,7 +592,18 @@ The generated pipelines download the compiler binary from:
592592
https://github.com/githubnext/ado-aw/releases/download/v{VERSION}/ado-aw-linux-x64
593593
```
594594

595-
A SHA256 checksum file (`ado-aw-linux-x64.sha256`) is also downloaded and verified to ensure binary integrity. This replaces the previous approach of downloading from an internal ADO pipeline artifact.
595+
A `checksums.txt` file is also downloaded and verified via `sha256sum -c checksums.txt --ignore-missing` to ensure binary integrity.
596+
597+
## {{ firewall_version }}
598+
599+
Should be replaced with the pinned version of the AWF (Agentic Workflow Firewall) binary (defined as `AWF_VERSION` constant in `src/compile/common.rs`). This version is used to construct the GitHub Releases download URL for the AWF binary.
600+
601+
The generated pipelines download the AWF binary from:
602+
```
603+
https://github.com/github/gh-aw-firewall/releases/download/v{VERSION}/awf-linux-x64
604+
```
605+
606+
A `checksums.txt` file is also downloaded and verified via `sha256sum -c checksums.txt --ignore-missing` to ensure binary integrity.
596607

597608
### 1ES-Specific Template Markers
598609

@@ -965,7 +976,7 @@ mcp-servers:
965976

966977
Network isolation is provided by AWF (Agentic Workflow Firewall), which provides L7 (HTTP/HTTPS) egress control using Squid proxy and Docker containers. AWF restricts network access to a whitelist of approved domains.
967978

968-
The `ado-aw` compiler binary is distributed via [GitHub Releases](https://github.com/githubnext/ado-aw/releases) with SHA256 checksum verification. The AWF binary is downloaded from an internal ADO pipeline (pipeline 2450, branch `ms/main`, artifact `gh-aw-firewall-linux-x64`). Docker is sourced via the `DockerInstaller@0` ADO task.
979+
The `ado-aw` compiler binary is distributed via [GitHub Releases](https://github.com/githubnext/ado-aw/releases) with SHA256 checksum verification. The AWF binary is distributed via [GitHub Releases](https://github.com/github/gh-aw-firewall/releases) with SHA256 checksum verification. Docker is sourced via the `DockerInstaller@0` ADO task.
969980

970981
### Default Allowed Domains
971982

‎src/compile/common.rs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -455,6 +455,11 @@ pub fn sanitize_filename(name: &str) -> String {
455455
/// Default pool name
456456
pub const DEFAULT_POOL: &str = "AZS-1ES-L-MMS-ubuntu-22.04";
457457

458+
/// Version of the AWF (Agentic Workflow Firewall) binary to download from GitHub Releases.
459+
/// Update this when upgrading to a new AWF release.
460+
/// See: https://github.com/github/gh-aw-firewall/releases
461+
pub const AWF_VERSION: &str = "0.23.1";
462+
458463
/// Generate source path for the execute command.
459464
///
460465
/// Returns a path using `{{ workspace }}` as the base, which gets resolved

‎src/compile/onees.rs‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ use std::path::Path;
1717

1818
use super::Compiler;
1919
use super::common::{
20-
self, DEFAULT_POOL, compute_effective_workspace, generate_copilot_params,
20+
self, AWF_VERSION, DEFAULT_POOL, compute_effective_workspace, generate_copilot_params,
2121
generate_checkout_self, generate_checkout_steps, generate_ci_trigger,
2222
generate_pipeline_path, generate_pipeline_resources, generate_pr_trigger,
2323
generate_repositories, generate_schedule, generate_source_path,
@@ -117,6 +117,8 @@ displayName: "Finalize""#,
117117
let compiler_version = env!("CARGO_PKG_VERSION");
118118
let replacements: Vec<(&str, &str)> = vec![
119119
("{{ compiler_version }}", compiler_version),
120+
// No-op for 1ES (template doesn't use AWF), but included for forward-compatibility
121+
("{{ firewall_version }}", AWF_VERSION),
120122
("{{ pool }}", &pool),
121123
("{{ schedule }}", &schedule),
122124
("{{ pr_trigger }}", &pr_trigger),

‎src/compile/standalone.rs‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ use std::path::Path;
1414

1515
use super::Compiler;
1616
use super::common::{
17-
self, DEFAULT_POOL, compute_effective_workspace, generate_copilot_params,
17+
self, AWF_VERSION, DEFAULT_POOL, compute_effective_workspace, generate_copilot_params,
1818
generate_cancel_previous_builds, generate_checkout_self, generate_checkout_steps,
1919
generate_ci_trigger, generate_pipeline_path, generate_pipeline_resources, generate_pr_trigger,
2020
generate_repositories, generate_schedule, generate_source_path, generate_working_directory,
@@ -123,6 +123,7 @@ impl Compiler for StandaloneCompiler {
123123
let compiler_version = env!("CARGO_PKG_VERSION");
124124
let replacements: Vec<(&str, &str)> = vec![
125125
("{{ compiler_version }}", compiler_version),
126+
("{{ firewall_version }}", AWF_VERSION),
126127
("{{ pool }}", &pool),
127128
("{{ setup_job }}", &setup_job),
128129
("{{ teardown_job }}", &teardown_job),

‎templates/1es-base.yml‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -58,16 +58,16 @@ extends:
5858
COMPILER_VERSION="{{ compiler_version }}"
5959
DOWNLOAD_DIR="$(Pipeline.Workspace)/agentic-pipeline-compiler"
6060
DOWNLOAD_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/ado-aw-linux-x64"
61-
CHECKSUM_URL="${DOWNLOAD_URL}.sha256"
61+
CHECKSUM_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/checksums.txt"
6262
6363
mkdir -p "$DOWNLOAD_DIR"
6464
echo "Downloading ado-aw v${COMPILER_VERSION} from GitHub Releases..."
6565
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64" "$DOWNLOAD_URL"
66-
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64.sha256" "$CHECKSUM_URL"
66+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
6767
6868
echo "Verifying checksum..."
6969
cd "$DOWNLOAD_DIR"
70-
sha256sum --check ado-aw-linux-x64.sha256
70+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
7171
mv ado-aw-linux-x64 ado-aw
7272
chmod +x ado-aw
7373
displayName: "Download agentic pipeline compiler (v{{ compiler_version }})"
@@ -171,16 +171,16 @@ extends:
171171
COMPILER_VERSION="{{ compiler_version }}"
172172
DOWNLOAD_DIR="$(Pipeline.Workspace)/agentic-pipeline-compiler"
173173
DOWNLOAD_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/ado-aw-linux-x64"
174-
CHECKSUM_URL="${DOWNLOAD_URL}.sha256"
174+
CHECKSUM_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/checksums.txt"
175175
176176
mkdir -p "$DOWNLOAD_DIR"
177177
echo "Downloading ado-aw v${COMPILER_VERSION} from GitHub Releases..."
178178
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64" "$DOWNLOAD_URL"
179-
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64.sha256" "$CHECKSUM_URL"
179+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
180180
181181
echo "Verifying checksum..."
182182
cd "$DOWNLOAD_DIR"
183-
sha256sum --check ado-aw-linux-x64.sha256
183+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
184184
mv ado-aw-linux-x64 ado-aw
185185
chmod +x ado-aw
186186
displayName: "Download agentic pipeline compiler (v{{ compiler_version }})"
@@ -315,16 +315,16 @@ extends:
315315
COMPILER_VERSION="{{ compiler_version }}"
316316
DOWNLOAD_DIR="$(Pipeline.Workspace)/agentic-pipeline-compiler"
317317
DOWNLOAD_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/ado-aw-linux-x64"
318-
CHECKSUM_URL="${DOWNLOAD_URL}.sha256"
318+
CHECKSUM_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/checksums.txt"
319319
320320
mkdir -p "$DOWNLOAD_DIR"
321321
echo "Downloading ado-aw v${COMPILER_VERSION} from GitHub Releases..."
322322
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64" "$DOWNLOAD_URL"
323-
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64.sha256" "$CHECKSUM_URL"
323+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
324324
325325
echo "Verifying checksum..."
326326
cd "$DOWNLOAD_DIR"
327-
sha256sum --check ado-aw-linux-x64.sha256
327+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
328328
mv ado-aw-linux-x64 ado-aw
329329
chmod +x ado-aw
330330
displayName: "Download agentic pipeline compiler (v{{ compiler_version }})"

‎templates/base.yml‎

Lines changed: 43 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -56,16 +56,16 @@ jobs:
5656
COMPILER_VERSION="{{ compiler_version }}"
5757
DOWNLOAD_DIR="$(Pipeline.Workspace)/agentic-pipeline-compiler"
5858
DOWNLOAD_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/ado-aw-linux-x64"
59-
CHECKSUM_URL="${DOWNLOAD_URL}.sha256"
59+
CHECKSUM_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/checksums.txt"
6060
6161
mkdir -p "$DOWNLOAD_DIR"
6262
echo "Downloading ado-aw v${COMPILER_VERSION} from GitHub Releases..."
6363
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64" "$DOWNLOAD_URL"
64-
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64.sha256" "$CHECKSUM_URL"
64+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
6565
6666
echo "Verifying checksum..."
6767
cd "$DOWNLOAD_DIR"
68-
sha256sum --check ado-aw-linux-x64.sha256
68+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
6969
mv ado-aw-linux-x64 ado-aw
7070
chmod +x ado-aw
7171
displayName: "Download agentic pipeline compiler (v{{ compiler_version }})"
@@ -172,23 +172,25 @@ jobs:
172172
- task: DockerInstaller@0
173173
displayName: "Install Docker"
174174

175-
- task: DownloadPipelineArtifact@2
176-
displayName: "Download AWF (Agentic Workflow Firewall)"
177-
name: awfdrop
178-
inputs:
179-
source: "specific"
180-
project: "4x4"
181-
pipeline: 2450
182-
runVersion: "latestFromBranch"
183-
branchName: "refs/heads/ms/main"
184-
artifact: "gh-aw-firewall-linux-x64"
185-
targetPath: "$(Pipeline.Workspace)/awf"
186-
187175
- bash: |
188-
chmod +x "$(Pipeline.Workspace)/awf/awf"
176+
AWF_VERSION="{{ firewall_version }}"
177+
DOWNLOAD_DIR="$(Pipeline.Workspace)/awf"
178+
DOWNLOAD_URL="https://github.com/github/gh-aw-firewall/releases/download/v${AWF_VERSION}/awf-linux-x64"
179+
CHECKSUM_URL="https://github.com/github/gh-aw-firewall/releases/download/v${AWF_VERSION}/checksums.txt"
180+
181+
mkdir -p "$DOWNLOAD_DIR"
182+
echo "Downloading AWF v${AWF_VERSION} from GitHub Releases..."
183+
curl -fsSL -o "$DOWNLOAD_DIR/awf-linux-x64" "$DOWNLOAD_URL"
184+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
185+
186+
echo "Verifying checksum..."
187+
cd "$DOWNLOAD_DIR"
188+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
189+
mv awf-linux-x64 awf
190+
chmod +x awf
189191
echo "##vso[task.prependpath]$(Pipeline.Workspace)/awf"
190-
"$(Pipeline.Workspace)/awf/awf" --version || echo "AWF binary ready"
191-
displayName: "Setup AWF"
192+
./awf --version || echo "AWF binary ready"
193+
displayName: "Download AWF (Agentic Workflow Firewall) v{{ firewall_version }}"
192194
193195
- bash: |
194196
docker pull ghcr.io/github/gh-aw-firewall/squid:latest
@@ -312,40 +314,42 @@ jobs:
312314
COMPILER_VERSION="{{ compiler_version }}"
313315
DOWNLOAD_DIR="$(Pipeline.Workspace)/agentic-pipeline-compiler"
314316
DOWNLOAD_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/ado-aw-linux-x64"
315-
CHECKSUM_URL="${DOWNLOAD_URL}.sha256"
317+
CHECKSUM_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/checksums.txt"
316318
317319
mkdir -p "$DOWNLOAD_DIR"
318320
echo "Downloading ado-aw v${COMPILER_VERSION} from GitHub Releases..."
319321
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64" "$DOWNLOAD_URL"
320-
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64.sha256" "$CHECKSUM_URL"
322+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
321323
322324
echo "Verifying checksum..."
323325
cd "$DOWNLOAD_DIR"
324-
sha256sum --check ado-aw-linux-x64.sha256
326+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
325327
mv ado-aw-linux-x64 ado-aw
326328
chmod +x ado-aw
327329
displayName: "Download agentic pipeline compiler (v{{ compiler_version }})"
328330
329331
- task: DockerInstaller@0
330332
displayName: "Install Docker"
331333

332-
- task: DownloadPipelineArtifact@2
333-
displayName: "Download AWF (Agentic Workflow Firewall)"
334-
name: awfdrop
335-
inputs:
336-
source: "specific"
337-
project: "4x4"
338-
pipeline: 2450
339-
runVersion: "latestFromBranch"
340-
branchName: "refs/heads/ms/main"
341-
artifact: "gh-aw-firewall-linux-x64"
342-
targetPath: "$(Pipeline.Workspace)/awf"
343-
344334
- bash: |
345-
chmod +x "$(Pipeline.Workspace)/awf/awf"
335+
AWF_VERSION="{{ firewall_version }}"
336+
DOWNLOAD_DIR="$(Pipeline.Workspace)/awf"
337+
DOWNLOAD_URL="https://github.com/github/gh-aw-firewall/releases/download/v${AWF_VERSION}/awf-linux-x64"
338+
CHECKSUM_URL="https://github.com/github/gh-aw-firewall/releases/download/v${AWF_VERSION}/checksums.txt"
339+
340+
mkdir -p "$DOWNLOAD_DIR"
341+
echo "Downloading AWF v${AWF_VERSION} from GitHub Releases..."
342+
curl -fsSL -o "$DOWNLOAD_DIR/awf-linux-x64" "$DOWNLOAD_URL"
343+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
344+
345+
echo "Verifying checksum..."
346+
cd "$DOWNLOAD_DIR"
347+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
348+
mv awf-linux-x64 awf
349+
chmod +x awf
346350
echo "##vso[task.prependpath]$(Pipeline.Workspace)/awf"
347-
"$(Pipeline.Workspace)/awf/awf" --version || echo "AWF binary ready"
348-
displayName: "Setup AWF"
351+
./awf --version || echo "AWF binary ready"
352+
displayName: "Download AWF (Agentic Workflow Firewall) v{{ firewall_version }}"
349353
350354
- bash: |
351355
docker pull ghcr.io/github/gh-aw-firewall/squid:latest
@@ -504,16 +508,16 @@ jobs:
504508
COMPILER_VERSION="{{ compiler_version }}"
505509
DOWNLOAD_DIR="$(Pipeline.Workspace)/agentic-pipeline-compiler"
506510
DOWNLOAD_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/ado-aw-linux-x64"
507-
CHECKSUM_URL="${DOWNLOAD_URL}.sha256"
511+
CHECKSUM_URL="https://github.com/githubnext/ado-aw/releases/download/v${COMPILER_VERSION}/checksums.txt"
508512
509513
mkdir -p "$DOWNLOAD_DIR"
510514
echo "Downloading ado-aw v${COMPILER_VERSION} from GitHub Releases..."
511515
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64" "$DOWNLOAD_URL"
512-
curl -fsSL -o "$DOWNLOAD_DIR/ado-aw-linux-x64.sha256" "$CHECKSUM_URL"
516+
curl -fsSL -o "$DOWNLOAD_DIR/checksums.txt" "$CHECKSUM_URL"
513517
514518
echo "Verifying checksum..."
515519
cd "$DOWNLOAD_DIR"
516-
sha256sum --check ado-aw-linux-x64.sha256
520+
sha256sum -c checksums.txt --ignore-missing | grep -q ": OK"
517521
mv ado-aw-linux-x64 ado-aw
518522
chmod +x ado-aw
519523
displayName: "Download agentic pipeline compiler (v{{ compiler_version }})"

‎tests/compiler_tests.rs‎

Lines changed: 27 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,8 +145,26 @@ fn test_compiled_yaml_structure() {
145145
"Template should download the compiler from GitHub Releases"
146146
);
147147
assert!(
148-
template_content.contains("sha256sum --check"),
149-
"Template should verify checksum of downloaded compiler"
148+
template_content.contains("sha256sum -c checksums.txt --ignore-missing | grep -q \": OK\""),
149+
"Template should verify checksum using checksums.txt with grep confirmation"
150+
);
151+
152+
// Verify AWF (Agentic Workflow Firewall) is downloaded from GitHub Releases, not ADO pipeline artifacts
153+
assert!(
154+
!template_content.contains("pipeline: 2450"),
155+
"Template should not reference ADO pipeline 2450 for the firewall"
156+
);
157+
assert!(
158+
!template_content.contains("DownloadPipelineArtifact"),
159+
"Template should not use DownloadPipelineArtifact task"
160+
);
161+
assert!(
162+
template_content.contains("github.com/github/gh-aw-firewall/releases"),
163+
"Template should download AWF from GitHub Releases"
164+
);
165+
assert!(
166+
template_content.contains("{{ firewall_version }}"),
167+
"Template should contain firewall_version marker"
150168
);
151169
}
152170

@@ -348,7 +366,13 @@ fn test_compiled_output_no_unreplaced_markers() {
348366
);
349367
assert!(
350368
compiled.contains("github.com/githubnext/ado-aw/releases"),
351-
"Compiled output should reference GitHub Releases"
369+
"Compiled output should reference GitHub Releases for the compiler"
370+
);
371+
372+
// Verify the AWF firewall version was correctly substituted
373+
assert!(
374+
compiled.contains("github.com/github/gh-aw-firewall/releases"),
375+
"Compiled output should reference GitHub Releases for AWF"
352376
);
353377

354378
let _ = fs::remove_dir_all(&temp_dir);

0 commit comments

Comments
 (0)