Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/e2e-phase-7-post-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"playground-cli": patch
---

CI now validates the consumer install path after every stable release: `e2e-post-release.yml` fires on `release: published`, runs `install.sh`, and runs the same smoke tests as `e2e-release.yml` (Phase 7) but against the installed binary at `~/.polkadot/bin/dot`. Catches `install.sh` regressions that the prerelease/SEA-download path doesn't.
99 changes: 99 additions & 0 deletions .github/workflows/e2e-post-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
name: E2E (Post-Release)

on:
release:
types: [published]
workflow_dispatch:
inputs:
tag:
description: "Release tag to test (e.g. v0.16.0)"
required: true

permissions:
contents: read

jobs:
smoke:
name: "E2E · post-release-smoke"
if: github.event.release.prerelease != true || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 15
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4

- uses: actions/setup-node@v4
with:
node-version: "22"

- run: pnpm install

- name: Resolve release tag
id: tag
shell: bash
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
TAG="${{ inputs.tag }}"
else
TAG="${{ github.event.release.tag_name }}"
fi
echo "tag=$TAG" >> "$GITHUB_OUTPUT"

- name: Wait for SEA asset on the release
shell: /usr/bin/bash -euo pipefail {0}
run: |
TAG="${{ steps.tag.outputs.tag }}"
for i in $(seq 1 30); do
if gh release view "$TAG" --json assets --jq '.assets[].name' | grep -q "^dot-linux-x64$"; then
echo "Found dot-linux-x64 asset on attempt $i"
exit 0
fi
echo " attempt $i/30: waiting for asset..."
sleep 10
done
echo "::error::Timed out waiting for dot-linux-x64 asset on $TAG"
exit 1

- name: Install via install.sh (consumer path)
shell: bash
run: |
TAG="${{ steps.tag.outputs.tag }}"
# Pin to the just-released version; install.sh resolves "latest"
# by default which can lag behind a just-published release.
# install.sh runs `dot init` at the end; init requires an
# interactive QR scan and will exit non-zero in headless CI —
# that is expected. The binary is installed before init runs,
# so we tolerate the init failure and verify the binary below.
curl -fsSL "https://raw.githubusercontent.com/${{ github.repository }}/${{ github.sha }}/install.sh" \
| VERSION="$TAG" bash || true
# Hard-fail here if the binary was not installed — that IS a
# real regression in the install script's download/chmod path.
ls -la "$HOME/.polkadot/bin/dot"

- name: Compute installed-binary path
shell: bash
run: echo "DOT_E2E_BINARY=$HOME/.polkadot/bin/dot" >> "$GITHUB_ENV"

- name: Smoke test the installed binary
uses: nick-fields/retry@v3
with:
timeout_minutes: 5
max_attempts: 2
retry_wait_seconds: 15
command: pnpm exec vitest run --config e2e/vitest.config.ts e2e/cli/published.test.ts
env:
DOT_TAG: e2e-ci-post-release
DOT_TELEMETRY: "1"

- name: Upload forensic artefacts
if: always()
continue-on-error: true
uses: actions/upload-artifact@v4
with:
name: e2e-reports-post-release-smoke
path: e2e-reports/
retention-days: 7
if-no-files-found: ignore
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ These are things that aren't self-evident from reading the code and have bitten
- **CI workflow:** `.github/workflows/e2e.yml` — runs on PR / push:main / cron 06:00 UTC / workflow_dispatch.
- **CI matrix:** 13 cells across four matrices — `test-no-publish` (parallel: pr-install, pr-preflight, pr-mod, pr-init-session) + `test-publish` (max-parallel: 1: pr-deploy-frontend, pr-deploy-foundry) + `test-nightly-no-publish` (parallel, schedule/dispatch only: nightly-mod-miss, nightly-diagnostic, nightly-rejections, nightly-chaos-sigint) + `test-nightly-publish` (max-parallel: 1, schedule/dispatch only: nightly-deploy-hardhat, nightly-deploy-multi, nightly-chaos-rpc). Each cell runs a subset via `vitest -t "<pattern>"`.
- **Release smoke:** `.github/workflows/e2e-release.yml` fires on `release: prereleased`, downloads the `dot-linux-x64` SEA asset, and runs `e2e/cli/published.test.ts` against it. Validates the published binary before stable release.
- **Post-release smoke:** `.github/workflows/e2e-post-release.yml` fires on `release: published` (stable only — `prerelease != true`), waits for the SEA asset, runs `install.sh` (consumer install path via `VERSION=<tag> curl … | bash`), then runs `published.test.ts` against the installed `~/.polkadot/bin/dot`. Catches `install.sh` regressions that the prerelease/SEA-download path doesn't.
- **Test files:** `e2e/cli/*.test.ts` (vitest, spawned via `bun run src/index.ts`).
- **Reports directory:** `e2e-reports/junit.xml` + `e2e-reports/dot-runs.log` (gitignored).
- **Tag prefix:** `DOT_TAG=e2e-{ci|local}-{trigger}` so Sentry dashboards filter test traffic. The CLI plumbs `DOT_TAG` into the `cli.tag` root-span attribute via `src/telemetry-config.ts`.
Expand Down
Loading