Skip to content

WORKFLOWS.md

Latest commit

 

History

History
306 lines (215 loc) · 13 KB

WORKFLOWS.md

File metadata and controls

306 lines (215 loc) · 13 KB

GitHub Actions Workflows

This document describes the GitHub Actions workflows used in this repository and explains the recent improvements made for better performance and maintainability.

Overview

The repository uses four main workflows:

  1. CI (ci.yml) - Continuous Integration for code quality and deployment
  2. Version Bump (version-bump.yml) - Automatic or manual version updates with changelog
  3. Create Version Tag (create-version-tag.yml) - Creates release tags for merged version bump PRs
  4. Release (release.yml) - Build and publish releases

CI Workflow

Trigger: Push to main, Pull Requests, Manual dispatch

Jobs

Parallel Quality Checks (runs concurrently)

  1. Lint - Code linting with Deno
  2. Format - Code formatting check with Deno
  3. Type Check - TypeScript type checking for all entry points
  4. Test - Run test suite with coverage; coverage artifact uploaded on both PRs and main push
  5. Security - Trivy vulnerability scanning
  6. Frontend Lint & Test (frontend-lint-test) - Angular frontend lint and test
  7. Frontend Build (frontend-build) - Angular frontend build and artifact upload
  8. Validate Cloudflare Schema - Runs deno task schema:cloudflare and verifies that docs/api/cloudflare-schema.yaml (Cloudflare API Shield schema generated from the OpenAPI spec) is up to date

PR-Only Parallel Job (needs frontend-build artifact)

  1. Verify Deploy - Cloudflare Worker build dry-run (deno task wrangler:verify); runs on PRs only, waits for the frontend-build artifact but otherwise runs in parallel with the quality checks above

Sequential Jobs (run after all checks pass)

  1. CI Gate - Python script verifying all upstream jobs passed or were acceptably skipped; blocks publish and deploy
  2. Publish - Publish to JSR (main only, after CI gate passes)
  3. Deploy - Deploy adblock-compiler backend Worker to Cloudflare (main only, when enabled, after CI gate passes)
  4. Deploy Frontend - Deploy adblock-frontend SSR Worker to Cloudflare; downloads the frontend-dist artifact, runs scripts/build-worker.sh to inject/remove the CF_WEB_ANALYTICS_TOKEN placeholder, then runs pnpm run deploy. Triggered on main push (after ci-gate + frontend-build pass); also deployed on tag push via release.yml deploy-frontend job (full pnpm run build + token injection + deploy)

Composite Actions

All composite actions live in .github/actions/ and are called with uses: ./.github/actions/<name>.

Action Purpose Used by
deno-install 3-attempt retry loop for deno install with DENO_TLS_CA_STORE=system setup-deno-env
setup-deno-env Install Deno, cache ~/.cache/deno + ~/.deno, optionally run deno install lint-format, typecheck, test, validate-artifacts, check-slow-types, audit-public-surface, validate-migrations, verify-deploy, deploy, publish
setup-env Load .env / .env.<branch> files into the runner environment test, verify-deploy, deploy
setup-pnpm-node Setup pnpm, cache the pnpm store, and install Node.js frontend-lint-test, frontend-build
zta-checks Run all four ZTA security lint checks (wildcard CORS, hardcoded secrets, eval/new Function, string-interpolated SQL) zta-lint
validate-wrangler-toml Assert that wrangler.toml contains no placeholder binding IDs before deploying verify-deploy, deploy
deploy-worker Run D1 migrations, generate version, set up queues/R2, deploy tail worker + main worker, record outcome deploy

setup-pnpm-node

- name: Setup pnpm and Node.js
  uses: ./.github/actions/setup-pnpm-node
  with:
    node-version: '24.15'   # optional, defaults to '24.15'

zta-checks

- name: Run ZTA security checks
  uses: ./.github/actions/zta-checks

validate-wrangler-toml

- name: Validate wrangler.toml
  uses: ./.github/actions/validate-wrangler-toml

deploy-worker

- name: Deploy worker
  uses: ./.github/actions/deploy-worker
  with:
    github-sha: ${{ github.sha }}

Requires CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, and CF_WEB_ANALYTICS_TOKEN to be set in the job-level env: (inherited automatically; no need to pass as inputs).

Key Improvements

  • ✅ Parallelization: Lint, format, typecheck, test, and security scans run simultaneously
  • ✅ Proper Gating: ci-gate blocks publish/deploy until lint, format, typecheck, test, security, frontend-build, and verify-deploy all pass
  • ✅ Worker Build Verified on PRs: verify-deploy runs a Cloudflare Worker dry-run on every PR so Worker build failures are caught before merge
  • ✅ Composite Actions: Repeated step sequences extracted to .github/actions/ — no duplication across jobs
  • ✅ Modular CI: ci.yml reduced from 783 → 609 lines (−22%) by delegating step logic to focused composite actions
  • ✅ Separate Frontend Jobs: frontend-lint-test (lint + test) and frontend-build (build + artifact upload) run as independent parallel jobs so a lint failure does not block the build artifact from being produced
  • ✅ Frozen Lockfile: pnpm install --frozen-lockfile enforced — CI fails if pnpm-lock.yaml drifts from package.json
  • ✅ Coverage on PRs: Test coverage artifact uploaded on pull requests, not just main push
  • ✅ SHA-Pinned Actions: All third-party actions pinned to full commit SHAs with version comments (supply-chain hardening)
  • ✅ Better Caching: Includes deno.lock in cache key for more precise invalidation
  • ✅ Comprehensive Type Checking: Checks all entry points (index.ts, cli.ts, worker.ts, tail.ts)
  • ✅ Consolidated Worker Deployment: Main and tail Cloudflare Workers deployed from a single CI deploy job (no separate Pages deployment)
  • ✅ Frontend Worker CI Deployment: deploy-frontend job deploys adblock-frontend on every main push, after the frontend-build artifact is available; CF_WEB_ANALYTICS_TOKEN is injected/removed by scripts/build-worker.sh before wrangler deploy
  • ✅ Migration Error Handling: D1 migration retry loop distinguishes auth errors from transient failures and surfaces actionable remediation steps
  • ✅ wrangler.toml Validation: Both verify-deploy (PRs) and deploy (main) validate that no placeholder binding IDs exist before any Cloudflare operation

Performance Gains

  • Before: ~5-7 minutes (sequential execution)
  • After: ~2-3 minutes (parallel execution)
  • Improvement: ~40-50% faster

Release Workflow

Trigger: Push tags (v*), Manual dispatch with version input

Jobs

  1. Validate - Run full CI suite before building anything
  2. Build Binaries - Build native binaries for all platforms (parallel matrix)
  3. Build Docker - Build and push multi-platform Docker images
  4. Create Release - Generate GitHub release with all artifacts
  5. Deploy Frontend - Deploy adblock-frontend SSR Worker to Cloudflare after validate passes; full pnpm run build + scripts/build-worker.sh token injection + pnpm run deploy

Key Improvements

  • ✅ Pre-build Validation: Ensures code quality before expensive build operations
  • ✅ Better Caching: Per-target caching for binary builds
  • ✅ Simplified Asset Prep: Uses find instead of complex loop
  • ✅ Cleaner Structure: Removed verbose comments, organized logically

Performance Gains

  • Before: ~15-20 minutes (no validation, potential failures late)
  • After: ~12-15 minutes (early validation prevents wasted builds)
  • Improvement: Faster failure detection, ~20% reduction in failed build time

Version Bump Workflow

Trigger: Push to main, Manual dispatch

Jobs

  1. Version Bump - Automatically analyze commits and bump version, or manually specify bump type
  2. Trigger Release - Optionally trigger release workflow (if requested via manual dispatch)

Key Features

  • ✅ Automatic Detection: Uses conventional commits to determine version bump type
  • ✅ Manual Override: Can manually specify patch/minor/major bump
  • ✅ Changelog Generation: Automatically generates changelog entries from commits
  • ✅ PR-Based: Creates pull request with version changes for review
  • ✅ Skip Logic: Skips if [skip ci] or [skip version] in commit message

Conventional Commits Support

  • feat: → minor bump
  • fix: → patch bump
  • perf: → patch bump
  • feat!: or BREAKING CHANGE: → major bump

Changes from Previous Version

  • Consolidated: Merged auto-version-bump.yml and version-bump.yml into single workflow
  • Simplified: Single workflow handles both automatic and manual triggers
  • Improved: Better error handling and verification steps

Create Version Tag Workflow

Trigger: PR closed (for version bump PRs only)

Jobs

  1. Create Tag - Creates release tag when version bump PR is merged

Key Features

  • ✅ Automatic Tagging: Creates v<version> tag when version bump PR is merged
  • ✅ Idempotent: Checks if tag exists before creating
  • ✅ Cleanup: Deletes version bump branch after tagging
  • ✅ Release Trigger: Tag automatically triggers release workflow

Caching Strategy

All workflows now use an improved caching strategy:

key: deno-${{ runner.os }}-${{ hashFiles('deno.json', 'deno.lock') }}
restore-keys: |
    deno-${{ runner.os }}-

This ensures:

  • Cache is invalidated when dependencies change
  • Fallback to OS-specific cache if exact match not found
  • Faster dependency installation

Environment Variables

Common

  • DENO_VERSION: '2.x' - Deno version used across all workflows

CI Workflow

  • CLOUDFLARE_API_TOKEN - For Cloudflare deployments (optional)
  • CLOUDFLARE_ACCOUNT_ID - For Cloudflare deployments (optional)

Required Variables

  • ENABLE_CLOUDFLARE_DEPLOY - Repository variable to enable/disable Cloudflare deployments

Permissions

All workflows use minimal permissions following the principle of least privilege:

CI

  • contents: read - For checking out code
  • id-token: write - For JSR publishing (publish job only)
  • security-events: write - For uploading security scan results (security job only)

Release

  • contents: write - For creating releases and tags
  • packages: write - For publishing Docker images

Version Bump

  • contents: write - For committing version changes
  • actions: write - For triggering release workflow

Concurrency

All workflows use concurrency groups to prevent multiple runs on the same ref:

concurrency:
    group: ${{ github.workflow }}-${{ github.ref }}
    cancel-in-progress: true

This ensures:

  • Only one workflow runs per branch/PR at a time
  • Outdated runs are automatically cancelled when new commits are pushed
  • Saves CI minutes and prevents race conditions

Best Practices

When to Use Each Workflow

  1. CI: Automatically runs on every push/PR - no manual intervention needed
  2. Version Bump: Run manually when you want to bump the version
  3. Release: Automatically triggered by version tags, or run manually for specific versions

Recommended Release Process

  1. Make your changes on a feature branch
  2. Create a PR and wait for CI to pass
  3. Merge to main
  4. Version bump workflow automatically runs and creates a version bump PR
  5. Review and merge the version bump PR
  6. Create version tag workflow automatically creates the release tag
  7. Release workflow automatically builds and publishes the release

Or for manual version bump:

  1. Make your changes on a feature branch
  2. Create a PR and wait for CI to pass
  3. Merge to main
  4. Run "Version Bump" workflow manually with desired bump type
  5. Optionally check "Create a release after bumping" to skip the PR review step

Troubleshooting

Publish Fails with "Version Already Exists"

This is expected and not an error. The workflow treats this as success to allow re-running the workflow.

Deploy Jobs Don't Run

Check that ENABLE_CLOUDFLARE_DEPLOY repository variable is set to 'true' (as a string).

Binary Build Fails for ARM64 Linux

The ARM64 Linux build uses cross-compilation. If it fails, check Deno's compatibility with the target platform in the Deno release notes.

Migration Notes

If you're migrating from the old workflows:

Breaking Changes

  • Version bump no longer runs automatically on PR open
  • Example files are no longer automatically updated during version bump
  • Deploy jobs now combined into single job

Non-Breaking Changes

  • All existing secrets and variables work the same way
  • Workflow dispatch inputs are backwards compatible
  • Release process is unchanged

Future Improvements

Potential areas for further optimization:

  • Add workflow to automatically create PRs for dependency updates
  • Add scheduled security scanning (weekly)
  • Consider splitting test job by test type (unit vs integration)
  • Add benchmark tracking over time
  • Add automatic changelog generation
  • Add path-based filtering to skip frontend-build on backend-only PRs (currently blocked by verify-deploy's artifact dependency)