Skip to content

DEVGUIDE.md

Latest commit

 

History

History
249 lines (183 loc) · 9.71 KB

DEVGUIDE.md

File metadata and controls

249 lines (183 loc) · 9.71 KB
Raw

Developer Guide

This guide is the maintainer-oriented reference for gh-aw-threat-detection.

It is intentionally shorter than the parent gh-aw guide and only covers the surfaces that exist in this repository.

Scope

This repository contains the threat detection component used by GitHub Agentic Workflows.

Core responsibilities:

  • read agent artifacts from an artifacts directory
  • evaluate those artifacts for prompt injection, secret leakage, and malicious patch risk
  • produce a machine-readable detection result before downstream safe-outputs handling proceeds

The canonical behavior specification for this repository lives in specs/threat-detection-spec.md.

Development Environment

Preferred setup:

The dev container keeps model-related tooling available, including GitHub CLI, Docker, Copilot CLI, and optional Vertex-backed Claude setup. Vertex credentials are optional and configured by setting VERTEX_API_JSON before container creation.

Local Setup

git clone https://github.com/github/gh-aw-threat-detection.git
cd gh-aw-threat-detection
make deps
make build

Full Maintainer Setup

make deps-dev
make test
make build

Daily Workflow

Common commands:

make build          # build the CLI
make test           # run Go tests
make lint           # run go vet
make fmt            # format Go code
make security-scan  # run gosec and govulncheck
make agent-finish   # maintainer validation flow

What make agent-finish currently runs:

  • deps-dev
  • fmt
  • lint
  • build
  • test
  • security-scan

Use make help to see the full list of maintained targets.

Repository Layout

/
├── .devcontainer/       # Codespaces and Dev Container setup
├── .github/workflows/   # CI and release workflows
├── cmd/threat-detect/   # CLI entrypoint
├── pkg/artifacts/       # Artifact loading and validation
├── pkg/detector/        # Threat detection logic and prompt templates
├── pkg/engine/          # AI engine abstraction and adapters
├── scratchpad/          # Design references retained from gh-aw where still relevant
├── skills/              # Small repo-relevant agent skills
├── specs/               # Threat detection specification
└── Makefile             # Build and maintainer commands

Detection-Focused Development Notes

Inputs To The Detection Module

The detector operates on an artifacts directory. The current expected shape is documented in README.md and specs/threat-detection-spec.md, including:

  • aw-prompts/prompt.txt
  • agent_output.json
  • optional aw-*.patch
  • optional aw-*.bundle
  • optional comment-memory/*.md

When changing artifact handling, review:

In-Session Result Reporting (report-result)

The agentic CLI engine path (copilot, claude, codex) gives the detection model an in-session, validated reporting channel instead of relying solely on post-hoc transcript scraping:

  • threat-detect report-result is an internal subcommand (dispatched in main() before global flag parsing; intentionally omitted from --help). It validates the verdict against the existing result schema, then atomically records canonical JSON to the path in --result-file / THREAT_DETECTION_RESULT_FILE. Invalid input prints THREAT_DETECTION_RESULT_ERROR: and exits non-zero without recording (exit 2); a missing sink path exits 3. The first valid write wins (idempotent); a later valid call prints "already recorded" without overwriting.
  • Before each engine run, pkg/engine provisions a threat_detection_result wrapper script on PATH (provisionResultTool) that execs report-result, and sets THREAT_DETECTION_RESULT_FILE. watchResultSink polls the sink and cancels the engine subprocess as soon as a valid result is recorded (early termination). A subprocess kill is treated as success when a valid sink result exists.
  • analyzeWithRetries prefers the sink result (detector.ReadResultFile) and only falls back to detector.ParseResult transcript scraping when the sink is absent or invalid. Helpers live in pkg/detector/result.go (WriteResultFile, ReadResultFile, BuildResultFromReport, ValidateReportFields).
  • Claude is invoked with --allowed-tools Bash when the sink is enabled so it can execute the wrapper; engines that cannot run shell tools fall back to the legacy THREAT_DETECTION_RESULT:{...} transcript line, which remains supported.

Safe Outputs Context

Threat detection sits upstream of safe-outputs. This repo does not implement the full safe-outputs system, but changes here should preserve the assumptions made by downstream consumers.

Useful references:

Maintainer Guidance

When changing code in this repository:

  • keep behavior aligned with specs/threat-detection-spec.md
  • prefer small, local packages and targeted tests
  • update README or spec text when behavior changes
  • preserve the JSON result contract unless the spec intentionally changes it

Useful retained references:

Troubleshooting

Common reset flow:

make clean
make deps
make build
make test

If tooling is missing:

  • run make deps-dev to install maintainer dependencies
  • reopen the dev container if local environment drift is suspected

Release Process

This section stays in place even though the release flow is still being built out.

Promotion Model

Releases follow a prerelease → promote model:

  1. Create tag (manual) — a maintainer triggers the Create Release Tag workflow via Actions → Create Release Tag → Run workflow and selects a patch or minor bump. The workflow validates main and pushes the next vX.Y.Z tag.

  2. Build & Publish (automated) — pushing a tag matching v* triggers the release workflow. It builds the threat-detect-linux-amd64 binary, attaches it (plus checksums.txt) to a prerelease on GitHub, and records the asset sha256 in the release notes. The release-publish environment gate pauses the workflow before publishing so maintainers can abort if needed.

  3. Promote (manual) — after verifying the prerelease, a maintainer triggers the promote-release workflow via Actions → Promote Release → Run workflow, entering the tag name. This workflow (gated by the release-promote environment):

    • verifies the release is still a prerelease
    • re-downloads the asset and verifies its sha256 against the recorded value
    • marks the GitHub release as stable and explicitly selects it as Latest (--prerelease=false --latest)

The GitHub "Latest" release pointer only moves when a maintainer explicitly promotes. This gives the team time to validate a release before it becomes the default for users downloading the latest stable asset.

Branch builds: the rolling main pre-release

In addition to release tags, every push to main triggers the publish-main workflow, which builds the binary and republishes a single rolling main pre-release:

  • The main pre-release always carries the threat-detect-linux-amd64 asset built from the most recent successful build from main, versioned main-<shortsha>.

These are unverified branch builds. The main pre-release is not eligible for promotion. The Latest stable release pointer is unaffected by this workflow and continues to track the most recently promoted release.

CI Support

The main CI workflow in .github/workflows/ci.yml runs:

  • go vet
  • go test -race
  • go build

Release Steps

  1. Go to Actions → Create Release Tag → Run workflow.
  2. Select patch or minor. The visible major option may still appear in the workflow UI, but it is currently rejected by the workflow until major releases are enabled.
  3. Run the workflow. It validates main, computes the next semantic-version tag, and pushes it to trigger the release workflow.

After the tag is pushed:

  1. Approve the release-publish environment gate when the workflow pauses.
  2. Verify the prerelease on the Releases page and test the version-tagged threat-detect-linux-amd64 asset.
  3. When satisfied, go to Actions → Promote Release, enter the tag, and run the workflow. Approve the release-promote environment gate.
  4. Confirm the Latest release now resolves to the new version.

If a promoted release is later found unsafe, delete or replace the affected GitHub release and promote a known-good version instead.