feat(redact): path-ignore for the pre-push credential scan#2328
feat(redact): path-ignore for the pre-push credential scan#2328frederik-kaster-noygear wants to merge 6 commits into
Conversation
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
Machine-local, comma-separated supplement to the committed .gstack/redact-prepush-ignore file. Empty by default (no behavior change). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Large generated data files (multi-MB CSV/Parquet exports) oversize the 1 MiB scan cap and trip a false-positive HIGH engine.input_too_large, forcing a full GSTACK_REDACT_PREPUSH=skip that also skips real code in the same push. Add an opt-in, auditable path-ignore. Ignore globs merge two sources: the committed .gstack/redact-prepush-ignore file (one glob per line, # comments — reviewable in git history, shared across contributors) and the machine-local redact_prepush_ignore_globs config key. Matched repo-root-relative paths (Bun.Glob, so ** spans directories) are dropped from the scanned diff via a git :(exclude) pathspec, and a one-line stderr notice always reports how many paths / bytes were skipped — an exemption is never silent. Preserves the garrytan#1946 fail-closed invariant: an ignore glob is an explicit opt-in; anything NOT matched still goes through scan(), so a bare oversize non-ignored file still blocks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ignored large file passes with a skip notice; non-ignored large file still fail-closed blocks; a real secret in a non-ignored file still blocks even with ignore rules present; an explicit ignore glob exempts its file (auditable opt-in); the machine-local config key is an additional source; no ignore rules leaves default behavior unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial review found a reproduced fail-open: a plain `:(exclude)<name>` git pathspec is glob-ENABLED, so a legal on-disk filename holding a `?`/`[`/`]` metacharacter was interpreted as a PATTERN and could drop OTHER, non-ignored files from the scan — a real credential in a sibling file (verified with a live AKIA/PEM key) slipped past with exit 0. Use `:(top,exclude,literal)` for excludes and `:(top,literal)` for the skip-accounting diff. `literal` disables glob interpretation so only the exact ignored path is exempted; `top` anchors at the repo root so excludes line up with the root-relative names from `--name-only` regardless of cwd (the positive pathspec is `:/`, also root-anchored). Also pass `core.quotePath=false` on the name-only diff so non-ASCII filenames arrive as raw UTF-8 and the glob matches (otherwise git C-quotes them and the user's ignore rule silently no-ops). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(g) a non-ASCII filename is exempted by its glob (core.quotePath=false).
(h) a `?`-bearing ignored filename does NOT drag a sibling holding a real
secret out of the scan — the sibling credential still blocks (exit 1).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gstack-config reads values whitespace-delimited, so a space after a comma silently drops the trailing globs. Document it and steer multi-glob users to the committed .gstack/redact-prepush-ignore file (one glob per line). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7b49a2e to
7cce208
Compare
Problem
Pushing large generated data files (e.g.
prospecting/exports/**/*.csv, 4–8 MB each) is blocked by the pre-push credential guard with a HIGHengine.input_too_largefinding — a false positive that has nothing to do with credentials. The scan engine fail-closes on input over the 1 MiB cap, and the hook concatenates all added-line text before scanning, so one large generated file oversizes the whole push. The only escape isGSTACK_REDACT_PREPUSH=skipon every such push, which defeats the guard entirely (it also skips real code in the same push).Change
Add an opt-in, auditable path-ignore to
bin/gstack-redact-prepush. Generated data files can be exempted from the pre-push scan without weakening credential scanning on code.Ignore globs merge two sources:
.gstack/redact-prepush-ignorecommitted at the repo root (one glob per line,#comments, blank lines skipped) — the recommended mechanism: versioned, reviewable in git history, shared across contributors.redact_prepush_ignore_globsconfig key (gstack-config, comma-separated) — a machine-local supplement in~/.gstack(never committed).Globs match repo-root-relative paths via
Bun.Glob(so**spans directories, e.g.prospecting/exports/**/*.csv). Matched paths are dropped from the scanned diff via a git:(exclude)pathspec, so excluded content never reachesscan(). Every exemption prints a one-line stderr notice reporting how many paths / bytes were skipped and appends an audit record toprepush-skip.jsonl— an ignore is never silent.Fail-closed invariant preserved (#1946)
An ignore glob is an explicit, auditable opt-in. Anything not matched by an ignore glob still goes through
scan(), so:The accepted tradeoff: a secret inside an explicitly ignored path is exempted — but that path is a versioned opt-in and the skip is always reported.
Tests
Six new cases in
test/redact-prepush-hook.test.ts(all green, plus the existing 15):Docs
bin/gstack-config: newredact_prepush_ignore_globsdefault + annotated header entry.CLAUDE.md: redaction-guard section documents the config key and the committed-file mechanism.Config bump / CHANGELOG
None here — VERSION and CHANGELOG are branch-scoped and land at
/shiptime per repo convention, not during development.CI note
This PR comes from a personal fork by a non-collaborator, so the eval/E2E jobs won't receive base-repo secrets and will fail with empty-env auth. Per CONTRIBUTING / CLAUDE.md, a maintainer can move the branch to a base-repo branch to run those jobs. Free tests (
bun test) pass; the two unrelated failures observed locally (sidepanel-terminal.jsforceRestart, parent-process watchdog) reproduce on a clean tree with these changes stashed, so they are pre-existing and unrelated.Context: hit while shipping noygear/medical-mockup PR #282, which commits
prospecting/exports/<version>/<STATE>/suspects-*.csv(4–8 MB each). That repo will setredact_prepush_ignore_globs = prospecting/exports/**/*.csv(or the committed file) once this lands.🤖 Generated with Claude Code