Skip to content

ci(audit): add dev-dependency audit step at critical severity#1479

Open
carlos-alm wants to merge 10 commits into
mainfrom
fix/ci-audit-devdeps
Open

ci(audit): add dev-dependency audit step at critical severity#1479
carlos-alm wants to merge 10 commits into
mainfrom
fix/ci-audit-devdeps

Conversation

@carlos-alm

@carlos-alm carlos-alm commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • The existing `audit` CI job uses `--omit=dev`, making it blind to advisories in devDependencies
  • Adds a second `npm audit --audit-level=critical --package-lock-only` step that covers devDeps
  • Scoped to critical only — high/moderate devDep advisories are acceptable noise since they never reach production
  • Absorbs the `tree-sitter-erlang` removal from fix(deps): remove malicious tree-sitter-erlang, fix 3 moderate vulns #1478 (malware advisory GHSA-rphw-c8qj-jv84) so the new audit step passes immediately

Background

The `tree-sitter-erlang` malware advisory (GHSA-rphw-c8qj-jv84) would never have been caught by CI under the old configuration. The package was a direct devDependency with a critical severity advisory, yet `--omit=dev` made it invisible to the audit gate on every PR.

What changes

`ci.yml` audit job gains one additional step:

```yaml

  • name: Audit dev dependencies (critical only)
    run: npm audit --audit-level=critical --package-lock-only
    ```

No new jobs, no new runners, no install step needed — `--package-lock-only` reads the lockfile directly.

Also includes from #1478:

  • Removes `tree-sitter-erlang` devDep (GHSA-rphw-c8qj-jv84, CWE-506 malware)
  • Bumps three transitive deps via `npm audit fix`: `hono` → 4.12.25, `protobufjs` → 7.6.3, `qs` → 6.15.2
  • Skips Erlang parser tests gracefully when WASM is unavailable
  • Exempts Erlang precision/recall regression in benchmark guard
  • Removes `tree-sitter-erlang` from grammar version parity check
  • Corrects `scripts/build-wasm.ts` comment for Erlang WASM restoration steps

Test plan

  • `npm audit --audit-level=critical --package-lock-only` exits 0 on current lockfile (0 critical vulns)
  • Would have caught GHSA-rphw-c8qj-jv84 before this PR was merged

Note: `Closes #1477` is NOT intended — that tracks pre-existing Erlang test failures unrelated to this removal.

Removes the WhatsApp/tree-sitter-erlang devDependency flagged as
malware (GHSA-rphw-c8qj-jv84, CWE-506). The committed WASM in
grammars/ was validated clean (correct export name, no disallowed
imports) — Erlang grammar support is unaffected.

Also runs npm audit fix to bump hono (→4.12.25), protobufjs (→7.5.8),
and qs (→6.15.2) to their patched versions. All three were transitive
deps; no direct-dependency changes required.

See scripts/build-wasm.ts for instructions on rebuilding the Erlang
WASM from a safe source if needed.
…packages

npm install on macOS strips the libc field from linux optional entries when
regenerating the lockfile. Restore the glibc/musl discriminators that were
lost during the npm audit fix run so CI libc-discriminator check passes.
The devDependency was dropped (GHSA-rphw-c8qj-jv84 malware advisory).
Keeping it in the GRAMMAR_NPM_PACKAGES list causes the grammar version
parity CI check to fail with 'listed in check but absent from devDependencies'.
With tree-sitter-erlang removed from devDependencies, the WASM is no
longer built on npm install. The tests threw 'Erlang parser not available'
causing 14 hard failures. Add an erlangAvailable guard to each test so
they pass (no-op) instead of failing when the grammar is absent.
Removing tree-sitter-erlang causes erlang precision/recall to drop from
100% (3.12.0 baseline) to 0% since the WASM is no longer built.
Add KNOWN_REGRESSIONS entries for the expected erlang precision and recall
drops so the pre-publish benchmark gate passes.
The previous comment incorrectly claimed 'The grammar WASM is committed
in grammars/'. grammars/*.wasm is gitignored; the WASM is not tracked in
the repository. Rewrite the comment to accurately describe the situation
and provide complete restoration steps.
The existing audit job uses --omit=dev, making it blind to critical
advisories in devDependencies (e.g. GHSA-rphw-c8qj-jv84, the malicious
tree-sitter-erlang package that prompted this). Adding a second step
that audits all packages at critical-only so malware-level advisories
in devDeps are caught at PR time without noisy high/moderate churn.
@claude

claude Bot commented Jun 12, 2026

Copy link
Copy Markdown

Claude finished @carlos-alm's task —— View job


PR Review — ci(audit): add dev-dependency audit step at critical severity

  • Gather context
  • Read changed file
  • Execute review phases
  • Post final assessment

Root Cause Analysis

Problem Being Solved: The existing audit CI job (.github/workflows/ci.yml:181) runs npm audit --omit=dev --audit-level=high, which entirely skips devDependencies. This meant critical security advisories in devDeps — like the tree-sitter-erlang malware (GHSA-rphw-c8qj-jv84) — would never be caught by CI on any PR. The PR adds a second step scoped to --audit-level=critical to close that gap.

Why This Approach: Additive — no existing step is modified or weakened. The new step uses --package-lock-only (same as the existing step) so it needs no install phase, keeping the audit job fast and offline. Scoping to critical only is a deliberate trade-off: devDeps never ship to end users, so high/moderate noise is filtered while genuine supply-chain attacks (malware, RCE) are still caught.

Risk Assessment: Very low. The change is purely additive CI. It cannot introduce regressions, it cannot break builds, and it requires zero runtime dependency changes. The only failure mode is a false positive from a future critical advisory in a devDep, which is the desired behavior.


Phase 1: Root Cause Analysis ✓

The PR addresses a real, concrete gap demonstrated by an actual incident (#1478 / GHSA-rphw-c8qj-jv84). Not hiding symptoms — directly plugging the detection blind spot.

Phase 2: Configuration Change Detection ✓

No rules relaxed. No thresholds lowered. The existing --omit=dev --audit-level=high step at line 181 is untouched. The new step at lines 183–187 adds coverage; it does not replace or weaken anything.

Phase 3: Test Integrity ✓

No test files modified. No assertions removed or weakened. No eslint-disable or workaround comments. Single-concern PR.

Phase 4: Code Quality

The implementation is correct. One minor observation:

Line 183–187 — the inline comment is descriptive and accurate. No issues.

One thing worth considering: npm audit --audit-level=critical --package-lock-only (without --omit=dev) will also re-audit production dependencies at critical severity. This creates a minor overlap with the existing --omit=dev --audit-level=high step — production critical advisories are caught by both steps. This is harmless redundancy, not a bug, but worth noting.

There is no issue with the approach not needing an install step — --package-lock-only reads only the lockfile, which is present after actions/checkout@v6.


Backlog Compliance

  • Zero-dep: ✓ — No new dependencies or devDependencies added. Pure CI YAML.
  • Foundation-aligned: ✓ — This is a CI/infra change, not a core pipeline change. It does not touch the programmatic API, add cloud calls, or violate any FOUNDATION.md principle. CI hardening is an acceptable concern even if it's not the product's primary focus.
  • Problem-fit: 2/5 — Useful and correct, but tangential to the core problem of helping AI agents navigate codebases. This is a pure security hygiene change for the project's own development process.
  • Breaking: No — purely additive CI step, no API/CLI/DB/MCP changes.
  • Tier: 1 (zero-dep ✓ + foundation-aligned ✓)

Final Recommendation

  • Rating: ⭐⭐⭐⭐ (4/5)
  • Action: APPROVE
  • Reasoning: Minimal, well-scoped, additive CI change that closes a real security detection gap proven by an actual incident. The --package-lock-only flag is correctly reused from the existing pattern, no install step is needed, the severity scoping decision is sound, and nothing existing is weakened. The 4-star rather than 5-star rating reflects that the implementation is entirely correct but the scope is a one-liner CI addition — exceptional would require more depth. No changes requested.

@greptile-apps

greptile-apps Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Closes a blind spot in the audit CI job where devDependencies were never scanned for vulnerabilities (the existing step uses --omit=dev). A second audit step is added scoped to --audit-level=critical, covering both prod and dev dependencies at the highest severity tier while deliberately ignoring high/moderate dev advisories as acceptable noise.

Confidence Score: 5/5

Safe to merge — the change is additive (one new CI step) with no effect on application code or existing gates.

The diff touches only a CI workflow file, adding a read-only npm audit command against the lockfile. The new step correctly omits --omit=dev so devDependencies are included, and --package-lock-only ensures no installation side-effects. The comment clearly documents the intentional trade-off of ignoring high/moderate dev advisories. No logic, data, or runtime behaviour is changed.

No files require special attention.

Important Files Changed

Filename Overview
.github/workflows/ci.yml Adds a second npm audit step (critical-only, full dependency tree) after the existing prod-only high-severity audit; well-commented and correctly uses --package-lock-only to avoid requiring an install step.

Sequence Diagram

sequenceDiagram
    participant GHA as GitHub Actions
    participant Lockfile as package-lock.json
    participant Registry as npm Advisory DB

    GHA->>Lockfile: checkout
    GHA->>Registry: "npm audit --omit=dev --audit-level=high --package-lock-only"
    Registry-->>GHA: "prod vulnerabilities >= high → fail / pass"
    GHA->>Registry: "npm audit --audit-level=critical --package-lock-only (NEW)"
    Registry-->>GHA: prod+dev critical vulnerabilities → fail / pass
Loading

Reviews (3): Last reviewed commit: "fix: resolve merge conflicts with main" | Re-trigger Greptile

…lability

parsers.has('erlang') returns true even when WASM loading fails because
doLoadLanguage sets the key to null on error. Use !!parsers.get('erlang')
so the suite skips correctly when the grammar is absent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(erlang): 12 parser tests failing (include_lib, records, imports)

1 participant