Skip to content

security: fix rust-secrets — 12.5% of Rust repos were never scanned#518

Merged
hyperpolymath merged 2 commits into
mainfrom
security/rust-secrets-scope-and-fp
Jul 21, 2026
Merged

security: fix rust-secrets — 12.5% of Rust repos were never scanned#518
hyperpolymath merged 2 commits into
mainfrom
security/rust-secrets-scope-and-fp

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Found while verifying the 201-repo secret-scanner re-pin sweep. Two defects in rust-secrets.

1. Silent no-scan — the serious one

grep -rn --include="*.rs" -E "$pattern" src/     # <- root src/ only

A Cargo workspace keeps its code in crates/*/src, so src/ does not exist at the root. grep exits 2, the enclosing if reads that as no match, and the job passes.

Measured across the 64 Rust repos in the sweep: 8 (12.5%) have no root src/ and were never scanned at all.

Proven on a fixture — a planted secret in crates/core/src/lib.rs:

old job this PR
pub const SERVICE_TOKEN: &str = "…" in crates/core/src/lib.rs exit 0 ✅ green detected

Another gate that could not fail.

2. False positive on the correct pattern

let.*api_key.*=.*" matches any let api_key = line containing a quote — including:

let api_key = env::var("CLOUDGUARD_API_KEY").ok();

…which is precisely the practice this job exists to enforce. This was the only rust-secrets failure in the entire sweep (cloudguard-server src/main.rs:59). A gate whose sole route to green is to stop reading from the environment is inverted.

Exemptions

Mirror the four layers already documented on shell-secrets, and match on the value or an explicit pragma — never on a file path, because a path exemption blinds the job to a real secret in the same file (the failure mode .gitleaks.toml was rewritten to avoid in #512).

Layer Rationale
env-lookup RHS the correct pattern. Anchored to the assignment and stopped at ;, so a literal secret with an unrelated env::var mention later on the line still trips
URL values an OAuth endpoint is public, not a credential (const MS_TOKEN_URL = "https://login.microsoftonline.com/…" tripped const.*TOKEN.*=.*" 4× in one repo)
comment lines documentation of a pattern is not a live secret
// scanner-allow: rust-secrets explicit, per-line, auditable

Warn-first on the widened scope

Hits outside ./src are newly visible, so they warn until 2026-08-21 — the same cutoff and idiom as check-package-policy.sh / check-docs-presence.sh — rather than reddening repos that were never actually being scanned. Hits inside ./src block exactly as before, so there is no regression.

The advisory branch prints the finding and says NOT YET ENFORCED; it never claims a pass. A malformed cutoff refuses to run rather than defaulting to warn forever.

Verified, not assumed

Canary suite run against the script extracted from this YAML (yaml.safe_load → execute), not a copy:

PASS  env::var + comment + pragma + OAuth URL -> ALLOWED
PASS  secret in ./src                        -> BLOCKS
PASS  secret in crates/                      -> ADVISORY today
PASS  secret in crates/                      -> BLOCKS after cutoff
PASS  literal + env mention after ;          -> BLOCKS  (exemption not over-broad)
PASS  malformed cutoff                       -> refuses to run
6 passed, 0 failed

Replayed over all 64 Rust repos in the sweep: 0 regressions today, 1 fixed (cloudguard-server).

Follow-up for repo owners (before 2026-08-21)

Three repos carry fixture-shaped hits outside ./src and need a one-line pragma; each is named in the run output today:

  • echidnabotfuzz/fuzz_targets/fuzz_hmac.rs:17 (fuzz corpus)
  • filesoupplugins/secret-scanner/src/lib.rs:159 (the plugin's own detection fixture)
  • heterogenous-mobile-computingexamples/basic_usage.rs:61 (demo of a blocked query)

Pre-existing and unchanged by this PR: aerie and conative-gating fail under both old and new (request["api_key"] dynamic lookup; a scanner's own r#"…"# test fixtures).

🤖 Generated with Claude Code

Two defects in the `rust-secrets` job, both found while verifying the
201-repo secret-scanner re-pin sweep.

1. SILENT NO-SCAN (the serious one). The job ran
   `grep -rn --include="*.rs" -E "$pattern" src/`. A Cargo workspace keeps
   its code in `crates/*/src`, so `src/` does not exist at the root — grep
   exits 2, the enclosing `if` reads that as "no match", and the job passes.
   Measured across the 64 Rust repos in the sweep: 8 (12.5%) have no root
   `src/` and were therefore never scanned at all. Verified on a fixture: a
   planted `pub const SERVICE_TOKEN: &str = "…"` in `crates/core/src/lib.rs`
   exits 0 under the old job. Another gate that could not fail.

   Now scans every `*.rs` in the tree, pruning target/ vendor/ .git/.

2. FALSE POSITIVE ON THE CORRECT PATTERN. `let.*api_key.*=.*"` matches any
   `let api_key =` line containing a quote — including
   `let api_key = env::var("CLOUDGUARD_API_KEY").ok();`, which is precisely
   the practice this job exists to enforce. That was the *only* rust-secrets
   failure in the whole sweep (cloudguard-server src/main.rs:59). A gate whose
   sole way to go green is to stop reading from the environment is inverted.

Exemptions added mirror the four layers already documented on shell-secrets,
and match on the VALUE or an explicit pragma — never on a file path, because
a path exemption blinds the job to a real secret in the same file (the failure
mode .gitleaks.toml was rewritten to avoid in #512):
  - env-lookup RHS, anchored to the assignment and stopped at `;` so a literal
    secret with an unrelated env::var mention later on the line still trips;
  - URL values (an OAuth endpoint is public, not a credential);
  - comment lines;
  - inline `// scanner-allow: rust-secrets`.

Widened scope is warn-first to 2026-08-21 — the same cutoff and idiom as
check-package-policy.sh and check-docs-presence.sh — so repos that were never
actually being scanned get a window rather than an immediate red. The advisory
branch prints the finding and says NOT YET ENFORCED; it never claims a pass.
A malformed cutoff refuses to run rather than defaulting to warn forever.

Verified, not assumed:
  - 6/6 canary cases pass against the script extracted from this YAML (not a
    copy): correct-pattern allowed; secret in ./src blocks; secret in crates/
    advisory today and blocking after the cutoff; literal-plus-env-mention
    still blocks; malformed cutoff refuses.
  - Replayed over all 64 Rust repos in the sweep: 0 regressions today,
    1 fixed (cloudguard-server).
  - 3 repos (echidnabot, filesoup, heterogenous-mobile-computing) carry
    fixture-shaped hits outside ./src and need a one-line pragma before
    2026-08-21; each is listed in the run output today.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
hyperpolymath added a commit that referenced this pull request Jul 21, 2026
`Registry Verify` has been failing on **`main`** since #515 merged at
~12:34 today.

#515 changed tracked files under `rhodium-standard-repositories/`
without regenerating the derived registry, so the RSR entry's
`source_hash` — a sha256 over `git ls-files -s <home>` — no longer
matches the tree:

```
DRIFT: .machine_readable/REGISTRY.a2ml is stale — run 'just registry'
```

**Every open PR inherits this**, because `pull_request` checks run
against the merge ref. It is the sole cause of the current red on #518.

## The change

One line, produced by `bash scripts/build-registry.sh` with no manual
editing:

```diff
 name = "RSR — Rhodium Standard Repositories"
 home = "rhodium-standard-repositories/"
-source_hash = "sha256:b285134b…7b9b7abe"
+source_hash = "sha256:02b75b57…c382f1e"
```

## Verified

| | `scripts/build-registry.sh --check` |
|---|---|
| `origin/main` | `DRIFT: .machine_readable/REGISTRY.a2ml is stale` |
| this commit | `OK: registry + topology are in sync with the file
tree.` (exit 0) |

Run history confirms the regression point — `main` green at 04:48, red
at 12:37, with `feat/scaffold-lifecycle-canon` red at 12:34.

Worth considering as follow-up: `just hooks-install` (the pre-commit
guard the failure message already points at) would have caught this
before push.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@hyperpolymath
hyperpolymath merged commit f26a519 into main Jul 21, 2026
5 checks passed
@hyperpolymath
hyperpolymath deleted the security/rust-secrets-scope-and-fp branch July 21, 2026 12:46
@sonarqubecloud

Copy link
Copy Markdown

hyperpolymath added a commit that referenced this pull request Jul 21, 2026
…static` (#523)

Follow-up to #518, from triaging every remaining `rust-secrets` failure
in the estate.

## The false positives

The patterns matched `= ... "` — a quote **anywhere** after an equals.
That flags code which hardcodes nothing at all:

```rust
let api_key = request["api_key"].as_str().unwrap_or("");   // a lookup
let api_key_file = temp_dir.path().join("config.js");      // a path
Query::new("Here's my api_key=sk-12345");                    // a demo string
```

A hardcoded secret is an assignment **directly to a string literal**.
Each pattern now requires the quote to follow the `=`, with a minimum
value length of 8 — a credential shorter than that is not a credential.

## The missed detection (the more important half)

`const.*KEY.*=` never matched:

```rust
static AUTH_KEY: &str = "abcdef0123456789abcdef0123456789";
```

That is a genuine hardcoded secret the job was **silently passing**. I
found it because a 5-secret canary fixture detected only 4 — not by
reading the regex. `static` and `mut` are now accepted alongside
`const`/`let`.

## Verified, not assumed

Canary run against the script **extracted from this YAML**
(`yaml.safe_load` → execute), not a hand-copy:

```
PASS  5 real hardcoded secrets           -> BLOCKS   (detected 5/5)
PASS  clean (env::var, comment, pragma, OAuth URL)
PASS  secret in ./src                    -> BLOCKS
PASS  crates/                            -> ADVISORY today
PASS  crates/                            -> BLOCKS after cutoff
PASS  literal + env mention after ;      -> BLOCKS
PASS  malformed cutoff                   -> refuses to run
7 passed, 0 failed
```

Replayed over **all 64 Rust repos** at both `2026-07-21` and post-cutoff
`2026-09-01`: **0 regressions, 3 verdicts fixed** (`aerie`, and
`heterogenous-mobile-computing` post-cutoff).

## What deliberately still fails

`conative-gating`, `echidnabot`, `filesoup` carry genuinely
secret-shaped fixtures — a fuzz corpus, and two secret-scanners' own
detection test data inside `r#"…"#` raw strings. Those are **real
matches on real secret-shaped literals**, so they get the explicit `//
scanner-allow: rust-secrets` pragma rather than a pattern loophole that
would weaken the gate for everyone. Separate one-line PRs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant