Skip to content

Add package filesystem evidence scanner#2189

Draft
jesse-merhi wants to merge 1 commit into
mainfrom
jesse/filesystem-evidence-scanner
Draft

Add package filesystem evidence scanner#2189
jesse-merhi wants to merge 1 commit into
mainfrom
jesse/filesystem-evidence-scanner

Conversation

@jesse-merhi
Copy link
Copy Markdown
Member

Summary

Adds a narrow pure scanner for package file contents that reports deterministic evidence for:

  • raw Node filesystem module imports and calls
  • OpenClaw fs-safe module imports and helper calls
  • bounded evidence buckets sorted by file and line

This intentionally does not add DB tables, background jobs, HTTP endpoints, admin UI, or CLI commands. Those can layer on top of this scanner in follow-up PRs.

Validation

  • bun run test convex/lib/packageFilesystemEvidenceScan.test.ts
  • bun run format:check
  • bun run lint
  • bunx tsc --noEmit --pretty false
  • git diff --cached --check

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
clawhub Ready Ready Preview, Comment May 13, 2026 1:03am

@Patrick-Erichsen
Copy link
Copy Markdown
Contributor

Nice, so this is to ensure plugin authors are using things like fs-safe correctly?

@github-actions
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please update it or it will be closed.

@github-actions github-actions Bot added the stale label May 20, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. labels May 22, 2026
@clawsweeper
Copy link
Copy Markdown

clawsweeper Bot commented May 22, 2026

Codex review: found issues before merge. Reviewed June 1, 2026, 1:00 AM ET / 05:00 UTC.

Summary
This PR adds a pure Convex package filesystem evidence scanner and tests that report raw Node fs and OpenClaw fs-safe usage with bounded evidence buckets.

Reproducibility: yes. for the PR findings: source inspection shows the unintegrated production export, pre-seeded fs namespace, and line-oriented import parsing, while live check runs show static and types-build failures on the PR head.

Review metrics: 2 noteworthy metrics.

  • Diff scope: 2 files added, 620 additions, 0 deletions. This is a new production security scanner plus tests, so integration and security-signal correctness matter before merge.
  • Head check failures: static and types-build failed. The failing gates align with the unreferenced production helper and need a real entry point or approved exemption.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🌊 off-meta tidepool
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Get maintainer agreement on whether filesystem evidence belongs in package static scan, admin dry-run, or local lint.
  • Wire the helper into the chosen entry point or mark it as a deliberate standalone target so static/dead-code gates pass.
  • [P2] Fix binding detection and add regressions for virtual fs objects and multi-line imports.

Risk before merge

  • [P1] If merged after integration without fixes, the scanner could misclassify package-local virtual fs objects as raw Node filesystem usage and miss formatted multi-line fs imports.
  • [P1] The new production helper is currently only test-referenced, so static/dead-code automation remains blocked until maintainers approve and wire an entry point.
  • [P1] Dedicated package filesystem evidence changes the security signal model and should land with a matching spec note or explicit maintainer decision.

Maintainer options:

  1. Choose the scanner entry point first (recommended)
    Decide whether this evidence belongs in package static scan, admin dry-run, or local lint, then wire the production helper there and update the intent note.
  2. Tighten filesystem binding detection
    Start raw fs bindings empty, bind only from confirmed Node fs imports/requires, and add regressions for virtual fs objects plus multi-line imports.
  3. Keep the draft paused
    Leave this PR unmerged until maintainers decide whether dedicated filesystem evidence is in scope for package scanning.

Next step before merge

  • The remaining blocker is a maintainer product/security decision about scanner scope and entry point, followed by focused correctness fixes.

Security
Cleared: The diff adds a pure text scanner and tests with no new dependency, secret handling, network access, install hook, or code execution path; security-signal correctness is covered by review findings.

Review findings

  • [P2] Wire the scanner into an approved entry point — convex/lib/packageFilesystemEvidenceScan.ts:78-85
  • [P2] Require a confirmed raw fs binding — convex/lib/packageFilesystemEvidenceScan.ts:213-219
  • [P2] Parse multi-line fs imports — convex/lib/packageFilesystemEvidenceScan.ts:109-113
Review details

Best possible solution:

Land a binding-aware scanner only through an approved package scan, admin dry-run, or local lint entry point, with regression tests and an updated package-scan intent note.

Do we have a high-confidence way to reproduce the issue?

Yes for the PR findings: source inspection shows the unintegrated production export, pre-seeded fs namespace, and line-oriented import parsing, while live check runs show static and types-build failures on the PR head.

Is this the best way to solve the issue?

No. The scanner is useful foundation work, but the maintainable path is to choose the approved package evidence entry point and fix binding-aware detection before merging it as a security signal.

Full review comments:

  • [P2] Wire the scanner into an approved entry point — convex/lib/packageFilesystemEvidenceScan.ts:78-85
    scanPackageFilesystemEvidence is exported from a production convex/lib file, but the PR adds no production/admin/CLI caller and tests are ignored by the normal Knip project graph. This leaves the static gate failing until the helper is wired into an approved package scan path or made an explicit standalone entry.
    Confidence: 0.93
  • [P2] Require a confirmed raw fs binding — convex/lib/packageFilesystemEvidenceScan.ts:213-219
    rawFsNamespaces starts with fs, so package-local code like const fs = makeVirtualFilesystem(); fs.readFile(path) would be reported as raw Node filesystem usage without any fs or node:fs import. Start empty and add bindings only from confirmed Node filesystem imports/requires.
    Confidence: 0.9
  • [P2] Parse multi-line fs imports — convex/lib/packageFilesystemEvidenceScan.ts:109-113
    The module reference and binding extraction operate on one physical line, so formatted imports with specifiers split across lines will not bind the raw fs helper or namespace and later filesystem calls can escape evidence collection. Accumulate import statements or use a parser-level approach before scanning calls.
    Confidence: 0.84

Overall correctness: patch is incorrect
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 9fc2da4dc437.

Label changes

Label justifications:

  • P2: This is a normal-priority package security scanner feature with limited blast radius but blocking integration and accuracy issues.
  • merge-risk: 🚨 security-boundary: The scanner is intended to produce package security evidence, and the current binding logic can produce false positives and false negatives if integrated.
  • merge-risk: 🚨 automation: The diff adds an unreferenced production Convex helper that leaves required static/dead-code automation failing on the PR head.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: Not applicable because this is a member-authored draft PR; the external contributor real-behavior proof gate does not apply.
Evidence reviewed

What I checked:

  • Live PR state: GitHub API shows this PR is open, draft, member-authored, and still at head dedb88f with labels for P2, waiting on author, security-boundary risk, and automation risk. (dedb88fa49c1)
  • Current main lacks this scanner: A current-main file search found no packageFilesystemEvidenceScan file, so the central proposed scanner is not already implemented on main. (9fc2da4dc437)
  • Diff scope: The PR adds exactly two files: the production scanner and its Vitest coverage, with 620 added lines and no deletions. (dedb88fa49c1)
  • Unintegrated production helper: The new scanner exports scanPackageFilesystemEvidence from a Convex production library file, but the PR body explicitly says it adds no DB tables, background jobs, HTTP endpoints, admin UI, or CLI commands. (convex/lib/packageFilesystemEvidenceScan.ts:78, dedb88fa49c1)
  • Static/dead-code policy: Knip treats convex/**/*.ts as production project files while tests are ignored unless KNIP_INCLUDE_TESTS is set, so a production helper referenced only by tests remains a static-gate problem. (knip.config.ts:22, 9fc2da4dc437)
  • Head check state: GitHub check runs for the PR head show static and types-build failed while unit/packages/e2e/playwright checks passed. (dedb88fa49c1)

Likely related people:

  • Patrick-Erichsen: Current-main blame ties the package static scan wrapper, package security status resolver, and package scan intent note to Patrick Erichsen commits around the v0.18.0 release. (role: recent package scanner/security contributor; confidence: high; commits: 875f026a2300, cc16d7fbd924; files: convex/lib/staticPublishScan.ts, convex/lib/packageSecurity.ts, specs/security-moderation.md)
  • jesse-merhi: Beyond opening this draft, Jesse Merhi has current-main package trust/security policy history in the package surface this scanner would feed into. (role: adjacent package trust contributor; confidence: medium; commits: 90de729fe135; files: convex/httpApiV1/packagesV1.ts, convex/lib/packageSecurity.ts, convex/packages.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper
Copy link
Copy Markdown

clawsweeper Bot commented May 22, 2026

ClawSweeper PR egg

🔥 Warming up: real-behavior proof passed; findings, security review, or rank-up moves are still in progress.

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.
What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@github-actions github-actions Bot removed the stale label May 23, 2026
@clawsweeper clawsweeper Bot added merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. labels May 23, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 1, 2026

This pull request has been automatically marked as stale due to inactivity.
Please update it or it will be closed.

@github-actions github-actions Bot added the stale label Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal backlog priority with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. stale status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants