fix(yara): scan only edit replacement text + tighten PII/host rule precision#818
Closed
gewenyu99 wants to merge 1 commit into
Closed
fix(yara): scan only edit replacement text + tighten PII/host rule precision#818gewenyu99 wants to merge 1 commit into
gewenyu99 wants to merge 1 commit into
Conversation
…ecision
The pi-path YARA false positives from the 2026-07-07 triage (109 blocks / 26 runs on pii_in_capture_call, 22/11 on hardcoded_posthog_host). The triage plan assumed these were warlock rules, but the pi fence scans with the wizard-local yara-scanner — all five remark-reported FP classes trace here:
1. pi edit scanning covered oldText: security.ts scanned JSON.stringify(edits), which includes the pre-existing file content being replaced. That blocked edits whose surroundings held a violation ("capture edit adjacent to existing identify() PII") and even edits REMOVING one ("pre-existing violations blocking their own replacement"). Now scans only the newText of each edit — same semantics as the anthropic path, which scans new_string only.
2. pii_in_capture_call was far looser than warlock's equivalent: /\.capture\([^)]{0,200}email/i matched the word anywhere in the call — event names (capture('email_verified')), substrings of benign keys, minimal-payload auth events, and $set payloads. The $set match was the worst: our own runtime notes tell the agent to move PII onto the person via identify()/$set, so complying produced another block. Ported warlock's top-level-key-in-first-object patterns (with the same known-FN tradeoff) and dropped the $set patterns.
3. hardcoded_posthog_host now permits host literals in fallback position (process.env.X || '…', ?? '…', python or '…') — the documented pattern where the env var still wins.
Every remark-reported FP is a test fixture in the new yara-scanner.test.ts; true-positive regression cases included.
Generated-By: PostHog Code
Task-Id: bcb96bcb-e37f-421d-841d-738647c73b8e
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
gewenyu99
requested review from
sarahxsanders
and removed request for
sarahxsanders
Collaborator
Author
|
BAd PR |
gewenyu99
added a commit
that referenced
this pull request
Jul 7, 2026
…(stacked on #804) The YARA guard split out of #816, rebuilt on top of #804's warlock-based pi fence, plus the one surviving piece of the closed #818: 1. Repeat-block escalation: agents retry identically-blocked payloads (~4.2 blocks per affected run) despite the prompt saying not to. Both fences (pi pre-execution scan, anthropic PreToolUse Bash hook) now hash the payloads they block; an identical retry gets an escalated reason ("this exact content was ALREADY blocked — change the code, not the retry"), and from the third attempt the agent is told to note it in the setup report and move on. The escalation decorates #804's remediation-bearing block message, never replaces it. Tracker lives in yara-hooks, shared by both fences; policy denies are unaffected. 2. Edit scanning scoped to replacement text: #804 still scans JSON.stringify(input.edits), which includes each edit's oldText — pre-existing file content. That blocked edits whose surroundings contained a violation and even edits REMOVING one (field FPs: capture edits adjacent to existing identify() PII; pre-existing violations blocking their own replacement). Now scans only the joined newText, matching the anthropic path which scans new_string only. Generated-By: PostHog Code Task-Id: bcb96bcb-e37f-421d-841d-738647c73b8e
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item D from the 2026-07-07 triage — YARA precision on the pi path (109
pii_in_capture_callblocks across 26 runs; 22hardcoded_posthog_hostblocks across 11 runs).Repo note: the triage plan filed this under warlock, but the pi fence scans with the wizard-local
src/lib/yara-scanner.ts(the regex port, no WASM) — all five remark-reported FP classes trace to this repo. Warlock'sposthog_pii_in_capture_callalready has the tighter shape; this PR ports that discipline back.1. pi edit scanning covered the OLD text (
pi/security.ts)preExecutionYaraBlockscannedJSON.stringify(input.edits)— and pi's edit input is[{oldText, newText}], so the scan included pre-existing file content being replaced. Two FP classes from the remarks fall out of that directly:identify()PII" — violation in the anchor context, not the changeNow scans only the joined
newText, matching the anthropic path (which scansnew_string/contentonly).2.
pii_in_capture_calltightened to warlock's shapeThe local patterns matched a PII word anywhere within 200 chars of
.capture(— so event names (capture('email_verified')), substrings of benign keys (emailFieldCount), minimal-payload auth events, and$setpayloads all fired. The$setmatch was self-defeating:PI_RUNTIME_NOTEStells the agent a PII block means "move that field onto the person viaidentify()/$set" — complying produced another block, which is a chunk of the ~4.2-retries-per-affected-run loop.Ported warlock's top-level-key-in-first-object patterns (same documented FN tradeoff: top-level PII alongside
$setin one call), keptidentify()scoped to sensitive PII only, dropped the$setpatterns.3.
hardcoded_posthog_hostpermits fallback positionprocess.env.NEXT_PUBLIC_POSTHOG_HOST || 'https://us.i.posthog.com'is the documented pattern — the env var still wins. A lookbehind skips literals directly preceded by||,??, or pythonor; bare literals (api_host: 'https://us.i.posthog.com') still block.Tests
Every remark-reported FP is a fixture in the new
src/lib/__tests__/yara-scanner.test.ts(benignlocation, auth events, event names,$set, fallback hosts), plus true-positive regressions and two pi-security tests for the oldText/newText scoping.pnpm build && pnpm testgreen (1222 tests).Verify in the field: blocks-per-affected-run on the switchboard YARA tile (target <1.5, currently ~4.2), combined with the repeat-block escalation in #816.
🤖 Generated with Claude Code