fix(wizard): parse detection JSON defensively with Zod + retry - #1036
Draft
gewenyu99 wants to merge 1 commit into
Draft
Conversation
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
gewenyu99
changed the base branch from
main
to
posthog-code/tolerant-detection-json-parsing
July 30, 2026 13:28
The agentic detector's extractJson did a strict JSON.parse that threw a SyntaxError on any non-JSON or truncated agent reply. In a headless monorepo run that exception propagated to scopeInstallDirToProject, whose catch only logs and returns — so the wizard silently integrated against the repo root instead of the detected sub-project. Replace the single strict parse with a Zod-validated retry loop: - run detection, locate + JSON.parse the reply, validate against a Zod schema - on a malformed reply, re-prompt the agent with the specific problems - up to 3 attempts, then fall through best-effort (coerce the last reply that parsed as JSON; empty report if none did) instead of throwing parseAgentReport never throws and is exported for testing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Generated-By: PostHog Code Task-Id: ce1f323c-14f1-4ca2-85c4-072b9c0e2afb
gewenyu99
force-pushed
the
posthog-code/wizard-defensive-detection-parse
branch
from
July 30, 2026 15:53
4378de5 to
4aefff3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The agentic project detector's
extractJsonused a strictJSON.parse, so any non-JSON or truncated agent reply threw aSyntaxError. In a headless monorepo run that exception propagated up toscopeInstallDirToProject, whosecatchonly logs and returns — so the run quietly abandoned project scoping and integrated against the repo root instead of the detected sub-project.Why: Raised from a PostHog Signals inbox item flagging that detection's strict JSON parsing can crash the scan and, in the monorepo path, silently mis-target the wrong directory. This makes detection resilient to imperfect LLM output.
Changes
Replaced the single strict parse with a Zod-validated retry loop in
detectProjectsWithAgent:parseAgentReportnever throws and is exported for unit testing. Existing security clamping (coerceAgenticReport: path clamping, targetId validation,recommendeddedup) still runs on the validated output.Test plan
parseAgentReport(well-formed, code-fenced, no-JSON, malformed-JSON, and shape-invalid-but-parseable cases).pnpm buildand the detection vitest suites pass locally (82 tests).Created with PostHog from a Slack thread