fix(detection): stop strict JSON.parse from breaking project scoping - #1035
Draft
posthog[bot] wants to merge 2 commits into
Draft
fix(detection): stop strict JSON.parse from breaking project scoping#1035posthog[bot] wants to merge 2 commits into
posthog[bot] wants to merge 2 commits into
Conversation
Two detection paths handed raw, untrusted text to a strict JSON.parse.
extractJson sliced the detection agent's output from the first `{` to the
LAST `}`, so truncated single-line JSON ended inside the `projects` array
and threw a raw SyntaxError. That propagated out of
scopeInstallDirToProject, which then abandoned project scoping — a
headless monorepo run integrated against the un-narrowed install dir. It
now balances braces and brackets from the opening `{` and reports
truncated / malformed output as an AgentOutputParseError with a clear
message, which project-scope records as an outcome without also raising
an exception.
addNpmDeps strict-parsed every file named package.json the project walk
found, so a JSONC-style manifest lost all its npm signals (and generated
error-tracking noise). It now uses the jsonc-parser the repo already
depends on, tolerating comments and trailing commas.
Adds coverage for both: extractJson (fenced/prose/trailing-brace/
truncated/malformed), the truncated-output path through
scopeInstallDirToProject, and the malformed-package.json skip.
Generated-By: PostHog Code
Task-Id: 0b01b412-c260-469c-9a1f-84fd14e209f1
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two detection paths handed raw, untrusted text straight to a strict
JSON.parse, and both threwSyntaxErrors in production.The consequential one is
extractJsoninsrc/lib/detection/agentic.ts: it sliced the detection agent's output from the first{to the last}. When the agent's single-line JSON was cut off mid-projects, that last}belonged to a nested project object, so the slice ended inside the array and the parse blew up. The error propagated out ofscopeInstallDirToProject, which then gave up on narrowing the target — a headless monorepo run integrated against the un-narrowed install dir, i.e. the wrong directory.The other is
addNpmDepsinsrc/lib/warehouse-sources/detect.ts, which strict-parsed every file namedpackage.jsonthe project walk found. A JSONC-style manifest (comments, trailing comma) threw, the project silently lost all its npm signals, and we got error-tracking noise for an expected input.Changes
extractJsonnow finds the object by balancing braces and brackets from the opening{(string- and escape-aware) instead of reaching for the last}. An unbalanced tail is reported as truncation, and a failed parse of a balanced slice as malformed JSON — both via a newAgentOutputParseErrorwith a readable message rather than a rawSyntaxError. As a bonus this also handles trailing prose that happens to contain a}.scopeInstallDirToProjectrecords anAgentOutputParseErrorthrough the usualagentic detectionoutcome event but no longer routes it tocaptureException— unparseable agent output is expected input, not a wizard bug.addNpmDepsparses with thejsonc-parserthe repo already depends on (comments + trailing commas tolerated) and drops thecaptureExceptionfor this case.Note: a truncated scan still degrades to "continue with the install dir as-is"; salvaging complete projects out of a partial array is deliberately out of scope here. This change makes the failure legible, stops it from being an exception, and removes the last-
}overreach that caused it in recoverable cases.Test plan
pnpm build && pnpm test && pnpm fix— 118 files / 1683 tests pass. New coverage:src/lib/detection/__tests__/agentic.test.ts—extractJsonover bare / fenced / prose-wrapped output, trailing stray brace, truncated mid-projectsoutput, malformed JSON and no-object input.src/lib/detection/__tests__/project-scope.test.ts— truncated agent output fires theerroroutcome with the message and does not callcaptureException.src/lib/warehouse-sources/__tests__/detect.test.ts— a package.json with comments and a trailing comma still yields its npm signals; unparseable and non-object manifests are skipped without throwing.LLM context
Authored by PostHog Code from the linked inbox report.
Created with PostHog Desktop from this inbox report.