Skip to content

Harden fenced JSON parsing#2557

Closed
koriyoshi2041 wants to merge 1 commit into
vectorize-io:mainfrom
koriyoshi2041:rios/parse-validated-code-fences
Closed

Harden fenced JSON parsing#2557
koriyoshi2041 wants to merge 1 commit into
vectorize-io:mainfrom
koriyoshi2041:rios/parse-validated-code-fences

Conversation

@koriyoshi2041

Copy link
Copy Markdown
Contributor

Summary

  • only strip fenced content when the candidate parses as JSON
  • fall back to the outer JSON object/array when a fence is partial or nested backticks truncate the first candidate
  • keep non-JSON fenced text on the raw-content path so it retries with the original response

Fixes #2542.

Tests

  • uv run --directory hindsight-api-slim ruff format hindsight_api/engine/providers/openai_compatible_llm.py tests/test_strip_code_fences.py
  • uv run --directory hindsight-api-slim ruff check hindsight_api/engine/providers/openai_compatible_llm.py tests/test_strip_code_fences.py
  • uv run --directory hindsight-api-slim pytest tests/test_strip_code_fences.py -q

Risk

Low. This only changes the fenced-response cleanup helper and adds focused coverage for parse validation, non-JSON fences, partial fences, and backticks inside JSON strings.

@benfrank241

Copy link
Copy Markdown
Member

Superseded by #2610, which combines the line-based fence stripping from #2563 (already merged) with the parse-validated outer-JSON-span fallback from this PR. Thanks @koriyoshi2041 — the parse-validation idea here is the more robust part and it's carried forward in #2610 (with your partial-fence / non-JSON-fence cases covered in tests).

@benfrank241 benfrank241 closed this Jul 7, 2026
benfrank241 added a commit that referenced this pull request Jul 7, 2026
…ON (#2610)

Grafts the parse-validated fallback from #2557 onto the line-based fence
stripper merged in #2563: after stripping, if the candidate is not valid
JSON (partial/absent fence, prose-wrapped or truncated output), fall back
to the outermost parseable {..}/[..] span. Never returns a worse candidate
than the raw content.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants