Skip to content

Tolerate unescaped control characters in AI JSON output #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kevinmatspie wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Tolerate unescaped control characters in AI JSON output #3

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions src/reviewd/reviewer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -475,16 +475,26 @@ def extract_json(output: str) -> dict:
logger.error('No JSON block found in AI output. Last 500 chars:\n%s', tail)
raise ValueError('No JSON block found in AI output')
raw = matches[-1]
# strict=False permits literal control characters (newlines, tabs, CR) inside
# string values — a common LLM failure mode when emitting multi-line `issue`
# or `fix` fields without escaping.
try:
return json.loads(raw)
return json.loads(raw, strict=False)
except json.JSONDecodeError:
# Strip trailing commas before } or ] (common LLM JSON error) and retry
fixed = re.sub(r',\s*([}\]])', r'\1', raw)
try:
logger.warning('Fixed trailing commas in AI JSON output')
return json.loads(fixed)
return json.loads(fixed, strict=False)
except json.JSONDecodeError as e:
logger.error('Malformed JSON in AI output: %s\nRaw JSON:\n%s', e, raw[:1000])
# Dump the full AI output so the review can be salvaged manually
# rather than throwing away the entire (paid-for) call.
dump_path = Path(tempfile.gettempdir()) / f'reviewd-failed-{int(time.time())}.txt'
try:
dump_path.write_text(output)
logger.error('Malformed JSON in AI output: %s. Full output saved to %s', e, dump_path)
except OSError:
logger.error('Malformed JSON in AI output: %s\nRaw JSON:\n%s', e, raw[:1000])
raise ValueError(f'Malformed JSON in AI output: {e}') from e


Expand Down