fix: ai mode json parsing

[RohitR311]

What this PR does?

Fixes:

When creating a robot, the AI was ignoring which fields the user asked for and returning everything instead. This was caused by the AI's response containing formatting that the system couldn't read correctly.

Summary by CodeRabbit

• Bug Fixes

  • Improved JSON parsing robustness by sanitizing unescaped control characters in LLM responses to reduce parse failures.

• New Features / Enhancements

  • Robust multi-candidate decision flow with fallback heuristics to increase successful workflow generation from LLM output.
  • Enhanced page/group analysis with richer structural, accessibility and text-derived signals (semantic parent, ARIA role, link ratios, headings, child/tag metrics) for better selection and ranking.

[coderabbitai]

Caution

Review failed

Failed to post review comments

Walkthrough

Adds richer DOM group signals in the page analyzer and refactors vision-driven workflow generation: LLM JSON responses are sanitized before parse, vision prompts are unified, ranked group candidates are parsed and attempted iteratively, and a multi-signal heuristic fallback selects groups when candidates fail.

Changes

Vision-driven group selection and page analysis

Layer / File(s)	Summary
Client page group metadata enrichment server/src/sdk/browserSide/pageAnalyzer.js	Adds semantic ancestor detection, isNavOrFooter, childTagCount, attributeCount, normalized sampleTexts, ariaRole, avgTextLength, linkTextRatio, headingCount, and stores the extended uniqueGroups payload. (ranges: range_7da6f816565a, range_2e8bc4c5e473, range_63e05623c4fb, range_31e04cec0b87, range_286d52ef3cfb, range_f985736ad1fd)
Sanitizer and generation entry updates server/src/sdk/workflowEnricher.ts	Adds sanitizeJsonString helper and updates generation entry points to invoke the multi-candidate execution path rather than building directly from a single LLM decision. (ranges: range_81aef7dea5b4, range_6fc70b7f1cba, range_f3001a50d569)
Vision prompt, response parsing, and candidate conversion server/src/sdk/workflowEnricher.ts	Refactors vision request construction (keywords, exclusions), updates structured message formats for Anthropic/OpenAI, sanitizes and parses extracted JSON, and converts parsed decisions into ranked group candidates and workflow metadata. (ranges: range_c64bdb43c174, range_bac1a0a83459, range_3f1b8d2b98c2, range_3a0da9869f7e, range_660e6949ae6d, range_909e35cef008, range_0bcb3d14b735)
Try-candidates loop and heuristic fallback server/src/sdk/workflowEnricher.ts	Implements tryGroupCandidates to iterate candidate groups, avoid repeats, and fall back to fallbackHeuristicDecision which now scores groups using roles, semantic parents, text/link metrics, headings, and child/tag counts; return shape includes selectedGroupIndex. (ranges: range_c65e1931686a, range_8b242c8a1e56, range_98ca74d329d7)
Intent parsing and search-generation call sites server/src/sdk/workflowEnricher.ts	Applies JSON sanitization to intent-based field filtering parsing and routes generateWorkflowFromPromptWithSearch through the new candidate-try flow. (ranges: range_652ee4a9e219, range_059633e9b381)

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested labels: Type: Bug, Scope: Ext

Suggested reviewers:

• amhsirak

🐰 I nibble through tangled newline threads,
I mend the JSON where the LLM treads.
I count the headings, roles, and links,
Then hop through candidates—no panic, just thinks.
Workflow assembled, tidy as beds.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title 'fix: ai mode json parsing' is partially related to the changeset. It accurately describes one aspect of the changes (JSON parsing sanitization in the WorkflowEnricher), but the PR objectives indicate the main intent is to improve AI robot element group selection accuracy and field filtering behavior, which is broader than just JSON parsing. The title captures a technical detail but not the primary goal.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch ai-mode

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}