Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (1)
WalkthroughThis PR adds comprehensive support for a new ChangesSummary Output Format Implementation
Sequence Diagram(s)sequenceDiagram
participant ScrapeRunner
participant MarkdownConverter
participant summarizeMarkdown
participant SerializableOutput
participant Webhook
ScrapeRunner->>MarkdownConverter: when 'markdown' or 'summary' requested
MarkdownConverter-->>ScrapeRunner: markdown content
ScrapeRunner->>summarizeMarkdown: markdown + llmConfig
summarizeMarkdown-->>ScrapeRunner: summary text
ScrapeRunner->>SerializableOutput: store summary[0].content
ScrapeRunner->>Webhook: include summary in run_completed
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
server/src/utils/output-post-processor.ts (1)
64-90:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGenerate summaries before dropping source content.
At Line 76/159,
textis deleted before summary extraction, and at Line 64/147 markdown is only derived whenmarkdownis requested. For requests like['summary'],pageText/resultTextbecomes empty, so summary is silently skipped.Proposed fix
- if (!effectiveFormats.includes('html') && markdownConversionSucceeded) { - delete pageResult.html; - } - if (!effectiveFormats.includes('text')) delete pageResult.text; - if (!effectiveFormats.includes('links')) delete pageResult.links; - - if (effectiveFormats.includes('summary')) { - const pageText = (pageResult.markdown || pageResult.text || '').substring(0, 40000); + if (effectiveFormats.includes('summary')) { + let summarySource = pageResult.markdown || pageResult.text || ''; + if (!summarySource && pageResult.html) { + try { + summarySource = await parseMarkdown(pageResult.html, pageResult.metadata?.url); + } catch (e: any) { + logger.log('warn', `Failed to derive crawl markdown for summary: ${e.message}`); + } + } + const pageText = summarySource.substring(0, 40000); if (pageText.trim()) { try { const { summarizeMarkdown } = require('../utils/summarizer'); pageResult.summary = await summarizeMarkdown(pageText, llmConfig); } catch (e: any) { logger.log('warn', `Failed to generate crawl page summary: ${e.message}`); } } } + + if (!effectiveFormats.includes('html') && markdownConversionSucceeded) { + delete pageResult.html; + } + if (!effectiveFormats.includes('text')) delete pageResult.text; + if (!effectiveFormats.includes('links')) delete pageResult.links;Apply the same ordering/derivation change in the search block.
Also applies to: 147-173
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@server/src/utils/output-post-processor.ts` around lines 64 - 90, The summary is being generated after source fields are deleted and markdown is only produced when 'markdown' is requested, which causes summaries to be skipped for requests like ['summary']; update the logic in output-post-processor so that pageResult/text and markdown derivation (call parseMarkdown for pageResult.html when available) happen before any deletions and before computing pageResult.summary (use summarizeMarkdown on the assembled pageText from markdown || text), and then prune fields based on effectiveFormats; apply the same ordering/derivation change in the analogous search block as well so summarizeMarkdown always has source text to work with.server/src/task-runner.ts (1)
433-447:⚠️ Potential issue | 🟠 Major | ⚡ Quick winHonor per-run output format overrides during crawl/search post-processing.
This branch ignores
run.interpreterSettings?.formatsand always falls back torecording_meta.formats, so a run that explicitly requestedsummary(or any other override) can be post-processed with the wrong format set.server/src/api/record.tsalready uses the per-run override for the equivalent path.Suggested fix
- const outputFormats = (recording.recording_meta as any).formats as string[] | undefined; + const outputFormats = + (run.interpreterSettings?.formats as string[] | undefined) || + ((recording.recording_meta as any).formats as string[] | undefined);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@server/src/task-runner.ts` around lines 433 - 447, The crawl/search post-processing currently always uses recording.recording_meta.formats, ignoring any per-run override; update the call to processRobotOutputFormats (inside the robotType === 'crawl' || 'search' branch) to pass run.interpreterSettings?.formats (when present) instead of always using recording.recording_meta.formats — i.e., compute outputFormats = run.interpreterSettings?.formats ?? (recording.recording_meta as any).formats and pass that into processRobotOutputFormats so per-run format overrides are honored.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@server/src/task-runner.ts`:
- Around line 352-358: The webhook payload for the 'run_completed' event emits
mixed camelCase keys which breaks schema consistency—change the payload sent by
sendWebhook in task-runner.ts to use the snake_case schema used elsewhere:
replace runId→run_id, robotId→robot_id, robotName→robot_name,
finishedAt→finished_at, and include started_at (take from plainRun.startedAt or
plainRun.started_at) and metadata (take from plainRun.metadata or plainRun.meta
or recording.recording_meta.metadata) as present; keep serializableOutput fields
(markdown/html/links/summary) but attach them as markdown/html/links/summary in
the same snake_case wrapper and then call sendWebhook(plainRun.robotMetaId,
'run_completed', webhookPayload) with the normalized object.
In `@src/components/run/RunContent.tsx`:
- Around line 261-266: Normalize the summary payload into a single canonical
string/object before any rendering or copy/download logic: detect if
output.summary is an array (take output.summary[0].content if present), or if
it's already a string/object use it directly, then pass that normalized value
into setSummaryContent and all rendering/copy/download code paths currently
reading output.summary (including the crawl/search summary renderers); update
the summary extraction logic near setSummaryContent so all consumers use the
same normalized variable and apply the same change to the other summary-handling
locations referenced in the review (crawl/search renderers and copy/download
handlers).
---
Outside diff comments:
In `@server/src/task-runner.ts`:
- Around line 433-447: The crawl/search post-processing currently always uses
recording.recording_meta.formats, ignoring any per-run override; update the call
to processRobotOutputFormats (inside the robotType === 'crawl' || 'search'
branch) to pass run.interpreterSettings?.formats (when present) instead of
always using recording.recording_meta.formats — i.e., compute outputFormats =
run.interpreterSettings?.formats ?? (recording.recording_meta as any).formats
and pass that into processRobotOutputFormats so per-run format overrides are
honored.
In `@server/src/utils/output-post-processor.ts`:
- Around line 64-90: The summary is being generated after source fields are
deleted and markdown is only produced when 'markdown' is requested, which causes
summaries to be skipped for requests like ['summary']; update the logic in
output-post-processor so that pageResult/text and markdown derivation (call
parseMarkdown for pageResult.html when available) happen before any deletions
and before computing pageResult.summary (use summarizeMarkdown on the assembled
pageText from markdown || text), and then prune fields based on
effectiveFormats; apply the same ordering/derivation change in the analogous
search block as well so summarizeMarkdown always has source text to work with.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c565d26f-744c-40aa-bb52-cfdde0a74da8
📒 Files selected for processing (9)
server/src/api/record.tsserver/src/api/sdk.tsserver/src/constants/output-formats.tsserver/src/task-runner.tsserver/src/utils/output-post-processor.tsserver/src/utils/summarizer.tsserver/src/workflow-management/scheduler/index.tssrc/components/run/RunContent.tsxsrc/constants/outputFormats.ts
Comment on lines
352
to
+358
| try { | ||
| await sendWebhook(plainRun.robotMetaId, 'run_completed', { runId: data.runId, robotId: plainRun.robotMetaId, robotName: recording.recording_meta.name, status: 'success', finishedAt: new Date().toLocaleString() }); | ||
| const webhookPayload: any = { runId: data.runId, robotId: plainRun.robotMetaId, robotName: recording.recording_meta.name, status: 'success', finishedAt: new Date().toLocaleString() }; | ||
| if (serializableOutput.markdown) webhookPayload.markdown = serializableOutput.markdown[0]?.content || ''; | ||
| if (serializableOutput.html) webhookPayload.html = serializableOutput.html[0]?.content || ''; | ||
| if (serializableOutput.links) webhookPayload.links = serializableOutput.links; | ||
| if (serializableOutput.summary) webhookPayload.summary = serializableOutput.summary[0]?.content || ''; | ||
| await sendWebhook(plainRun.robotMetaId, 'run_completed', webhookPayload); |
There was a problem hiding this comment.
Normalize the run_completed webhook payload schema here.
sendWebhook forwards data verbatim, but this path emits camelCase fields (runId, robotId, finishedAt) while the scrape completion payloads in server/src/api/record.ts and server/src/workflow-management/scheduler/index.ts use snake_case plus started_at and metadata. That makes webhook consumers parse the same event differently depending on which execution path produced it.
Suggested direction
- const webhookPayload: any = { runId: data.runId, robotId: plainRun.robotMetaId, robotName: recording.recording_meta.name, status: 'success', finishedAt: new Date().toLocaleString() };
+ const webhookPayload: any = {
+ robot_id: plainRun.robotMetaId,
+ run_id: data.runId,
+ robot_name: recording.recording_meta.name,
+ status: 'success',
+ started_at: plainRun.startedAt,
+ finished_at: new Date().toLocaleString(),
+ metadata: {
+ browser_id: plainRun.browserId,
+ user_id: data.userId,
+ },
+ };📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| try { | |
| await sendWebhook(plainRun.robotMetaId, 'run_completed', { runId: data.runId, robotId: plainRun.robotMetaId, robotName: recording.recording_meta.name, status: 'success', finishedAt: new Date().toLocaleString() }); | |
| const webhookPayload: any = { runId: data.runId, robotId: plainRun.robotMetaId, robotName: recording.recording_meta.name, status: 'success', finishedAt: new Date().toLocaleString() }; | |
| if (serializableOutput.markdown) webhookPayload.markdown = serializableOutput.markdown[0]?.content || ''; | |
| if (serializableOutput.html) webhookPayload.html = serializableOutput.html[0]?.content || ''; | |
| if (serializableOutput.links) webhookPayload.links = serializableOutput.links; | |
| if (serializableOutput.summary) webhookPayload.summary = serializableOutput.summary[0]?.content || ''; | |
| await sendWebhook(plainRun.robotMetaId, 'run_completed', webhookPayload); | |
| try { | |
| const webhookPayload: any = { | |
| robot_id: plainRun.robotMetaId, | |
| run_id: data.runId, | |
| robot_name: recording.recording_meta.name, | |
| status: 'success', | |
| started_at: plainRun.startedAt, | |
| finished_at: new Date().toLocaleString(), | |
| metadata: { | |
| browser_id: plainRun.browserId, | |
| user_id: data.userId, | |
| }, | |
| }; | |
| if (serializableOutput.markdown) webhookPayload.markdown = serializableOutput.markdown[0]?.content || ''; | |
| if (serializableOutput.html) webhookPayload.html = serializableOutput.html[0]?.content || ''; | |
| if (serializableOutput.links) webhookPayload.links = serializableOutput.links; | |
| if (serializableOutput.summary) webhookPayload.summary = serializableOutput.summary[0]?.content || ''; | |
| await sendWebhook(plainRun.robotMetaId, 'run_completed', webhookPayload); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@server/src/task-runner.ts` around lines 352 - 358, The webhook payload for
the 'run_completed' event emits mixed camelCase keys which breaks schema
consistency—change the payload sent by sendWebhook in task-runner.ts to use the
snake_case schema used elsewhere: replace runId→run_id, robotId→robot_id,
robotName→robot_name, finishedAt→finished_at, and include started_at (take from
plainRun.startedAt or plainRun.started_at) and metadata (take from
plainRun.metadata or plainRun.meta or recording.recording_meta.metadata) as
present; keep serializableOutput fields (markdown/html/links/summary) but attach
them as markdown/html/links/summary in the same snake_case wrapper and then call
sendWebhook(plainRun.robotMetaId, 'run_completed', webhookPayload) with the
normalized object.
Comment on lines
+261
to
+266
| if (output?.summary && Array.isArray(output.summary)) { | ||
| const summaryData = output.summary[0]; | ||
| if (summaryData?.content) { | ||
| setSummaryContent(summaryData.content); | ||
| } | ||
| } |
There was a problem hiding this comment.
Normalize summary payload shape consistently before rendering.
The new summary flow mixes shapes: top-level parsing assumes summary[0].content, but crawl/search rendering assumes summary is a string/object. If crawl/search summaries come as array format, the UI renders/copies/downloads empty content.
💡 Suggested fix
+ const getSummaryText = (raw: any): string => {
+ if (!raw) return '';
+ if (typeof raw === 'string') return raw;
+ if (Array.isArray(raw)) {
+ const first = raw[0];
+ if (typeof first === 'string') return first;
+ if (first && typeof first === 'object') return first.content || '';
+ return '';
+ }
+ if (typeof raw === 'object') return raw.content || '';
+ return '';
+ };
...
- if (output?.summary && Array.isArray(output.summary)) {
- const summaryData = output.summary[0];
- if (summaryData?.content) {
- setSummaryContent(summaryData.content);
- }
- }
+ const normalizedSummary = getSummaryText(output?.summary);
+ if (normalizedSummary) setSummaryContent(normalizedSummary);
...
- {typeof crawlData[0][currentCrawlIndex].summary === 'string'
- ? crawlData[0][currentCrawlIndex].summary
- : crawlData[0][currentCrawlIndex].summary?.content || ''}
+ {getSummaryText(crawlData[0][currentCrawlIndex].summary)}
...
- <CopyButton content={typeof crawlData[0][currentCrawlIndex].summary === 'string' ? crawlData[0][currentCrawlIndex].summary : crawlData[0][currentCrawlIndex].summary?.content || ''} darkMode={darkMode} />
+ <CopyButton content={getSummaryText(crawlData[0][currentCrawlIndex].summary)} darkMode={darkMode} />
...
- const content = typeof crawlData[0][currentCrawlIndex].summary === 'string' ? crawlData[0][currentCrawlIndex].summary : crawlData[0][currentCrawlIndex].summary?.content || '';
+ const content = getSummaryText(crawlData[0][currentCrawlIndex].summary);
...
- {typeof searchData[currentSearchIndex].summary === 'string'
- ? searchData[currentSearchIndex].summary
- : searchData[currentSearchIndex].summary?.content || ''}
+ {getSummaryText(searchData[currentSearchIndex].summary)}
...
- content={typeof searchData[currentSearchIndex].summary === 'string' ? searchData[currentSearchIndex].summary : searchData[currentSearchIndex].summary?.content || ''}
+ content={getSummaryText(searchData[currentSearchIndex].summary)}
...
- const content = typeof searchData[currentSearchIndex].summary === 'string' ? searchData[currentSearchIndex].summary : searchData[currentSearchIndex].summary?.content || '';
+ const content = getSummaryText(searchData[currentSearchIndex].summary);Also applies to: 2149-2168, 2576-2597
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/components/run/RunContent.tsx` around lines 261 - 266, Normalize the
summary payload into a single canonical string/object before any rendering or
copy/download logic: detect if output.summary is an array (take
output.summary[0].content if present), or if it's already a string/object use it
directly, then pass that normalized value into setSummaryContent and all
rendering/copy/download code paths currently reading output.summary (including
the crawl/search summary renderers); update the summary extraction logic near
setSummaryContent so all consumers use the same normalized variable and apply
the same change to the other summary-handling locations referenced in the review
(crawl/search renderers and copy/download handlers).
RohitR311
changed the title
amhsirak
approved these changes
Jun 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does?
closes #1090
Adds the summary format for scrape, crawl and search + scrape robot types.
2026-06-01.20-38-00.mp4
Summary by CodeRabbit