Add Kestra integration guides to Getting Started#3626
Merged
qdequele merged 3 commits intoJul 12, 2026
Conversation
Add a collapsible Kestra group under Getting Started > Integrations with an overview page and five data-sync guides (PostgreSQL, MongoDB, Amazon S3, Kafka, REST API), adapted from the kestra-plugin-meilisearch article series. Each guide covers the backfill-then-incremental-sync pattern using the Kestra Meilisearch plugin. Adapted to the docs style: added frontmatter, removed em dashes, converted blockquote callouts to Note/Warning components, and turned cross-references into internal links.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Contributor
|
Warning Review limit reached
Next review available in: 13 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughAdds Kestra navigation and an overview page, plus guides for syncing PostgreSQL, MongoDB, Amazon S3, Kafka, and REST API data into Meilisearch. ChangesKestra integration documentation
Estimated code review effort: 3 (Moderate) | ~25 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 11
🧹 Nitpick comments (3)
getting_started/integrations/kestra/mongodb.mdx (1)
149-174: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winClarify that the delete tasks belong in the incremental sync flow.
The
extract_deletesandapply_deletestasks are shown as a separate code block, but their 2-space indentation indicates they are additional tasks in themongodb_incremental_syncflow from Step 2. A brief introductory sentence would help readers understand they should combine these tasks with the upsert tasks.✏️ Proposed clarification
`DocumentAdd` only ever adds or replaces; it never removes. A book deleted in MongoDB would otherwise haunt your search results forever. The fix is soft deletes: your application sets `deletedAt` to a timestamp instead of removing the document, and the sync flow removes recently-deleted ids from Meilisearch: +Add these tasks to the `mongodb_incremental_sync` flow so that each run handles both upserts and deletes: + ```yaml - id: extract_deletes🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@getting_started/integrations/kestra/mongodb.mdx` around lines 149 - 174, Add a brief introductory sentence before the extract_deletes/apply_deletes YAML block clarifying that these tasks should be added to the existing mongodb_incremental_sync flow from Step 2 alongside the upsert tasks. Keep the YAML structure and task definitions unchanged.getting_started/integrations/kestra/kafka.mdx (2)
9-9: 📐 Maintainability & Code Quality | 🔵 TrivialAvoid first person pronouns in general documentation.
The coding guidelines say to avoid first person pronouns in general documentation. "We build up" (line 9), "Our events" (line 48), and "We write it" (line 146) use first person plural. The surrounding guides (overview, PostgreSQL) don't use first person, so these lines are inconsistent with the series tone.
♻️ Suggested rewording
- We build up in two stages: first a batch consume-and-index flow to understand the moving parts, then a **real-time** trigger that indexes each message as it arrives, produced-to-searchable in seconds. Because a Kafka consumer group tracks its own offsets, this pattern is incremental by construction: there's no "first load versus incremental" split to manage, only the stream. + This guide builds up in two stages: first a batch consume-and-index flow to understand the moving parts, then a **real-time** trigger that indexes each message as it arrives, produced-to-searchable in seconds. Because a Kafka consumer group tracks its own offsets, this pattern is incremental by construction: there's no "first load versus incremental" split to manage, only the stream.- Our events are JSON messages describing products, keyed by product id: + The events are JSON messages describing products, keyed by product id:- Here the message payload is available directly as `{{ trigger.value }}`. We write it to internal storage as an ION document and index it. That's the entire live pipeline: produce a message to `live-products`, and it shows up in search seconds later. + Here the message payload is available directly as `{{ trigger.value }}`. The flow writes it to internal storage as an ION document and indexes it. That's the entire live pipeline: produce a message to `live-products`, and it shows up in search seconds later.As per coding guidelines: "Avoid first person pronouns ('I', 'me', 'my', 'mine') in general documentation (acceptable in FAQs or similar)" and "Match the existing tone and structure of the docs."
Also applies to: 48-48, 146-146
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@getting_started/integrations/kestra/kafka.mdx` at line 9, Rewrite the first-person phrasing in the Kafka guide at the opening flow description, the “Our events” text, and the “We write it” text to use neutral, instructional wording. Preserve the existing meaning, tone, and surrounding documentation structure while removing first-person pronouns.Source: Coding guidelines
17-17: 📐 Maintainability & Code Quality | 🔵 TrivialReduce semicolon usage for clarity.
The coding guidelines say to use semicolons sparingly and prefer splitting into two sentences. This file has four semicolons across lines 17, 107, 156, and 164. Consider rewording them into separate sentences.
♻️ Suggested rewording
- Meilisearch is managed; for a local broker, Redpanda is a lightweight, Kafka-API-compatible option: + Meilisearch is managed. For a local broker, Redpanda is a lightweight, Kafka-API-compatible option:- (In this demo the `produce` task stands in for your real upstream; in production you'd delete it.) + (In this demo the `produce` task stands in for your real upstream. In production you'd delete it.)- In production your services (or a Debezium connector) produce to the topic; Kestra only consumes. + In production your services (or a Debezium connector) produce to the topic. Kestra only consumes.- The `RealtimeTrigger` gives you second-scale freshness with offset tracking handled for you; add-or-replace semantics make the stream an idempotent upsert feed; and paired with Debezium, the same flow is a full CDC sink. + The `RealtimeTrigger` gives you second-scale freshness with offset tracking handled for you. Add-or-replace semantics make the stream an idempotent upsert feed. Paired with Debezium, the same flow is a full CDC sink.As per coding guidelines: "Use semicolons sparingly; prefer splitting into two sentences or rewording for clarity."
Also applies to: 107-107, 156-156, 164-164
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@getting_started/integrations/kestra/kafka.mdx` at line 17, In the Kestra Kafka integration documentation, remove the semicolons on the referenced lines, including the sentence beginning “A running Kestra...” and the occurrences at lines 107, 156, and 164. Reword each case as separate sentences or with clearer conjunctions while preserving the original meaning.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@getting_started/integrations/kestra/amazon_s3.mdx`:
- Line 9: Revise the opening text in the Amazon S3 integration documentation to
remove first-person phrasing such as “We cover” and “We’ll index,” and shorten
the affected sentences to fewer than 40 words while preserving the one-shot load
and event-driven indexing meaning.
- Line 39: Update the prose around the credential and S3 configuration guidance
to remove semicolons, splitting clauses into separate sentences or rewording
them while preserving the existing instructions and technical details.
- Line 46: Update the three-step list in the integration description to add
“and” before “index it,” while retaining the Oxford comma after “format.”
In `@getting_started/integrations/kestra/kafka.mdx`:
- Around line 150-152: Expand the “Handling deletes” section with a
Kafka-specific YAML example showing per-message branching on trigger.value:
route tombstones or events where op equals delete to
io.kestra.plugin.core.http.Request using documents/delete-batch, and route other
messages to DocumentAdd. Keep the example scoped to the real-time Kafka flow
rather than referring readers only to the PostgreSQL pattern.
In `@getting_started/integrations/kestra/mongodb.mdx`:
- Line 97: Update the documentation sentence beginning “Instead, sync only what
changed” to remove the first-person pronoun “we,” using an impersonal phrasing
while preserving the existing guidance about storing and updating `updatedAt`.
- Line 43: Update the introductory sentence in the MongoDB integration
documentation to remove the first-person pronoun “We’ll,” using an impersonal or
direct phrasing while preserving the existing meaning and guidance about syncing
the books collection.
In `@getting_started/integrations/kestra/postgresql.mdx`:
- Line 88: Update the “Step 2:” heading in the documentation so its first word
after the colon is capitalized, changing “incremental sync” to “Incremental
sync” while preserving the rest of the heading.
- Line 111: Revise the sentence near the deleted_at column to remove the
first-person pronoun “we,” while preserving the explanation that deletions use
soft deletes and the reference to the following rationale.
- Line 55: Capitalize the first word after the colon in the “Step 1:” heading,
changing “the first load (backfill)” to “The first load (backfill)” while
preserving the rest of the heading.
- Around line 7-9: Revise the introductory copy in the PostgreSQL integration
guide to remove first-person pronouns, including “my” and “we.” Preserve the
rhetorical question and the existing explanation of one-shot backfill and
scheduled incremental synchronization while using neutral, reader-focused
wording.
In `@getting_started/integrations/kestra/rest_api.mdx`:
- Around line 117-135: Update the date filter expression in the http_download
task’s uri to use a double-quoted Pebble format string, preserving the UTC ISO
timestamp output with literal T and Z characters. Keep the existing trigger.date
and timeZone='UTC' arguments unchanged.
---
Nitpick comments:
In `@getting_started/integrations/kestra/kafka.mdx`:
- Line 9: Rewrite the first-person phrasing in the Kafka guide at the opening
flow description, the “Our events” text, and the “We write it” text to use
neutral, instructional wording. Preserve the existing meaning, tone, and
surrounding documentation structure while removing first-person pronouns.
- Line 17: In the Kestra Kafka integration documentation, remove the semicolons
on the referenced lines, including the sentence beginning “A running Kestra...”
and the occurrences at lines 107, 156, and 164. Reword each case as separate
sentences or with clearer conjunctions while preserving the original meaning.
In `@getting_started/integrations/kestra/mongodb.mdx`:
- Around line 149-174: Add a brief introductory sentence before the
extract_deletes/apply_deletes YAML block clarifying that these tasks should be
added to the existing mongodb_incremental_sync flow from Step 2 alongside the
upsert tasks. Keep the YAML structure and task definitions unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a5b2978e-d374-4edc-9046-730bb236cdf8
📒 Files selected for processing (7)
config/navigation.jsongetting_started/integrations/kestra/amazon_s3.mdxgetting_started/integrations/kestra/kafka.mdxgetting_started/integrations/kestra/mongodb.mdxgetting_started/integrations/kestra/overview.mdxgetting_started/integrations/kestra/postgresql.mdxgetting_started/integrations/kestra/rest_api.mdx
- Remove first-person pronouns in favor of the docs' second-person voice - Capitalize the first word after the colon in all 'Step N:' headings - Split semicolon-joined sentences in amazon_s3, kafka, and overview - Add Oxford comma to the S3 three-step list - Fix the REST API Pebble date format to the double-quoted form (matches mongodb) - Add a concrete Kafka delete-branching flow instead of only referencing PostgreSQL - Clarify that the MongoDB delete tasks belong in the incremental sync flow
Add four more data-source guides to the Kestra integration group: - Elasticsearch and OpenSearch: index migrations (Scroll backfill + parallel-run cutover) - RabbitMQ: real-time AMQP indexing with a concrete delete-branching flow - Shopify: backfill + native updatedAtMin incremental sync, webhooks for deletes Adapted to the docs style established in the earlier guides: frontmatter, no em dashes, second-person voice (no first-person pronouns), Oxford commas, split semicolons, capitalized 'Step N:' headings, Note/Warning callouts, and internal links. The Shopify guide keeps its upstream validation caveat as a Warning.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a Kestra integration section to Getting Started → Integrations, rendered as a collapsible group in the sidebar (a nested group inside the existing Integrations group, which Mintlify renders as an expand/collapse toggle).
The group contains an overview page plus five data-sync guides, adapted from the
kestra-plugin-meilisearcharticle series:CardGrouplinking the guidesupdated_at, soft deletes via delete-batchupdatedAtISO string, soft deletes via delete-batch?updated_sincefilter with a KV watermark, or scheduled full re-indexWhy: these guides give users an orchestration path for keeping Meilisearch in sync with a primary data source, which the Integrations section didn't previously cover.
Adaptations from the source articles to match docs style:
title/sidebarTitle/descriptionfrontmatter to every page<Note>/<Warning>componentsVerified locally with
mint dev: the toggle renders in the sidebar, all pages load, and the MDX compiles cleanly.Checklist
For internal Meilisearch team member only:
For external maintainers
Summary by CodeRabbit