out_es: Add documentation for index record accessor

[Wiston999]

Summary by CodeRabbit

• Documentation
  • Added documentation for the new target_index parameter for Elasticsearch output, enabling dynamic selection of destination indexes from record accessor fields with fallback to the default index setting when fields are absent.

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[eschabell]

@esmerel can you review this PR, still waiting on code PR fluent/fluent-bit#7716 where I nudged the code owner and a review?

[esmerel]

I accidentally overwrote while fixing the conflict and then fixed the changes - @eschabell, there's an example at the end that needs a newer code sample.

[coderabbitai]

Warning

Review limit reached

@eschabell, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 7 minutes and 19 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8b6835bd-3637-44c7-9afd-d80bbe5f6a84

📥 Commits

Reviewing files that changed from the base of the PR and between 002ac1d and 7c3d080.

📒 Files selected for processing (1)

• pipeline/outputs/elasticsearch.md

📝 Walkthrough

Walkthrough

This PR documents a new target_index configuration parameter for Elasticsearch output in Fluent Bit. The parameter enables dynamic index selection by reading values from record fields, with a fallback to the static index setting when the specified field is absent. The documentation includes a parameter table entry and a detailed usage section with example configuration.

Changes

Elasticsearch target_index Parameter Documentation

Layer / File(s)	Summary
target_index configuration and usage documentation pipeline/outputs/elasticsearch.md	Adds target_index parameter entry to the configuration table and introduces a new "Target_index" documentation section explaining how to render destination indices from record accessor fields with fallback to the index setting, including example output configuration.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Suggested labels

5.0

Poem

🐰 A parameter springs to life so bright,
Where indices dance left and right,
From record fields they take their name,
With fallback paths to stay the same,
Elasticsearch flows with dynamic grace!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the main change: adding documentation for Elasticsearch's index record accessor feature.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[eschabell]

@Wiston999 fixed conflicts.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

pipeline/outputs/elasticsearch.md (1)

443-451: ⚡ Quick win

Consider adding YAML format example for consistency.

Throughout this document, configuration examples are shown in both YAML and classic .conf formats using tabs (see lines 112-149, 177-212, 222-255, etc.). The Target_index section only shows the classic .conf format. As per coding guidelines, providing both formats helps users working with either configuration style.

📋 Suggested dual-format example

+{% tabs %}
+{% tab title="fluent-bit.yaml" %}
+
+```yaml
+pipeline:
+
+  outputs:
+    - name: es
+      match: '*'
+      # ...
+      index: fallback
+      target_index: fluent-$destination_index
+      # ...
+```
+
+{% endtab %}
+{% tab title="fluent-bit.conf" %}
+
 ```text
 [OUTPUT]
     Name es
     Match *
     # ...
     Index fallback
     Target_index fluent-$destination_index
     # ...
 ```
+
+{% endtab %}
+{% endtabs %}

Based on learnings, YAML examples use lowercase keys (e.g., target_index) while classic .conf examples use Title_Case (e.g., Target_index).

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pipeline/outputs/elasticsearch.md` around lines 443 - 451, The examples for
the Elasticsearch output only include the fluent-bit .conf style (Target_index)
and should also include the YAML-style example for consistency; add a YAML block
under the pipeline.outputs section showing the equivalent keys (e.g., name: es,
match: '*', index: fallback, target_index: fluent-$destination_index) alongside
the existing [OUTPUT] block so readers see both formats and the lowercase YAML
key mapping to Title_Case (.conf) (reference symbols: pipeline.outputs, name es,
Target_index, target_index).

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pipeline/outputs/elasticsearch.md`:
- Line 453: Replace the incorrect field name `destination__index` with the
correct `destination_index` in the documentation sentence so it matches the
other occurrences (e.g., the single-underscore `destination_index` used
elsewhere); update the text that currently reads "For records that do not have
the field `destination__index`, the value of `Index` (`fallback`) will be used."
to use `destination_index` instead.

---

Nitpick comments:
In `@pipeline/outputs/elasticsearch.md`:
- Around line 443-451: The examples for the Elasticsearch output only include
the fluent-bit .conf style (Target_index) and should also include the YAML-style
example for consistency; add a YAML block under the pipeline.outputs section
showing the equivalent keys (e.g., name: es, match: '*', index: fallback,
target_index: fluent-$destination_index) alongside the existing [OUTPUT] block
so readers see both formats and the lowercase YAML key mapping to Title_Case
(.conf) (reference symbols: pipeline.outputs, name es, Target_index,
target_index).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4cb5c45e-09e6-478e-b18e-1cf83ddbafb8

📥 Commits

Reviewing files that changed from the base of the PR and between 5231934 and 002ac1d.

📒 Files selected for processing (1)

• pipeline/outputs/elasticsearch.md

[eschabell]

@Wiston999 cleaned up this older contribution to get past all the vale checks.

[eschabell]

@Wiston999 waiting on code PR merging.