Skip to content

in_mem_metrics: initial documentation.#1170

Open
pwhelan wants to merge 10 commits into
masterfrom
pwhelan-doc-in_mem_metrics
Open

in_mem_metrics: initial documentation.#1170
pwhelan wants to merge 10 commits into
masterfrom
pwhelan-doc-in_mem_metrics

Conversation

@pwhelan
Copy link
Copy Markdown
Contributor

@pwhelan pwhelan commented Jul 26, 2023

This adds documentation for the new in_mem_metrics plugin: fluent/fluent-bit#7615.

Summary by CodeRabbit

  • Documentation
    • Added documentation for the memory metrics plugin covering configuration parameters (process filters, interval settings, /proc path), usage examples for command-line and configuration files with Prometheus remote write support, and complete metrics reference with supported label dimensions.

Review Change Stack

This adds documentation for the new `in_mem_metrics` plugin: fluent/fluent-bit#7615.

Signed-off-by: Phillip Whelan <phil@calyptia.com>
@pwhelan pwhelan self-assigned this Jul 26, 2023
@pwhelan
Copy link
Copy Markdown
Contributor Author

pwhelan commented Jul 26, 2023

@patrick-stephens should I add a specific example for using docker to log metrics for kubernetes nodes?

@patrick-stephens
Copy link
Copy Markdown
Contributor

@patrick-stephens should I add a specific example for using docker to log metrics for kubernetes nodes?

I don't really understand so probably :)

Although why would you use it over node exporter, kubelet metrics, etc. is probably the main question I'd have? Be good to have an example of what it provides in addition.

@pwhelan
Copy link
Copy Markdown
Contributor Author

pwhelan commented Jul 26, 2023

Although why would you use it over node exporter, kubelet metrics, etc. is probably the main question I'd have? Be good to have an example of what it provides in addition.

The main reason to use it over those options is granularity. This plugin grabs the memory usage of each individual process and various different types, ie: shared, etc... If you want to be able to correctly diagnose which process is actually leaking this would be what you need to be able to discern that correctly.

@lecaros lecaros requested review from a team as code owners March 25, 2025 14:34
@eschabell eschabell requested review from esmerel and removed request for ehienabs October 22, 2025 19:00
@eschabell eschabell added the waiting-on-review Waiting on a review from mainteners label Oct 22, 2025
@eschabell
Copy link
Copy Markdown
Collaborator

@esmerel would you give this a docs review please? Still waiting on code merge PR fluent/fluent-bit#7615, but I've given that a nudge.

@eschabell eschabell self-assigned this Oct 22, 2025
Copy link
Copy Markdown
Contributor

@esmerel esmerel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally stylistic changes. This looks fairly ok otherwise.

Comment thread pipeline/inputs/mem-metrics.md Outdated
Comment thread pipeline/inputs/mem-metrics.md Outdated
Comment thread pipeline/inputs/mem-metrics.md Outdated
Comment thread pipeline/inputs/mem-metrics.md Outdated
Comment thread pipeline/inputs/mem-metrics.md Outdated
Comment thread pipeline/inputs/mem-metrics.md Outdated
Comment thread pipeline/inputs/mem-metrics.md Outdated
Comment thread pipeline/inputs/mem-metrics.md Outdated
@eschabell eschabell added the waiting-for-user Waiting for user/contributors feedback or requested changes label Nov 4, 2025
@eschabell
Copy link
Copy Markdown
Collaborator

@pwhelan we have some review suggestions for your attention.

pwhelan and others added 8 commits November 4, 2025 15:31
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
Co-authored-by: Lynette  Miles <6818907+esmerel@users.noreply.github.com>
Signed-off-by: Phillip Whelan <pwhelan@exis.cl>
@eschabell eschabell added lint-vale-issues and removed waiting-for-user Waiting for user/contributors feedback or requested changes labels Dec 3, 2025
eschabell added a commit that referenced this pull request May 27, 2026
  - Fix 'Fluent Bit' casing (was 'fluent-bit') in body text
  - Sort configuration parameters table alphabetically
  - Fix classic config block indentation from 4 spaces to 2 spaces

  Applies to #1170

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell eschabell self-requested a review as a code owner May 27, 2026 18:31
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 27, 2026

Warning

Review limit reached

@eschabell, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 10 minutes and 27 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9301be49-fb16-4c8b-8174-3cca58146dfb

📥 Commits

Reviewing files that changed from the base of the PR and between 83a12dd and 68fddc1.

📒 Files selected for processing (1)
  • pipeline/inputs/mem-metrics.md
📝 Walkthrough

Walkthrough

This PR adds a new documentation file pipeline/inputs/mem-metrics.md (114 lines) that describes Fluent Bit's Linux-only mem_metrics input plugin. It documents configuration parameters, provides command-line and configuration-file examples (including Prometheus remote-write integration), and enumerates the available memory metrics derived from /proc/*/smaps_rollup with their supported label dimensions.

Changes

mem_metrics Plugin Documentation

Layer / File(s) Summary
Plugin overview and configuration parameters
pipeline/inputs/mem-metrics.md (lines 1–21)
Introduces the mem_metrics plugin as a Linux-only input that generates memory metrics using smaps_rollup, documents configuration parameters (filter_pid, interval, proc_path), and explains the special filter_pid values (0 for all processes, self for the Fluent Bit process).
Quick start examples
pipeline/inputs/mem-metrics.md (lines 22–67)
Provides a "Get started" section with a direct command-line invocation example and a minimal input/output configuration showing basic usage with metrics emitted to stdout.
Advanced configuration and metrics reference
pipeline/inputs/mem-metrics.md (lines 68–113)
Demonstrates filtering by command (Filter_cmd) and forwarding metrics to a Prometheus remote-write endpoint; enumerates all exposed node_smaps_rollup_* gauge metrics and their supported type label variants (e.g., anon, file, swap, swapcache).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

  • fluent/fluent-bit-docs#2281: Adds or updates documentation for the same mem(-)ory-metrics input plugin with overlapping metric/parameter tables and PID examples.
  • fluent/fluent-bit-docs#2255: Adds new documentation under pipeline/inputs/ for Fluent Bit metrics-related input plugins with CLI and configuration examples.

Suggested labels

4.2.1

Suggested reviewers

  • esmerel
  • patrick-stephens

Poem

🐰 A memory feast, so neatly arranged,
Linux metrics flow, through Prometheus arranged,
From smaps_rollup's depths, the gauges take flight,
With filter_pid magic and examples so bright,
The docs hop forward with clarity and might! 🌟

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'in_mem_metrics: initial documentation' clearly describes the main change: adding initial documentation for the in_mem_metrics plugin, which matches the raw summary perfectly.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pwhelan-doc-in_mem_metrics

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@eschabell
Copy link
Copy Markdown
Collaborator

@pwhelan fixed issues and applied a lot of formatting standards (this was a bit of an older one).

@eschabell eschabell removed waiting-on-review Waiting on a review from mainteners lint-vale-issues labels May 27, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
pipeline/inputs/mem-metrics.md (1)

89-112: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix list indentation throughout metrics section.

The metrics list has inconsistent indentation causing multiple markdownlint violations. Top-level items should have no leading spaces, and sub-items should use 2-space indentation. Additionally, lines 96 and 99 have an extra leading space before the dash.

📝 Proposed fix for consistent indentation
-  - node_smaps_rollup_rss
-  - node_smaps_rollup_pss
-    - type=clean
-    - type=dirty
-    - type=anon
-    - type=file
-    - type=shmem
-  -  node_smaps_rollup_shared
-    - type=clean
-    - type=dirty
-  -  node_smaps_rollup_private
-    - type=dirty
-    - type=clean
-  - node_smaps_rollup_referenced
-  - node_smaps_rollup_anonymous
-  - node_smaps_rollup_lazy_free
-  - node_smaps_rollup_anon_huge_pages
-  - node_smaps_rollup_shmem_pmd_mapped
-  - node_smaps_rollup_file_pmd_mapped
-  - node_smaps_rollup_shared_hugetlb
-  - node_smaps_rollup_private_hugetlb
-  - node_smaps_rollup_swap
-  - node_smaps_rollup_swap_pss
-  - node_smaps_rollup_locked
+- node_smaps_rollup_rss
+- node_smaps_rollup_pss
+  - type=clean
+  - type=dirty
+  - type=anon
+  - type=file
+  - type=shmem
+- node_smaps_rollup_shared
+  - type=clean
+  - type=dirty
+- node_smaps_rollup_private
+  - type=dirty
+  - type=clean
+- node_smaps_rollup_referenced
+- node_smaps_rollup_anonymous
+- node_smaps_rollup_lazy_free
+- node_smaps_rollup_anon_huge_pages
+- node_smaps_rollup_shmem_pmd_mapped
+- node_smaps_rollup_file_pmd_mapped
+- node_smaps_rollup_shared_hugetlb
+- node_smaps_rollup_private_hugetlb
+- node_smaps_rollup_swap
+- node_smaps_rollup_swap_pss
+- node_smaps_rollup_locked
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pipeline/inputs/mem-metrics.md` around lines 89 - 112, The metrics list under
the mem-metrics section has inconsistent indentation: normalize all top-level
metric items (e.g., node_smaps_rollup_rss, node_smaps_rollup_pss,
node_smaps_rollup_shared, node_smaps_rollup_private,
node_smaps_rollup_referenced, etc.) so they start with no leading spaces before
the dash, and indent their sub-items (the type=... lines) with exactly two
spaces before the dash; also remove the stray extra space before the dash that
appears on the lines for node_smaps_rollup_pss and node_smaps_rollup_private to
ensure consistent 0-space top-level and 2-space nested-item indentation
throughout.
🧹 Nitpick comments (2)
pipeline/inputs/mem-metrics.md (2)

1-1: ⚡ Quick win

Remove trailing space from title.

The title has a trailing space that should be removed.

✨ Proposed fix
-# Memory metrics 
+# Memory metrics
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pipeline/inputs/mem-metrics.md` at line 1, Remove the trailing space from the
Markdown title "# Memory metrics " by editing the header line in the file so it
reads "# Memory metrics" (no trailing whitespace); ensure the change is applied
to the top-level title string to avoid any leftover trailing characters.

11-11: ⚡ Quick win

Hyphenate compound adjective.

"Comma delimited" should be hyphenated when used as a compound adjective modifying "list".

✍️ Proposed fix
-| `filter_pid`    | Filter by comma delimited list of `PIDs`.             | **inactive** |
+| `filter_pid`    | Filter by comma-delimited list of `PIDs`.             | **inactive** |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pipeline/inputs/mem-metrics.md` at line 11, In the table row for the
`filter_pid` option (the line containing "| `filter_pid`    | Filter by comma
delimited list of `PIDs`.             | **inactive** |"), change the phrase
"comma delimited list" to "comma-delimited list" so the compound adjective is
hyphenated correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pipeline/inputs/mem-metrics.md`:
- Line 7: Fix the table header typo by removing the stray asterisk: replace the
header string "**Key***" with "**Key**" (the header in the markdown table row
shown by the diff). Ensure the corrected table header maintains the same
pipe-delimited column formatting and spacing as the other headers.

---

Duplicate comments:
In `@pipeline/inputs/mem-metrics.md`:
- Around line 89-112: The metrics list under the mem-metrics section has
inconsistent indentation: normalize all top-level metric items (e.g.,
node_smaps_rollup_rss, node_smaps_rollup_pss, node_smaps_rollup_shared,
node_smaps_rollup_private, node_smaps_rollup_referenced, etc.) so they start
with no leading spaces before the dash, and indent their sub-items (the type=...
lines) with exactly two spaces before the dash; also remove the stray extra
space before the dash that appears on the lines for node_smaps_rollup_pss and
node_smaps_rollup_private to ensure consistent 0-space top-level and 2-space
nested-item indentation throughout.

---

Nitpick comments:
In `@pipeline/inputs/mem-metrics.md`:
- Line 1: Remove the trailing space from the Markdown title "# Memory metrics "
by editing the header line in the file so it reads "# Memory metrics" (no
trailing whitespace); ensure the change is applied to the top-level title string
to avoid any leftover trailing characters.
- Line 11: In the table row for the `filter_pid` option (the line containing "|
`filter_pid`    | Filter by comma delimited list of `PIDs`.             |
**inactive** |"), change the phrase "comma delimited list" to "comma-delimited
list" so the compound adjective is hyphenated correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f4ef7f79-3fd9-47b1-8047-a90b49f1fc03

📥 Commits

Reviewing files that changed from the base of the PR and between c576098 and 83a12dd.

📒 Files selected for processing (1)
  • pipeline/inputs/mem-metrics.md

Comment thread pipeline/inputs/mem-metrics.md Outdated
eschabell added a commit that referenced this pull request May 27, 2026
  - Fix 'Fluent Bit' casing (was 'fluent-bit') in body text
  - Sort configuration parameters table alphabetically
  - Fix classic config block indentation from 4 spaces to 2 spaces
  - Fixed header for config param table

  Applies to #1170

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell eschabell force-pushed the pwhelan-doc-in_mem_metrics branch from 83a12dd to 4dae327 Compare May 27, 2026 18:34
  - Fix 'Fluent Bit' casing (was 'fluent-bit') in body text
  - Sort configuration parameters table alphabetically
  - Fix classic config block indentation from 4 spaces to 2 spaces
  - Fixed header for config param table

  Applies to #1170

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell eschabell force-pushed the pwhelan-doc-in_mem_metrics branch from 4dae327 to 68fddc1 Compare May 27, 2026 18:36
Copy link
Copy Markdown
Collaborator

@eschabell eschabell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pwhelan this is now waiting on code PR merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants