Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions pipeline/outputs/influxdb.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
| `org` | InfluxDB organization name where the bucket is (v2 only). | `fluent` |
| `port` | TCP port of the target InfluxDB service. | `8086` |
| `sequence_tag` | The name of the tag whose value is incremented for consecutive simultaneous events. | _none_ |
| `strip_prefix` | String prefix to be removed from the front of `tag` when writing InfluxDB measurement names. `strip_prefix` is removed only if it matches exactly at the beginning of and is strictly shorter than `tag` (`tag` length will not be reduced to zero if `strip_prefix` and `tag` are identical strings). | _none_ |

Check warning on line 25 in pipeline/outputs/influxdb.md

View workflow job for this annotation

GitHub Actions / runner / vale

[vale] reported by reviewdog 🐶 [FluentBit.Contractions] Feel free to use 'won't' instead of 'will not'. Raw Output: {"message": "[FluentBit.Contractions] Feel free to use 'won't' instead of 'will not'.", "location": {"path": "pipeline/outputs/influxdb.md", "range": {"start": {"line": 25, "column": 235}}}, "severity": "INFO"}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ueli-g small fix here needed for vale check.

| `tag_keys` | Space-separated list of keys to tag. | _none_ |
| `uri` | Custom URI endpoint. | _none_ |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
Expand Down Expand Up @@ -143,6 +144,85 @@
`Auto_Tags=On` in this example causes an error, because every parsed field value type is `string`. The best usage of this option in metrics like record where one or more field value isn't `string` typed.


### Prefix stripping
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ueli-g please format all yaml and classic config example code with 2 spaced indents (currently using 4?).


When collecting data from many inputs into many buckets, it can be helpful to remove a common prefix using `Strip_prefix`.

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
- name: cpu
tag: cpu.one
- name: cpu
tag: cpu.two
- name: cpu
tag: gpu.one
- name: cpu
tag: gpu.two


outputs:
- name: influxdb
match: 'cpu*'
host: 127.0.0.1
port: 8086
bucket: cpubucket
strip_prefix: cpu.
- name: influxdb
match: 'gpu*'
host: 127.0.0.1
port: 8086
bucket: gpubucket
strip_prefix: gpu.
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[INPUT]
Name cpu
Tag cpu.one

[INPUT]
Name cpu
Tag cpu.two

[INPUT]
Name cpu
Tag gpu.one

[INPUT]
Name cpu
Tag gpu.two

[OUTPUT]
Name influxdb
Host 127.0.0.1
Port 8086
Match cpu*
Bucket cpubucket
Strip_prefix cpu.

[OUTPUT]
Name influxdb
Host 127.0.0.1
Port 8086
Match gpu*
Bucket gpubucket
Strip_prefix gpu.
```
Comment thread
coderabbitai[bot] marked this conversation as resolved.

{% endtab %}
{% endtabs %}


This will result in the buckets `cpubucket` and `gpubucket` each containing two measurement streams named `one` and `two`. Without prefix stripping, the measurement names would be `cpu.one` and `cpu.two` (stored in `cpubucket`), and `gpu.one`, `gpu.two` (stored in `gpubucket`).


### Testing

Before starting Fluent Bit, ensure the target database exists on InfluxDB. Using the previous example, insert the data into a `fluentbit` database.
Expand Down
Loading