Skip to content

Commit c9eece1

Browse files
committed
out_loki: Add descriptions about tenant_id and its around behavior
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
1 parent d3d0a7a commit c9eece1

1 file changed

Lines changed: 42 additions & 0 deletions

File tree

pipeline/outputs/loki.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ There is a separate Golang output plugin provided by [Grafana](https://grafana.c
2929
| `line_format` | Format to use when flattening the record to a log line. Valid values are `json` or `key_value`. If set to `json`, the log line sent to Loki will be the Fluent Bit record dumped as JSON. If set to `key_value`, the log line will be each item in the record concatenated together (separated by a single space) in the format. | `json` |
3030
| `auto_kubernetes_labels` | If set to `true`, adds all Kubernetes labels to the stream labels. | `false` |
3131
| `tenant_id_key` | Specify the name of the key from the original record that contains the Tenant ID. The value of the key is set as `X-Scope-OrgID` of HTTP header. Use to set Tenant ID dynamically. | _none_ |
32+
| `tenant_id_key_error_handling` | Set how `tenant_id_key` split request failures affect the whole chunk. Options are `partial_success`, to treat mixed success and failure as success, or `partial_error`, to retry mixed success and failure. | `partial_success` |
3233
| `buffer_size` | Maximum HTTP response buffer size. | `512KB` |
3334
| `compress` | Set payload compression mechanism. The only available option is `gzip`. | _none_ |
3435
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
@@ -308,6 +309,47 @@ Based on the JSON example provided, the internal stream labels will be:
308309
job="fluentbit", team="Santiago Wanderers"
309310
```
310311

312+
## Use `tenant_id_key`
313+
314+
Use `tenant_id_key` to route records to Loki tenants dynamically. Fluent Bit reads the tenant ID from the specified record key and sends it as the `X-Scope-OrgID` HTTP header. When a chunk contains records for multiple tenants, Fluent Bit splits the chunk into one request per tenant.
315+
316+
The `tenant_id_key_error_handling` option controls how Fluent Bit treats mixed results when one tenant request succeeds and another fails:
317+
318+
- `partial_success`: treat the chunk as successful when at least one tenant request succeeds. This is the default.
319+
- `partial_error`: retry the chunk when any tenant request fails, even if another tenant request succeeds.
320+
321+
{% tabs %}
322+
{% tab title="fluent-bit.yaml" %}
323+
324+
```yaml
325+
pipeline:
326+
outputs:
327+
- name: loki
328+
match: '*'
329+
host: 127.0.0.1
330+
port: 3100
331+
tenant_id_key: tenant_id
332+
tenant_id_key_error_handling: partial_error
333+
remove_keys: tenant_id
334+
```
335+
336+
{% endtab %}
337+
{% tab title="fluent-bit.conf" %}
338+
339+
```text
340+
[OUTPUT]
341+
Name loki
342+
Match *
343+
Host 127.0.0.1
344+
Port 3100
345+
Tenant_ID_Key tenant_id
346+
Tenant_ID_Key_Error_Handling partial_error
347+
Remove_Keys tenant_id
348+
```
349+
350+
{% endtab %}
351+
{% endtabs %}
352+
311353
## Use `drop_single_key`
312354

313355
If there's only one key remaining after removing keys, you can use the `drop_single_key` property to send its value to Loki, rather than a single `key=value` pair.

0 commit comments

Comments
 (0)