Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions pipeline/outputs/file.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,11 @@ The _File_ output plugin lets you write the data received through the input plug
| Key | Description | Default |
| :--- | :--- | :--- |
| `file` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. | _none_ |
| `files_rotation` | Enable size-based [log rotation](#log-rotation). When enabled, files that exceed `max_size` are rotated and optionally compressed. | `false` |
Comment thread
eschabell marked this conversation as resolved.
| `format` | The [format](#format) of the file content. | _none_ |
| `gzip` | Compress rotated files using gzip. Only applies when `files_rotation` is enabled. | `true` |
| `max_files` | Maximum number of rotated files to retain per output file. Oldest files are deleted first. Must be `1` or greater. Only applies when `files_rotation` is enabled. | `7` |
| `max_size` | Maximum size of the active output file before rotation is triggered. Supports size suffixes: `k` (kilobytes), `m` (megabytes), `g` (gigabytes). Only applies when `files_rotation` is enabled. | `100m` |
Comment thread
eschabell marked this conversation as resolved.
| `mkdir` | Recursively create output directory if it doesn't exist. Permissions set to `0755`. | `false` |
| `path` | Directory path to store files. If not set, Fluent Bit will write the files in its own working directory. | _none_ |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `1` |
Expand Down Expand Up @@ -112,6 +116,64 @@ You will get the following output:
1564462620.000254 used=1045448 free=31760160 total=32805608
```

## Log rotation

The File output plugin supports size-based log rotation.

When `files_rotation` is enabled, the plugin monitors the size of each output file. Once a file exceeds `max_size`, the next flush rotates the file by renaming it with a timestamp suffix in the format `<filename>.<YYYYMMDD_HHMMSS_XXXXXXXX>`. The `YYYYMMDD_HHMMSS` is machine local timestamp of the time the rotation occurred, and `XXXXXXXX` is a random identifier to guarantee unique filenames if multiple rotations happen within the same second.

If `gzip` is enabled (the default), rotated files are compressed with gzip and stored with an additional `.gz` extension (for example, `cpu.log.20260512_134500_a1b2c3d4.gz`).

The plugin retains up to `max_files` rotated files per output file. When the limit is reached, the oldest rotated files are deleted automatically.

Log rotation works with all supported output [formats](#format): `plain`, `CSV`, `LTSV`, `template`, and `msgpack`. File operations are thread-safe, so rotation can be used alongside multiple [workers](../../administration/multithreading.md#outputs).

### Log rotation example

The following configuration writes CPU metrics to file with rotation enabled. Files are rotated at 50 MB and the five most recent rotated files are retained with gzip compression:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
- name: cpu
tag: cpu

outputs:
- name: file
match: '*'
path: /var/log/fluent-bit
file: cpu.log
files_rotation: true
max_size: 50m
max_files: 5
gzip: true
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[INPUT]
Name cpu
Tag cpu

[OUTPUT]
Name file
Match *
Path /var/log/fluent-bit
File cpu.log
Files_Rotation true
Max_Size 50m
Max_Files 5
Gzip true
```

{% endtab %}
{% endtabs %}

## Get started

You can run the plugin from the command line or through the configuration file.
Expand Down
Loading