You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+47Lines changed: 47 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -110,6 +110,53 @@ When you create a new `.md` file for a new page, you must add an entry to this r
110
110
111
111
Similarly, if you update the `# h1` title header of an existing page, be sure to update that page's `SUMMARY.md` entry to match. `SUMMARY.md` entries takes precedence over in-page headers, which means that if you update a page's `# h1` title without updating `SUMMARY.md`, the unchanged `SUMMARY.md` title will persist in both the rendered page and the table of contents.
112
112
113
+
## Plugin documentation
114
+
115
+
When documenting a Fluent Bit plugin, follow these standards to keep plugin pages consistent.
116
+
117
+
### Configuration parameters table
118
+
119
+
-**Sort parameters alphabetically** by key name.
120
+
-**List all parameters** defined in the plugin's `config_map`, plus the common `workers` parameter.
121
+
-**Use `_none_` in the Default cell** when the parameter's default is `NULL` in the source `config_map`. Leave the cell empty only when the parameter has no entry in `config_map` at all (for example, `host` and `port`, which come from the network defaults).
122
+
- Use this table header format:
123
+
124
+
```markdown
125
+
| Key | Description | Default |
126
+
| --- | ----------- | ------- |
127
+
```
128
+
129
+
### Configuration examples
130
+
131
+
Every plugin page should include both a YAML and a classic configuration example, presented as tabs:
132
+
133
+
```markdown
134
+
{% tabs %}
135
+
{% tab title="fluent-bit.yaml" %}
136
+
...
137
+
{% endtab %}
138
+
{% tab title="fluent-bit.conf" %}
139
+
...
140
+
{% endtab %}
141
+
{% endtabs %}
142
+
```
143
+
144
+
### Key casing in examples
145
+
146
+
Key casing differs between the two configuration formats:
147
+
148
+
-**YAML** (`fluent-bit.yaml`): All keys are lowercase. For example: `api_key`, `log_level`, `match`.
149
+
-**Classic** (`fluent-bit.conf`): Keys use Title_Case — capitalize the first letter of every underscore-separated word. For example: `Api_Key`, `Log_Level`, `Match`.
150
+
151
+
| YAML key | Classic key |
152
+
| --- | --- |
153
+
|`api_key`|`Api_Key`|
154
+
|`log_group_name`|`Log_Group_Name`|
155
+
|`match`|`Match`|
156
+
|`name`|`Name`|
157
+
158
+
This applies to all keys in every section (`[SERVICE]`, `[INPUT]`, `[OUTPUT]`, and so on).
159
+
113
160
## Linters
114
161
115
162
This repository runs linters as GitHub Actions for each pull request. If a linter finds errors or makes suggested changes, you can view these results in the **Files changed** tab.
Copy file name to clipboardExpand all lines: README.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,6 +37,8 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces
37
37
38
38
For more details about changes in each release, refer to the [official release notes](https://fluentbit.io/announcements/).
39
39
40
+
If you are upgrading from the Fluent Bit `4.2` series, start with [What's new in Fluent Bit v5.0](installation/whats-new-in-fluent-bit-v5.0.md) and [Upgrade notes](installation/upgrade-notes.md).
41
+
40
42
## Fluent Bit, Fluentd, and CNCF
41
43
42
44
Fluent Bit is a [CNCF](https://www.cncf.io/) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org).
Both [Fluentd](https://www.fluentd.org) and [Fluent Bit](https://fluentbit.io) can work as Aggregators or Forwarders, and can complement each other or be used as standalone solutions.
Copy file name to clipboardExpand all lines: about/what-is-fluent-bit.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,18 +2,18 @@
2
2
description: Fluent Bit is a CNCF graduated project under the umbrella of Fluentd
3
3
---
4
4
5
-
# What is Fluent Bit?
5
+
# What's Fluent Bit?
6
6
7
-
[Fluent Bit](https://fluentbit.io) is an open source telemetry agent specifically designed to efficiently handle the challenges of collecting and processing telemetry data across a wide range of environments, from constrained systems to complex cloud infrastructures. Managing telemetry data from various sources and formats can be a constant challenge, particularly when performance is a critical factor.
7
+
[Fluent Bit](https://fluentbit.io) is an open source telemetry agent that processes logs, metrics, traces, and profiles. It's designed to efficiently handle the challenges of collecting and processing telemetry data across a wide range of environments, from constrained systems to complex cloud infrastructures. Managing telemetry data from various sources and formats can be a constant challenge, particularly when performance is a critical factor.
8
8
9
-
Rather than serving as a drop-in replacement, Fluent Bit enhances the observability strategy for your infrastructure by adapting and optimizing your existing logging layer, and adding metrics and traces processing. Fluent Bit supports a vendor-neutral approach, seamlessly integrating with other ecosystems such as Prometheus and OpenTelemetry. Trusted by major cloud providers, banks, and companies in need of a ready-to-use telemetry agent solution, Fluent Bit effectively manages diverse data sources and formats while maintaining optimal performance and keeping resource consumption low.
9
+
Rather than serving as a drop-in replacement, Fluent Bit enhances the observability strategy for your infrastructure. It adapts and optimizes your existing logging layer, and adds metrics and traces processing. Fluent Bit supports a vendor-neutral approach, with native OpenTelemetry (OTLP) ingestion and delivery and seamless integration with ecosystems such as Prometheus. Trusted by major cloud providers, banks, and companies that need a ready-to-use telemetry agent, Fluent Bit effectively manages diverse data sources and formats. It maintains optimal performance while keeping resource consumption low.
10
10
11
11
Fluent Bit can be deployed as an edge agent for localized telemetry data handling or utilized as a central aggregator/collector for managing telemetry data across multiple sources and environments.
In 2014, the [Fluentd](https://www.fluentd.org/) team at [Treasure Data](https://www.treasuredata.com/) was forecasting the need for a lightweight log processor for constraint environments like embedded Linux and gateways. To meet this need, Eduardo Silva created Fluent Bit, a new open-source solution and part of the Fluentd ecosystem.
17
+
In 2014, the [Fluentd](https://www.fluentd.org/) team at [Treasure Data](https://www.treasuredata.com/) was forecasting the need for a lightweight log processor for constraint environments like embedded Linux and gateways. To meet this need, Eduardo Silva created Fluent Bit, a new opensource solution and part of the Fluentd ecosystem.
18
18
19
19
After the project matured, it gained traction for normal Linux systems. With the new containerized world, the cloud native community asked to extend the project scope to support more sources, filters, and destinations. Not long after, Fluent Bit became one of the preferred solutions to solve the logging challenges in cloud environments.
All AWS plugins additionally support a `role_arn` (or `AWS_ROLE_ARN`, for [Elasticsearch](../pipeline/outputs/elasticsearch.md)) configuration parameter. If specified, the fetched credentials are used to assume the given role.
12
13
@@ -42,3 +43,7 @@ Credentials are fetched using a pod identity endpoint. See [Learn how EKS Pod I
42
43
## EC2 instance profile credentials (IMDS)
43
44
44
45
Fetches credentials for the EC2 instance profile's role. See [IAM roles for Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html). As of Fluent Bit version 1.8.8, IMDSv2 is used by default and IMDSv1 might be disabled. Prior versions of Fluent Bit require enabling IMDSv1 on EC2.
46
+
47
+
## `AWS Greengrass` credentials
48
+
49
+
Fluent Bit fetches credentials from a localhost endpoint provided by the AWS IoT `Greengrass` token exchange service. The token exchange service runs as a local server on `Greengrass` core devices and provides AWS credentials through the `AWS_CONTAINER_CREDENTIALS_FULL_URI` and `AWS_CONTAINER_AUTHORIZATION_TOKEN` environment variables. For more information, see the AWS documentation about [Token exchange service](https://docs.aws.amazon.com/greengrass/v2/developerguide/token-exchange-service-component.html).
Copy file name to clipboardExpand all lines: administration/configuring-fluent-bit.md
+16Lines changed: 16 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,3 +29,19 @@ podman run -rm -ti fluent/fluent-bit --help
29
29
# Docker container tooling.
30
30
docker run --rm -it fluent/fluent-bit --help
31
31
```
32
+
33
+
### Validate configuration with `--dry-run`
34
+
35
+
Use the `--dry-run` flag to validate a configuration file without starting Fluent Bit:
36
+
37
+
```shell
38
+
fluent-bit --dry-run -c /path/to/fluent-bit.yaml
39
+
```
40
+
41
+
A successful validation prints `configuration test is successful` and exits with code `0`. If validation fails, Fluent Bit exits with a non-zero code and prints the errors to stderr.
42
+
43
+
As of Fluent Bit 4.2, `--dry-run` performs full property validation in addition to syntax checking. Prior to 4.2, unknown or misspelled plugin property names would only surface as errors at runtime; `--dry-run` now catches them during validation. For example, a configuration with an unknown property on a `dummy` input produces:
Copy file name to clipboardExpand all lines: administration/configuring-fluent-bit/yaml/pipeline-section.md
+66Lines changed: 66 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -77,6 +77,49 @@ The `inputs` section defines one or more [input plugins](../../../pipeline/input
77
77
78
78
The `name` parameter is required and defines for Fluent Bit which input plugin should be loaded. The `tag` parameter is required for all plugins except for the `forward` plugin, which provides dynamic tags.
79
79
80
+
### Shared HTTP listener settings for inputs
81
+
82
+
Some HTTP-based input plugins share the same listener implementation and support the following common settings in addition to their plugin-specific parameters:
83
+
84
+
These settings are shared by HTTP-based inputs such as `http`, `splunk`,
85
+
`elasticsearch`, `opentelemetry`, and `prometheus_remote_write`.
86
+
Use these keys the same way across those plugins.
87
+
88
+
If a plugin page shows one of the `http_server.*` keys in its configuration
89
+
table, it is documenting one of these shared listener settings, not a
90
+
plugin-specific behavior.
91
+
92
+
| Key | Description | Default |
93
+
| --- | ----------- | ------- |
94
+
| `http_server.http2` | Enable HTTP/2 support for the input listener. | `true` |
95
+
| `http_server.buffer_max_size` | Set the maximum size of the HTTP request buffer. | `4M` |
96
+
| `http_server.buffer_chunk_size` | Set the allocation chunk size used for the HTTP request buffer. | `512K` |
97
+
| `http_server.max_connections` | Set the maximum number of concurrent active HTTP connections. `0` means unlimited. | `0` |
98
+
| `http_server.workers` | Set the number of HTTP listener worker threads. | `1` |
99
+
| `http_server.ingress_queue_event_limit` | Set the maximum number of deferred ingress queue entries. This setting applies only when `http_server.workers` is greater than `1`. | `8192` |
100
+
| `http_server.ingress_queue_byte_limit` | Set the maximum size of the deferred ingress queue. This setting applies only when `http_server.workers` is greater than `1`. | `256M` |
101
+
102
+
When `http_server.workers` is `1`, Fluent Bit does not use the deferred
103
+
ingress queue, so the two `http_server.ingress_queue_*` settings have no
104
+
effect.
105
+
106
+
For backward compatibility, some plugins also accept the legacy aliases `http2`, `buffer_max_size`, `buffer_chunk_size`, `max_connections`, and `workers`.
Copy file name to clipboardExpand all lines: administration/configuring-fluent-bit/yaml/service-section.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -36,6 +36,8 @@ The `service` section of YAML configuration files defines global properties of t
36
36
|`streams_file`| Path for the [stream processor](../../../stream-processing/overview.md) configuration file. This file defines the rules and operations for stream processing in Fluent Bit. Stream processor configurations can also be defined directly in the `streams` section of YAML configuration files. |_none_|
37
37
|`windows.maxstdio`| If specified, adjusts the limit of `stdio`. Only provided for Windows. Values from `512` to `2048` are allowed. |`512`|
38
38
39
+
The `service` section only controls the built-in monitoring and control HTTP server. Plugin-specific HTTP listener settings such as `http_server.http2`, `http_server.buffer_max_size`, `http_server.buffer_chunk_size`, `http_server.max_connections`, `http_server.workers`, `http_server.ingress_queue_event_limit`, and `http_server.ingress_queue_byte_limit` are configured on the relevant input plugin in the [`pipeline.inputs`](../yaml/pipeline-section.md#shared-http-listener-settings-for-inputs) section.
40
+
39
41
## Storage configuration
40
42
41
43
The following storage-related keys can be set as children to the `storage` key:
0 commit comments