Skip to content

http_server: document new ingress config and metrics #2546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
edsiper merged 3 commits into from
Apr 15, 2026
Merged

http_server: document new ingress config and metrics #2546

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0a6b639
administration: document shared HTTP ingress queue settings
edsiper Apr 13, 2026
14960d5
monitoring: document HTTP ingress queue metrics
edsiper Apr 13, 2026
bab3b18
inputs: clarify shared HTTP listener settings
edsiper Apr 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions administration/configuring-fluent-bit/yaml/pipeline-section.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,27 @@

Some HTTP-based input plugins share the same listener implementation and support the following common settings in addition to their plugin-specific parameters:

These settings are shared by HTTP-based inputs such as `http`, `splunk`,
`elasticsearch`, `opentelemetry`, and `prometheus_remote_write`.
Use these keys the same way across those plugins.

If a plugin page shows one of the `http_server.*` keys in its configuration
table, it is documenting one of these shared listener settings, not a

Check warning on line 89 in administration/configuring-fluent-bit/yaml/pipeline-section.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Contractions] Feel free to use 'it's' instead of 'it is'.

Raw Output:
{"message": "[FluentBit.Contractions] Feel free to use 'it's' instead of 'it is'.", "location": {"path": "administration/configuring-fluent-bit/yaml/pipeline-section.md", "range": {"start": {"line": 89, "column": 8}}}, "severity": "INFO"}
plugin-specific behavior.

| Key | Description | Default |
| --- | ----------- | ------- |
| `http_server.http2` | Enable HTTP/2 support for the input listener. | `true` |
| `http_server.buffer_max_size` | Set the maximum size of the HTTP request buffer. | `4M` |
| `http_server.buffer_chunk_size` | Set the allocation chunk size used for the HTTP request buffer. | `512K` |
| `http_server.max_connections` | Set the maximum number of concurrent active HTTP connections. `0` means unlimited. | `0` |
| `http_server.workers` | Set the number of HTTP listener worker threads. | `1` |
| `http_server.ingress_queue_event_limit` | Set the maximum number of deferred ingress queue entries. This setting applies only when `http_server.workers` is greater than `1`. | `8192` |
| `http_server.ingress_queue_byte_limit` | Set the maximum size of the deferred ingress queue. This setting applies only when `http_server.workers` is greater than `1`. | `256M` |

When `http_server.workers` is `1`, Fluent Bit does not use the deferred

Check warning on line 102 in administration/configuring-fluent-bit/yaml/pipeline-section.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Contractions] Feel free to use 'doesn't' instead of 'does not'.

Raw Output:
{"message": "[FluentBit.Contractions] Feel free to use 'doesn't' instead of 'does not'.", "location": {"path": "administration/configuring-fluent-bit/yaml/pipeline-section.md", "range": {"start": {"line": 102, "column": 47}}}, "severity": "INFO"}
ingress queue, so the two `http_server.ingress_queue_*` settings have no
effect.

For backward compatibility, some plugins also accept the legacy aliases `http2`, `buffer_max_size`, `buffer_chunk_size`, `max_connections`, and `workers`.

Expand Down
2 changes: 1 addition & 1 deletion administration/configuring-fluent-bit/yaml/service-section.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ The `service` section of YAML configuration files defines global properties of t
| `streams_file` | Path for the [stream processor](../../../stream-processing/overview.md) configuration file. This file defines the rules and operations for stream processing in Fluent Bit. Stream processor configurations can also be defined directly in the `streams` section of YAML configuration files. | _none_ |
| `windows.maxstdio` | If specified, adjusts the limit of `stdio`. Only provided for Windows. Values from `512` to `2048` are allowed. | `512` |

The `service` section only controls the built-in monitoring and control HTTP server. Plugin-specific HTTP listener settings such as `http_server.http2`, `http_server.buffer_max_size`, `http_server.buffer_chunk_size`, `http_server.max_connections`, and `http_server.workers` are configured on the relevant input plugin in the [`pipeline.inputs`](../yaml/pipeline-section.md#shared-http-listener-settings-for-inputs) section.
The `service` section only controls the built-in monitoring and control HTTP server. Plugin-specific HTTP listener settings such as `http_server.http2`, `http_server.buffer_max_size`, `http_server.buffer_chunk_size`, `http_server.max_connections`, `http_server.workers`, `http_server.ingress_queue_event_limit`, and `http_server.ingress_queue_byte_limit` are configured on the relevant input plugin in the [`pipeline.inputs`](../yaml/pipeline-section.md#shared-http-listener-settings-for-inputs) section.

## Storage configuration

Expand Down
7 changes: 7 additions & 0 deletions administration/monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,10 @@ The following terms are key to understanding how Fluent Bit processes metrics:

The Fluent Bit engine attempts to fit records into chunks of at most `2 MB`, but the size can vary at runtime. Chunks are then sent to an output. An output plugin instance can successfully send the full chunk to the destination and mark it as successful. If an unrecoverable error is encountered, the chunk fails entirely. Otherwise, the output can request a retry.

Some metrics are available only for specific plugins or runtime modes. For
example, the `fluentbit_input_http_server_ingress_queue_*` metrics are exposed
only for HTTP-based inputs that use `http_server.workers` greater than `1`.

| Metric Name | Labels | Description | Type | Unit |
| ----------- | ------ | ----------- | ---- | ---- |
| `fluentbit_build_info` | hostname: the hostname, version: the version of Fluent Bit, os: OS type | Build version information. The value is the Unix epoch timestamp of the configuration context initialization. | gauge | seconds |
Expand All @@ -202,6 +206,9 @@ The following terms are key to understanding how Fluent Bit processes metrics:
| `fluentbit_input_files_opened_total` | name: the name or alias for the input instance | The total number of opened files. Only available for the [Tail](../pipeline/inputs/tail.md) input plugin. | counter | files |
| `fluentbit_input_files_rotated_total` | name: the name or alias for the input instance | The total number of rotated files. Only available for the [Tail](../pipeline/inputs/tail.md) input plugin. | counter | files |
| `fluentbit_input_ingestion_paused` | name: the name or alias for the input instance | Indicates whether the input instance ingestion is currently paused (1) or not (0). | gauge | boolean |
| `fluentbit_input_http_server_ingress_queue_busy_total` | name: the name or alias for the input instance | The total number of times an HTTP worker input hit a full deferred ingress queue and applied backpressure. Only available for HTTP-based inputs using `http_server.workers` greater than `1`. | counter | events |
| `fluentbit_input_http_server_ingress_queue_pending_bytes` | name: the name or alias for the input instance | The current size of the deferred ingress queue in bytes. Only available for HTTP-based inputs using `http_server.workers` greater than `1`. | gauge | bytes |
| `fluentbit_input_http_server_ingress_queue_pending_events` | name: the name or alias for the input instance | The current number of deferred ingress queue entries. Only available for HTTP-based inputs using `http_server.workers` greater than `1`. | gauge | events |
| `fluentbit_input_long_line_skipped_total` | name: the name or alias for the input instance | The total number of skipped occurrences for long lines. Only available for the [Tail](../pipeline/inputs/tail.md) input plugin when `skip_long_lines` is enabled. | counter | occurrences |
| `fluentbit_input_long_line_truncated_total` | name: the name or alias for the input instance | The total number of truncated occurrences for long lines. Only available for the [Tail](../pipeline/inputs/tail.md) input plugin when `truncate_long_lines` is enabled. | counter | occurrences |
| `fluentbit_input_memrb_dropped_bytes` | name: the name or alias for the input instance | The number of bytes dropped by the memory ring buffer (`memrb`) storage type when the buffer is full. Only available for input plugins with `storage.type` set to `memrb`. | counter | bytes |
Expand Down
2 changes: 2 additions & 0 deletions installation/upgrade-notes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ The HTTP-based input plugins now use a shared HTTP listener configuration model.
- `http_server.buffer_max_size`
- `http_server.max_connections`
- `http_server.workers`
- `http_server.ingress_queue_event_limit`
- `http_server.ingress_queue_byte_limit`

Legacy per-plugin names such as `http2`, `buffer_chunk_size`, and `buffer_max_size` are still accepted as compatibility aliases, but new configurations should use the `http_server.*` names.

Expand Down
2 changes: 2 additions & 0 deletions installation/whats-new-in-fluent-bit-v5.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ HTTP-based inputs now use a shared listener configuration model. The preferred s
- `http_server.buffer_max_size`
- `http_server.max_connections`
- `http_server.workers`
- `http_server.ingress_queue_event_limit`
- `http_server.ingress_queue_byte_limit`

Legacy aliases such as `http2`, `buffer_chunk_size`, and `buffer_max_size` still work, but new configurations should use the `http_server.*` names.

Expand Down
14 changes: 14 additions & 0 deletions pipeline/inputs/elasticsearch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@

The plugin supports the following configuration parameters:

The table below includes both:

Check warning on line 13 in pipeline/inputs/elasticsearch.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Directional] Verify your use of 'below' with the Style Guide.

Raw Output:
{"message": "[FluentBit.Directional] Verify your use of 'below' with the Style Guide.", "location": {"path": "pipeline/inputs/elasticsearch.md", "range": {"start": {"line": 13, "column": 11}}}, "severity": "INFO"}

- settings specific to the Elasticsearch input plugin
- shared `http_server.*` listener settings that are used by several HTTP-based inputs

For a cross-plugin explanation of the shared listener settings, see
[Shared HTTP listener settings for inputs](../../administration/configuring-fluent-bit/yaml/pipeline-section.md#shared-http-listener-settings-for-inputs).

| Key | Description | Default value |
|:--------------------|:-----------------------------------------------------------------------------------------------------------------------------------------|:--------------|
| `buffer_chunk_size` | Set the buffer chunk size. Compatibility alias for `http_server.buffer_chunk_size`. | `512K` |
Expand All @@ -18,13 +26,19 @@
| `http2` | Enable HTTP/2 support. Compatibility alias for `http_server.http2`. | `true` |
| `http_server.max_connections` | Maximum number of concurrent active HTTP connections. `0` means unlimited. | `0` |
| `http_server.workers` | Number of HTTP listener worker threads. | `1` |
| `http_server.ingress_queue_event_limit` | Maximum number of deferred ingress queue entries. Applies only when `http_server.workers` is greater than `1`. | `8192` |
| `http_server.ingress_queue_byte_limit` | Maximum size of the deferred ingress queue. Applies only when `http_server.workers` is greater than `1`. | `256M` |
| `listen` | The address to listen on. | `0.0.0.0` |
| `meta_key` | Specify a key name for meta information. | `@meta` |
| `port` | The port for Fluent Bit to listen on. | `9200` |
| `tag_key` | Specify a key name for extracting as a tag. | `NULL` |
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
| `version` | Specify the Elasticsearch version that Fluent Bit reports to clients during sniffing and API requests. | `8.0.0` |

The `http_server.ingress_queue_event_limit` and
`http_server.ingress_queue_byte_limit` settings matter only when
`http_server.workers` is greater than `1`.

### TLS / SSL

The Elasticsearch input plugin supports TLS/SSL for receiving data from Beats agents or other clients over encrypted connections. For more details about the properties available and general configuration, refer to [Transport Security](../../administration/transport-security.md).
Expand Down
14 changes: 14 additions & 0 deletions pipeline/inputs/http.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@

## Configuration parameters

The table below includes both:

Check warning on line 12 in pipeline/inputs/http.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Directional] Verify your use of 'below' with the Style Guide.

Raw Output:
{"message": "[FluentBit.Directional] Verify your use of 'below' with the Style Guide.", "location": {"path": "pipeline/inputs/http.md", "range": {"start": {"line": 12, "column": 11}}}, "severity": "INFO"}

- settings specific to the HTTP input plugin
- shared `http_server.*` listener settings that are used by several HTTP-based inputs

For a cross-plugin explanation of the shared listener settings, see
[Shared HTTP listener settings for inputs](../../administration/configuring-fluent-bit/yaml/pipeline-section.md#shared-http-listener-settings-for-inputs).

| Key | Description | Default |
|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|-----------|
| `add_remote_addr` | Adds a `REMOTE_ADDR` field to the record. The value of `REMOTE_ADDR` is the client's address, which is extracted from the `X-Forwarded-For` header. | `false` |
Expand All @@ -17,6 +25,8 @@
| `http2` | Enable HTTP/2 support. Compatibility alias for `http_server.http2`. | `true` |
| `http_server.max_connections` | Maximum number of concurrent active HTTP connections. `0` means unlimited. | `0` |
| `http_server.workers` | Number of HTTP listener worker threads. | `1` |
| `http_server.ingress_queue_event_limit` | Maximum number of deferred ingress queue entries. Applies only when `http_server.workers` is greater than `1`. | `8192` |
| `http_server.ingress_queue_byte_limit` | Maximum size of the deferred ingress queue. Applies only when `http_server.workers` is greater than `1`. | `256M` |
| `listen` | The address to listen on. | `0.0.0.0` |
| `oauth2.allowed_audience` | Audience claim to enforce when validating incoming `OAuth 2.0` `JWT` tokens. | _none_ |
| `oauth2.allowed_clients` | Authorized `client_id` or `azp` claim values. Can be specified multiple times. | _none_ |
Expand All @@ -31,6 +41,10 @@
| `tag_key` | Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key. | _none_ |
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |

The `http_server.ingress_queue_event_limit` and
`http_server.ingress_queue_byte_limit` settings matter only when
`http_server.workers` is greater than `1`.

### TLS / SSL

HTTP input plugin supports TLS/SSL. For more details about the properties available and general configuration, refer to [Transport Security](../../administration/transport-security.md).
Expand Down
14 changes: 14 additions & 0 deletions pipeline/inputs/opentelemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,14 @@

## Configuration

The table below includes both:

Check warning on line 17 in pipeline/inputs/opentelemetry.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Directional] Verify your use of 'below' with the Style Guide.

Raw Output:
{"message": "[FluentBit.Directional] Verify your use of 'below' with the Style Guide.", "location": {"path": "pipeline/inputs/opentelemetry.md", "range": {"start": {"line": 17, "column": 11}}}, "severity": "INFO"}

- settings specific to the OpenTelemetry input plugin
- shared `http_server.*` listener settings that are used by several HTTP-based inputs

For a cross-plugin explanation of the shared listener settings, see
[Shared HTTP listener settings for inputs](../../administration/configuring-fluent-bit/yaml/pipeline-section.md#shared-http-listener-settings-for-inputs).

| Key | Description | Default |
|-------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|
| `alias` | Sets an alias for multiple instances of the same input plugin. If no alias is specified, a default name will be assigned using the plugin name followed by a dot and a sequence number. | _none_ |
Expand All @@ -24,6 +32,8 @@
| `http2` | Enable HTTP/2 protocol support for the OpenTelemetry receiver. | `true` |
| `http_server.max_connections` | Maximum number of concurrent active HTTP connections. `0` means unlimited. | `0` |
| `http_server.workers` | Number of HTTP listener worker threads. | `1` |
| `http_server.ingress_queue_event_limit` | Maximum number of deferred ingress queue entries. Applies only when `http_server.workers` is greater than `1`. | `8192` |
| `http_server.ingress_queue_byte_limit` | Maximum size of the deferred ingress queue. Applies only when `http_server.workers` is greater than `1`. | `256M` |
| `listen` | The network address to listen on. | `0.0.0.0` |
| `log_level` | Specifies the log level for this plugin. If not set here, the plugin uses the global log level specified in the `service` section of your configuration file. | `info` |
| `log_suppress_interval` | Suppresses log messages from this plugin that appear similar within a specified time interval. `0` no suppression. | `0` |
Expand Down Expand Up @@ -69,6 +79,10 @@
| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` |
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |

The `http_server.ingress_queue_event_limit` and
`http_server.ingress_queue_byte_limit` settings matter only when
`http_server.workers` is greater than `1`.

When `raw_traces` is set to `false` (default), the traces endpoint (`/v1/traces`) processes incoming trace data using the unified JSON parser with strict validation. The endpoint accepts both `protobuf` and `JSON` encoded payloads. When `raw_traces` is set to `true`, any data forwarded to the traces endpoint will be packed and forwarded as a log message without processing, validation, or conversion to the Fluent Bit internal trace format.

### OpenTelemetry transport protocol endpoints
Expand Down
Loading
Loading