From 17d60423c29aac502f54fe57cef7c2193a4b2354 Mon Sep 17 00:00:00 2001 From: "Cheruvu.Sahithee.Vaibhav@ibm.com" Date: Mon, 1 Jun 2026 15:11:29 +0530 Subject: [PATCH 1/3] docs: add Azure Blob service principal authentication Signed-off-by: Cheruvu.Sahithee.Vaibhav@ibm.com --- pipeline/outputs/azure_blob.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/pipeline/outputs/azure_blob.md b/pipeline/outputs/azure_blob.md index 8a83f85f8..1014d6fa3 100644 --- a/pipeline/outputs/azure_blob.md +++ b/pipeline/outputs/azure_blob.md @@ -23,7 +23,7 @@ Fluent Bit exposes the following configuration properties. | Key | Description | Default | | :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------- | | `account_name` | Azure Storage account name. | _none_ | -| `auth_type` | Specify the type to authenticate against the service. Supported values: `key`, `sas`. | `key` | +| `auth_type` | Specify the type to authenticate against the service. Supported values: `key`, `sas`, `service_principal`. | `key` | | `auto_create_container` | If `container_name` doesn't exist in the remote service, enabling this option handles the exception and auto-creates the container. | `true` | | `azure_blob_buffer_key` | Set the Azure Blob buffer key which needs to be specified when using multiple instances of Azure Blob output plugin and buffering is enabled. | `key` | | `blob_type` | Specify the desired blob type. Supported values: `appendblob`, `blockblob`. | `appendblob` | @@ -31,6 +31,9 @@ Fluent Bit exposes the following configuration properties. | `buffer_dir` | Specifies the location of directory where the buffered data will be stored. | `/tmp/fluent-bit/azure-blob/` | | `buffer_file_delete_early` | Whether to delete the buffered file early after successful blob creation. | `false` | | `buffering_enabled` | Enable buffering into disk before ingesting into Azure Blob. | `false` | +| `client_id` | Microsoft Entra ID application/client ID. Mandatory when `auth_type` is `service_principal`. | _none_ | +| `client_secret` | Microsoft Entra ID client secret. Mandatory when `auth_type` is `service_principal`. | _none_ | +| `tenant_id` | Microsoft Entra ID tenant ID. Mandatory when `auth_type` is `service_principal`. | _none_ | | `compress` | Sets payload compression in network transfer. Supported values: `gzip`, `zstd`. | _none_ | | `compress_blob` | Enables compression in the final `blockblob` file. When enabled without `compress`, it uses GZIP; if `compress` is also set, it inherits that codec. This option isn't compatible when `blob_type` = `appendblob`. Fluent Bit returns a configuration error and fails to start. | `false` | | `configuration_endpoint_bearer_token` | Bearer token for the configuration endpoint. | _none_ | @@ -60,6 +63,30 @@ Fluent Bit exposes the following configuration properties. | `upload_timeout` | Optional. Specify a timeout for uploads. Fluent Bit will start ingesting buffer files which have been created more than `x` minutes and haven't reached `upload_file_size` limit yet. | `30m` | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | +## Service principal authentication + +Azure Blob output supports Microsoft Entra ID service principal authentication. To use this authentication method, set `auth_type` to `service_principal` and provide the required credentials: `tenant_id`, `client_id`, and `client_secret`. + +The service principal must have permission to write to the target storage account or container. For example, assign the `Storage Blob Data Contributor` role to the service principal. + +Service principal authentication requires TLS to be enabled (`tls` set to `on`). + +### Example configuration + +```ini +[OUTPUT] + Name azure_blob + Match * + account_name mystorageaccount + container_name logs + blob_type blockblob + auth_type service_principal + tenant_id + client_id + client_secret + tls on +``` + ### Path templating When `path` is set, Fluent Bit resolves the value as a template (similar to the Amazon S3 output) before each upload. The resolved prefix is persisted alongside buffered files, so retries and restarts keep writing to the same Azure path. Leading and trailing slashes are removed automatically to avoid duplicate separators. From 373748ed84901ba5cbae28484277898755bd05b6 Mon Sep 17 00:00:00 2001 From: "Cheruvu.Sahithee.Vaibhav@ibm.com" Date: Mon, 1 Jun 2026 15:24:37 +0530 Subject: [PATCH 2/3] add example Signed-off-by: Cheruvu.Sahithee.Vaibhav@ibm.com --- pipeline/outputs/azure_blob.md | 66 +++++++++++++++++++++------------- 1 file changed, 42 insertions(+), 24 deletions(-) diff --git a/pipeline/outputs/azure_blob.md b/pipeline/outputs/azure_blob.md index 1014d6fa3..36154091a 100644 --- a/pipeline/outputs/azure_blob.md +++ b/pipeline/outputs/azure_blob.md @@ -63,30 +63,6 @@ Fluent Bit exposes the following configuration properties. | `upload_timeout` | Optional. Specify a timeout for uploads. Fluent Bit will start ingesting buffer files which have been created more than `x` minutes and haven't reached `upload_file_size` limit yet. | `30m` | | `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` | -## Service principal authentication - -Azure Blob output supports Microsoft Entra ID service principal authentication. To use this authentication method, set `auth_type` to `service_principal` and provide the required credentials: `tenant_id`, `client_id`, and `client_secret`. - -The service principal must have permission to write to the target storage account or container. For example, assign the `Storage Blob Data Contributor` role to the service principal. - -Service principal authentication requires TLS to be enabled (`tls` set to `on`). - -### Example configuration - -```ini -[OUTPUT] - Name azure_blob - Match * - account_name mystorageaccount - container_name logs - blob_type blockblob - auth_type service_principal - tenant_id - client_id - client_secret - tls on -``` - ### Path templating When `path` is set, Fluent Bit resolves the value as a template (similar to the Amazon S3 output) before each upload. The resolved prefix is persisted alongside buffered files, so retries and restarts keep writing to the same Azure path. Leading and trailing slashes are removed automatically to avoid duplicate separators. @@ -182,6 +158,48 @@ After you run the configuration file, you will be able to query the data using t ![Azure Blob](../../.gitbook/assets/azure_blob.png) +### Configuration with service principal authentication + +To authenticate with Microsoft Entra ID service principal credentials, set `auth_type` to `service_principal` and configure `tenant_id`, `client_id`, and `client_secret`. The service principal must have permission to write to the target storage account or container, such as the `Storage Blob Data Contributor` role. + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + outputs: + - name: azure_blob + match: "*" + account_name: YOUR_ACCOUNT_NAME + container_name: logs + blob_type: blockblob + auth_type: service_principal + tenant_id: YOUR_TENANT_ID + client_id: YOUR_CLIENT_ID + client_secret: YOUR_CLIENT_SECRET + tls: on +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[OUTPUT] + Name azure_blob + Match * + Account_Name YOUR_ACCOUNT_NAME + Container_Name logs + Blob_Type blockblob + Auth_Type service_principal + Tenant_Id YOUR_TENANT_ID + Client_Id YOUR_CLIENT_ID + Client_Secret YOUR_CLIENT_SECRET + Tls on +``` + +{% endtab %} +{% endtabs %} + ### Configuring and using Azure Emulator: Azurite #### Install and run Azurite From 4a0ea41c815bb403cc6046a5c5050ec3b0709acf Mon Sep 17 00:00:00 2001 From: "Cheruvu.Sahithee.Vaibhav@ibm.com" Date: Mon, 1 Jun 2026 21:31:42 +0530 Subject: [PATCH 3/3] docs: address review comments Signed-off-by: Cheruvu.Sahithee.Vaibhav@ibm.com --- pipeline/outputs/azure_blob.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/azure_blob.md b/pipeline/outputs/azure_blob.md index 36154091a..30c0fd0b5 100644 --- a/pipeline/outputs/azure_blob.md +++ b/pipeline/outputs/azure_blob.md @@ -160,7 +160,7 @@ After you run the configuration file, you will be able to query the data using t ### Configuration with service principal authentication -To authenticate with Microsoft Entra ID service principal credentials, set `auth_type` to `service_principal` and configure `tenant_id`, `client_id`, and `client_secret`. The service principal must have permission to write to the target storage account or container, such as the `Storage Blob Data Contributor` role. +To authenticate with Microsoft Entra ID service principal credentials, set `auth_type` to `service_principal` and configure `tenant_id`, `client_id`, and `client_secret`. The service principal must have permission to write to the target storage account or container, such as the `Storage Blob Data Contributor` role. Service principal authentication requires `tls` to be set to `on` to ensure secure communication. {% tabs %} {% tab title="fluent-bit.yaml" %}