Fluent Bit 4.0.1 azure_kusto output plugin SIGSEV

[neumanndaniel]

Bug Report

Describe the bug
With Fluent Bit 4.0.1 the buffering functionality was added to azure_kusto #9797. It seems that the flush routine for the log data to Azure Data Explorer is somehow broken.

The same output configuration works with Fluent Bit 4.0.0

To Reproduce

• Example log message if applicable:

❯ kubectl logs fluent-bit-2vm6h -f
Fluent Bit v4.0.1
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/04/25 21:50:13] [ info] [fluent bit] version=4.0.1, commit=b12e507090, pid=1
[2025/04/25 21:50:13] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/04/25 21:50:13] [ info] [simd    ] disabled
[2025/04/25 21:50:13] [ info] [cmetrics] version=1.0.0
[2025/04/25 21:50:13] [ info] [ctraces ] version=0.6.4
[2025/04/25 21:50:13] [ info] [filter:kubernetes:logs_filter_1] https=1 host=kubernetes.default.svc port=443
[2025/04/25 21:50:13] [ info] [filter:kubernetes:logs_filter_1]  token updated
[2025/04/25 21:50:13] [ info] [filter:kubernetes:logs_filter_1] local POD info OK
[2025/04/25 21:50:13] [ info] [filter:kubernetes:logs_filter_1] testing connectivity with Kubelet...
[2025/04/25 21:50:13] [ info] [filter:kubernetes:logs_filter_1] connectivity OK
[2025/04/25 21:50:13] [ info] [output:azure_kusto:azure_kusto.0] endpoint='https://ingest-adxaks.northeurope.kusto.windows.net', database='Kubernetes', table='ContainerLogs'
[2025/04/25 21:50:13] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020
[2025/04/25 21:50:13] [ info] [sp] stream processor started
[2025/04/25 21:50:27] [ info] [output:azure_kusto:azure_kusto.0] loading kusto ingestion resources and refresh interval is 6293524
[2025/04/25 21:50:27] [ info] [output:azure_kusto:azure_kusto.0] loading kusto ingestion resources and refresh interval is 6144561
[2025/04/25 21:50:27] [ info] [output:azure_kusto:azure_kusto.0] loading kusto ingestion resources and refresh interval is 5675841
[2025/04/25 21:50:27] [ info] [output:azure_kusto:azure_kusto.0] loading kusto ingestion resources and refresh interval is 4535354
[2025/04/25 21:50:27] [ info] [output:azure_kusto:azure_kusto.0] loading kusto ingestion resources and refresh interval is 3995692
[2025/04/25 21:50:27] [ info] [output:azure_kusto:azure_kusto.0] loading kusto ingestion resources and refresh interval is 6206214
[2025/04/25 21:50:27] [ info] [azure msi auth] HTTP Status=200
[2025/04/25 21:50:27] [ info] [azure msi auth] access token from '169.254.169.254:80' retrieved
[2025/04/25 21:50:27] [engine] caught signal (SIGSEGV)
#0  0x7e23a9f9c623      in  ???() at ???:0
#1  0x7e23aa18718b      in  ???() at ???:0
#2  0x7e23aa193f1e      in  ???() at ???:0
#3  0x7e23aa1947c3      in  ???() at ???:0
#4  0x7e23aa196335      in  ???() at ???:0
#5  0x7e23aa19740b      in  ???() at ???:0
#6  0x7e23aa3668ff      in  ???() at ???:0
#7  0x7e23aa3a448d      in  ???() at ???:0
#8  0x7e23aa3a06c4      in  ???() at ???:0
#9  0x5e94375660bc      in  tls_net_handshake() at src/tls/openssl.c:1097
#10 0x5e9437566e04      in  flb_tls_session_create() at src/tls/flb_tls.c:593
#11 0x5e9437575882      in  flb_io_net_connect() at src/flb_io.c:175
#12 0x5e943754bd20      in  create_conn() at src/flb_upstream.c:610
#13 0x5e943754bd20      in  flb_upstream_conn_get() at src/flb_upstream.c:799
#14 0x5e943761ffb3      in  execute_ingest_csl_command() at plugins/out_azure_kusto/azure_kusto.c:172
#15 0x5e9437620d03      in  azure_kusto_load_ingestion_resources() at plugins/out_azure_kusto/azure_kusto_conf.c:561
#16 0x5e943761e5cd      in  cb_azure_kusto_flush() at plugins/out_azure_kusto/azure_kusto.c:1356
#17 0x5e9437ba0e66      in  co_init() at lib/monkey/deps/flb_libco/amd64.c:117
#18 0xffffffffffffffff  in  ???() at ???:0

• Steps to reproduce the problem:

azure_kusto output plugin configuration:

Note

Ingestion_Endpoint, Database_Name, Table_Name and Ingestion_Mapping need to be adapted.

[OUTPUT]
    Name                        azure_kusto
    Match                       kubernetes.logs.*
    managed_identity_client_id  SYSTEM
    Ingestion_Endpoint          https://ingest-adxaks.northeurope.kusto.windows.net
    Database_Name               Kubernetes
    Table_Name                  ContainerLogs
    Ingestion_Mapping_Reference FluentBitMapping
    Log_Key                     log
    Include_Tag_Key             Off
    Include_Time_Key            Off
    Retry_Limit                 False
    Log_Level                   info
    compression_enabled         on
    ingestion_endpoint_connect_timeout 60
    ingestion_resources_refresh_interval 3600

Expected behavior
That azure_kusto in Fluent Bit 4.0.1 works exactly like in Fluent Bit 4.0.0

Screenshots

Your Environment

• Version used: 4.0.1
• Configuration:
• Environment name and version (e.g. Kubernetes? What version?): Kubernetes (Azure Kubernetes Service) version 1.32.3
• Server type and version:
• Operating System and version: Azure Linux 3.0
• Filters and plugins: tail, kubernetes, nest, modify, azure_kusto

Additional context

[doraamgui]

bug report: sends log data from Fluent Bit to an OpenSearch Ingestion pipeline, pod crashed. https://docs.fluentbit.io/manual/pipeline/outputs/http

kubectl logs fluent-bit-zk276 -n fluent-bit --previous
Fluent Bit v4.0.1

• Copyright (C) 2015-2025 The Fluent Bit Authors
• Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
• https://fluentbit.io/

---

| | | | | | ___ () | / || _ |
| | | | _ ___ _ __ | |_ | |/ /| |_ __ / /| || |/' |
| | | | | | |/ _ \ ' | | | ___ \ | | \ \ / / /| || /| |
| | | | || | / | | | | | |/ / | | \ V /_ |\ |/ /
_| ||_,|_|| ||_| _/||_| _/ |()__/

[2025/04/28 06:50:04] [engine] caught signal (SIGSEGV)
#0 0x5580e7e43128 in prepare_destroy_conn_safe() at src/flb_upstream.c:549
#1 0x5580e7e43128 in flb_upstream_conn_release() at src/flb_upstream.c:895
#2 0x5580e7f33834 in http_post() at plugins/out_http/http.c:352
#3 0x5580e7f34b87 in cb_http_flush() at plugins/out_http/http.c:641
#4 0x5580e8497e66 in co_init() at lib/monkey/deps/flb_libco/amd64.c:117
#5 0xffffffffffffffff in ???() at ???:0

[OUTPUT]
Name http
Match *
Host fluentbit-pipeline-******.us-east-1.osis.amazonaws.com
Port 443
URI /log/ingest
Format json
aws_auth true
aws_region us-east-1
aws_service osis
Log_Level info
tls On
json_date_key time
json_date_format iso8601

reproduce issue, follow up this doc:https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-client-fluentbit.html

may i know this is the same bug? if it was also fixed in 4.0.2? and if it is better to use 4.0.0 version?