Allow reloading the config without a restart.

[marcofranssen]

Is your feature request related to a problem? Please describe.

In the fluent-operator we use CRDs that generate the eventual fluentbit configuration. To make fluentbit processes restart we are building a wrapper image that adds this logic for fluentbit to restart the process.

Describe the solution you'd like

Would be great if fluent-bit has this logic built-in so we don't have to build this wrapper image for the operator.

• https://github.com/fluent/fluent-operator/blob/master/cmd/fluent-watcher/fluentbit/Dockerfile
• https://github.com/fluent/fluent-operator/blob/master/cmd/fluent-watcher/fluentbit/main.go

Unfortunately our team doesn't have the C skills to make that happen.

Describe alternatives you've considered

Continue maintaining this small wrapper and continue managing releases.

Additional context

We want to simplify operator releases to make it easier to keep on track with fluent-bit releases.

See here what we want to remove:

• https://github.com/fluent/fluent-operator/tree/master/cmd/fluent-watcher/fluentbit
• https://github.com/fluent/fluent-operator/blob/master/.github/workflows/build-fluentbit-image.yaml
• https://github.com/fluent/fluent-operator/blob/master/.github/workflows/bump-fluent-bit-version.yaml

So we only have to update the version for the helm chart and we don't have to release another image wrapping fluent-bit with this small go binary to reload the process on config changes.

[marcofranssen]

FYI: This has been requested in less detailed form earlier here fluent/fluent-operator#1854

[patrick-stephens]

I'm not sure I understand the request here, hot-reload is already implemented and supported by the helm chart too: https://github.com/fluent/helm-charts/blob/ab1ef141602a16edd6d61656be847c14195329d6/charts/fluent-bit/values.yaml#L522-L526

Hot reload can be achieved by signal or web API: https://docs.fluentbit.io/manual/administration/hot-reload

What else is required specifically?

[marcofranssen]

Idea is to not have to signal it, but fluenbit process itself just notices config changes, validates them and restarts the process. This also has advantage if fluent-operator would generate invalid config it wouldn't reload etc. Basically similar like nginx uses to do.

Saves us sidecars in the chart +. it works for both fluent-operator and fluentbit charts without having the need for another sidecar container or the fluent-operator way of wrapping.

[patrick-stephens]

Right so the request is more for an approach to detect change in configuration automatically which can then trigger hot-reload maybe, e.g. you hash the file like helm suggests or similar.

[marcofranssen]

Remember we run an operator, where CRDs result in the config to update. So it is really the configmap change that has to be detected by the fleuntbit pod/binary itself. Nothing out of process, nothing external.

[patrick-stephens]

Yeah although it will be the actual config file that is produced from whatever combination of CRs/mounts/configmaps/etc. that is used rather than monitoring anything in K8S directly. Monitoring the config file is probably a more generic approach as well for more platforms/configuration approaches although it may be limited by the filesystem used.