Add settings.kubernetes.feature-gates to configure kubelet feature gates

[BrunoChauvet]

Summary

Add a settings.kubernetes.feature-gates setting that allows users to enable or disable individual kubelet feature gates via Bottlerocket's settings API.

Motivation

Bottlerocket currently has no mechanism to configure kubelet feature gates. On AL2023, users can patch /etc/kubernetes/kubelet/kubelet-config.json directly in userData bash scripts, but Bottlerocket's immutable filesystem prevents this approach.

A concrete driving use case: the AWS Data on EKS best practices guide recommends enabling MemoryQoS=true on Kubernetes analytics nodes to activate cgroup v2 memory.high throttling and reduce OOM kills on memory-intensive workloads (e.g. Apache Spark). Bottlerocket already runs with cgroup v2 enabled — the only missing piece is the kubelet feature gate, which remains alpha and must be explicitly set.

This creates an inconsistency: an AWS-published best practices document recommends a setting that cannot be applied on Bottlerocket, another AWS project, without building a custom image.

Related issues requesting similar or overlapping functionality:

• kubelet: add credential provider feature-gate #1702 — kubelet: add credential provider feature-gate (open since 2021)
• Configure kubelet parameters via BR API #2481 — Configure kubelet parameters via BR API
• Need API Setting to allow modify kubelet config for Json logging format #4056 — Need API setting to allow modify kubelet config

Proposed API

[settings.kubernetes.feature-gates]
MemoryQoS = true

Individual gates stored as prefixed sub-keys (one key per gate), following the same pattern used by settings.kubernetes.eviction-hard. The value type would be bool, matching KubeletConfiguration.featureGates (map[string]bool in the Kubernetes API).

Implementation scope

This spans three repositories:

Repo	Change
bottlerocket-os/bottlerocket	Datastore migration (AddPrefixesMigration)
bottlerocket-os/bottlerocket-settings-sdk	Add feature_gates: Option<HashMap<String, bool>> to KubernetesSettingsV1
bottlerocket-os/bottlerocket-core-kit	Render featureGates block in the kubelet-config template

Open questions

• Is bool the right value type, or should it be String for forward compatibility?
• Should we validate gate names against a known allowlist, or accept arbitrary strings (unbounded map)?
• Is there a preferred way to coordinate changes across the three repos?