Commit b734f71
feat: remove product config (#976)
* build: switch to smooth-operator branch and vendor the java-properties writer
Patches operator-rs to the smooth-operator branch (matching trino/hdfs) as the
foundation for the v2 config_overrides adoption later in this series, and
vendors the Java-properties writer into config/writer (backed by the
java-properties crate, Apache-2.0) so ConfigMap rendering no longer goes through
product_config::writer. Repointed resource/configmap.rs. No behaviour change
(18 tests pass). Regenerated Cargo.nix/crate-hashes.json.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor: extract per-file kafka .properties builders
Splits server_properties_file into
controller/build/properties/{broker,controller}_properties builders (base map +
security settings + graceful-shutdown + user overrides), wired into
resource/configmap.rs by role. The property assembly was moved verbatim, so the
rendered broker.properties/controller.properties are unchanged (18 tests pass;
byte parity to be confirmed via the kuttl ConfigMap snapshot). Override input
stays BTreeMap<String,String>; no product-config removed yet (later increment).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* formatting
* refactor: move rolegroup ConfigMap build into controller/build/config_map
Relocates build_rolegroup_config_map (plus the jaas_config_file helper and its
Error enum) from resource/configmap.rs to controller/build/config_map.rs,
colocating the ConfigMap assembler with the per-file property builders and
matching the controller/build/config_map.rs layout in hdfs/airflow. Pure move +
repointed caller; no behaviour change (18 tests pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat: remove product-config; merge config/env overrides directly in validate
Replaces the product-config validation path with a ValidatedKafkaCluster that
carries, per role group, the merged config plus the config-file, jvm-security,
and env overrides resolved directly from the CRD (role <- role-group). Overrides
now use stackable_operator::v2::config_overrides::KeyValueConfigOverrides
(matching trino/hdfs); the v1 KeyValueOverridesProvider impls and the per-role
Configuration impls are removed, and KAFKA_CLUSTER_ID injection moves into the
override merge (collect_*_role_group_overrides). The dereferenced authorization
config is folded into the validated cluster.
Drops the product-config crate dependency (it remains transitive via
stackable-operator). The CRD gains `nullable: true` on configOverrides values
(v2 allows null to delete a key). Rendered .properties and env vars are
unchanged (18 tests pass; byte parity to be confirmed via kuttl). Regenerated
extra/crds.yaml and Cargo.nix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs: remove product-config CLI param + env var; gut config-spec; changelog
Removes the --product-config section from the commandline reference and the
PRODUCT_CONFIG section from the environment-variables reference (the flag/var is
now a no-op via the shared RunArguments), reduces the product-config
properties.yaml files to an empty shell (retained pending a later Helm config
refactor), and notes the product-config removal in the changelog.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* updated changelog
* refactor: consume the config-file writer from stackable-operator
Replace the vendored Java-properties writer
(rust/operator-binary/src/config/writer.rs) with
stackable_operator::v2::config_file_writer (moved there via operator-rs
#1217 on the smooth-operator branch). Kafka's copy was the java-only subset
of the canonical hdfs writer; the upstream module's additional to_hadoop_xml
simply goes unimported. Drop the now-unused java-properties dependency.
No behaviour change; rendered .properties output is byte-identical by
construction (same code, new home).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* pass validate cluster and rg rather than multiple parameters
* removed redundant parameter
* push opa connect string calc down to function
* push pod_descriptors calc down to build_rolegroup_config_map
* move resolution of metadata_manager and pod descriptors to validate stage
* add config map data explicitly
* use merge instead of extend with KafkaBrokerConfigOverrides
* updated changelog
* extend test comparison
* unnecessary let binding
* regenerate nix
* bump op-rs branch and fix clippy warning
* regenerate charts
* thread through name, namespace, uid via ObjectMeta
* refactor: introduce ValidatedClusterConfig
* refactor: add framework module and merging
* chore: regenerate charts
* fix: remove obsolete errors, dead enum variant, tighten visibility of constants
* refactor: move logging mod to controller, remove util mod
* refactor: move ValidatedCluster to controller/mod.rs; add ConfigFileName enum; remove raw KafkaCluster from build_configmap
* refactor: make better use of ValidatedCluster in property file builders
* Update rust/operator-binary/src/crd/mod.rs
Co-authored-by: maltesander <contact@maltesander.com>
* Apply suggestions from code review
Co-authored-by: maltesander <contact@maltesander.com>
* linting
* cleaned up comment
* refactor: switch to v2 JavaCommonConfig & remove framework module
* refactor: use v2 owerref, labels, RoleGroupName, cleanup
* fix(tests): reduce logging and resources rolgegroups length
* chore: regenerate
* fix: security properties defaults
* refactor: move resources to builder
* fix: consolidate merging
* refactor: share common sts functionality
* refactor: move ConfigFileName enum
* refactor: cleanup security properties
* refactor: adapt pdb to build instead of mutate
* refactor: use ValidatedCluster in statefulset
* refactor: adapt logging to v2, add vector test
* chore: fix fmt
* chore: bump dependencies
* refactor: use ConfigMapName, SecretName, ListenerName in CRD
* refactor: use v2 metrics service helper
* refactor: consolidate rbac resources
* refactor: use v2 Port
* fix: move to stackable-operator STACKABLE_LOG_DIR
* fix: remove hardcoded volume names
* refactor: remove more rolegroupref references.
* refactor: introduce ValidatedRoleConfig
* fix: remove v2 references in comments
* refactor: move security mode to controller, rename KafkaTlsSecurity -> ValidatedKafkaSecurity
* refactor: extract build logic from KafkaSecurity
* chore: bump dependencies
* refactor: use v2 RoleGroupConfig and optional replicas
* fix: remove upstream references
* fix: user ListenerName type
* fix: extract hardcoded constants
* test: improve unit test coverage
* revert: rolegroup length changes
* Apply suggestions from code review
Co-authored-by: maltesander <malte.sander.it@gmail.com>
* chore: bump stackable-operator 0.112.0, tokio 1.52, clap 4.6
* Update rust/operator-binary/src/controller/build/properties/product_logging/vector.yaml
Co-authored-by: Siegfried Weber <mail@siegfriedweber.net>
* bump anyhow to 1.0.103, linting
* regenerate nix
* make internal secret class name non-optional
* replace constant with caller-context container name
* regenerate charts
* changelog
* add discovery config map to tests
* move label-related functions out of controller into a separate module
* refactor: assemble Kafka container env vars via a single EnvVarSet
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Malte Sander <malte.sander.it@gmail.com>
Co-authored-by: maltesander <contact@maltesander.com>
Co-authored-by: Siegfried Weber <mail@siegfriedweber.net>1 parent f130ee3 commit b734f71
63 files changed
Lines changed: 6509 additions & 5958 deletions
File tree
- deploy
- config-spec
- helm/kafka-operator/configs
- docs/modules/kafka/pages/reference
- extra
- rust/operator-binary
- src
- config
- controller
- build
- properties
- product_logging
- resource
- crd
- role
- operations
- resource
- tests/templates/kuttl
- smoke-kraft
- smoke
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
22 | 27 | | |
23 | 28 | | |
24 | 29 | | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
28 | 33 | | |
| 34 | + | |
29 | 35 | | |
30 | 36 | | |
31 | 37 | | |
| |||
0 commit comments