Commit 6c71589
feat: remove product config & clean up (#1041)
* chore: Switch stackable-operator to smooth-operator branch
Point stackable-operator at the smooth-operator branch (via [patch]) to gain
access to the v2:: framework modules, in preparation for removing the
product-config dependency. Bump dependencies (cargo update) to satisfy the
branch's snafu >=0.9.1 requirement.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: Vendor Java properties writer into framework module
Add a vendored framework module (mirroring stackable_operator::v2) and move the
Java .properties writer into framework/writer.rs, backed by the same
java-properties crate as product_config::writer so the rendered output is
byte-identical. Swap the zoo.cfg / security.properties rendering in
zk_controller from product_config::writer to the local writer.
This removes the first product-config touch point. Properties only; the Hadoop
XML writer from the hdfs-operator source is not needed here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: Vendor role_utils framework module
Add framework/role_utils.rs mirroring stackable_operator::v2::role_utils from
the smooth-operator branch (as trino-operator does): RoleGroupConfig plus
with_validated_config, which merges default <- role <- rolegroup config
fragments and validates them via FromFragment.
Gated with allow(dead_code) until the reconciler is switched to produce a
ValidatedCluster in the following commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: Remove product-config, build config from typed ValidatedCluster
Replace the product-config-based validation pipeline with a typed, framework-
based one mirroring trino-operator:
- validate() now produces a ValidatedCluster { name, image, cluster_config,
role_group_configs } via framework::role_utils::with_validated_config, instead
of the product-config PropertyNameKind map. server.<myid> quorum entries are
precomputed into cluster_config so the build step never needs the CRD.
- Add zk_controller/build/config_map.rs which builds zoo.cfg and
security.properties directly from the typed config, referencing the
ZookeeperCluster only for the owner reference. The operator defaults that
product-config used to inject from deploy/config-spec/properties.yaml
(admin.serverPort, clientPort, dataDir, initLimit, syncLimit, tickTime,
metricsProvider.*, networkaddress.cache.*) are now seeded here, byte-identical
to before (pinned by the kuttl ConfigMap snapshot).
- Drop the product_config Configuration impl from the CRD; add a manual Merge
impl for ZookeeperConfigOverrides (CRD schema unchanged) and Ord for
ZookeeperRole.
- Minimal consumer changes: the StatefulSet derives MYID_OFFSET/ZOOCFGDIR from
the merged config plus envOverrides; the metrics Service takes a resolved port.
- Remove ProductConfigManager from main.rs/Ctx and the product-config dependency
from both Cargo.toml. It remains only as a transitive dependency of
stackable-operator itself.
Unit tests now exercise the product-config-free path and assert the seeded
defaults and security.properties byte-for-byte. The generated CRD is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: Empty out config-spec/properties.yaml
product-config no longer reads this file, but it is kept (empty) because the
build/packaging process still expects it (it is COPYd into the operator image).
Mirrors trino-operator. To be removed entirely in a later cleanup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: remove product config references from cli & env
* refactor: Split config_map into per-file property builders
Restructure zk_controller/build to mirror trino-operator and hdfs-operator:
- build/properties/mod.rs holds the ConfigFileName enum and shared helpers
(apply_overrides, into_optional_values).
- build/properties/zoo_cfg.rs and security_properties.rs each render one file;
metrics_http_port moves alongside the zoo.cfg builder.
- build/properties/logging.rs takes over the logback/log4j and vector.yaml
rendering, replacing the top-level product_logging.rs module (now removed).
- build/config_map.rs becomes a thin orchestrator that assembles the data map
via ConfigFileName and the property/logging builders, then .data(data).build().
Pure restructuring: no behavior change. The generated CRD is unchanged and all
16 unit tests (which render the full ConfigMap) still pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: Non-optional config overrides + ConfigFileName for file names
Mirror trino-operator and hdfs-operator:
- Switch ZookeeperConfigOverrides to stackable_operator::v2::config_overrides::
KeyValueConfigOverrides with non-optional fields and a derived Merge impl,
replacing the manual Merge impl and the KeyValueOverridesProvider/
get_key_value_overrides indirection. The zoo.cfg/security.properties builders
now read rg.config_overrides.<field> directly and resolve via resolved_overrides
(None values are dropped).
- Drop the hard-coded ZOOKEEPER_PROPERTIES_FILE / JVM_SECURITY_PROPERTIES_FILE
crd constants. The ConfigMap file names now come solely from the ConfigFileName
enum; jvm.rs keeps a local security.properties const for the JVM system property
(as trino does).
CRD regenerated: the configOverrides fields become non-nullable objects
(default {}) with nullable string values, matching trino/hdfs. Backwards-
compatible for existing string-valued overrides. All 16 tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: run linters & regenerate
* refactor: Move discovery ConfigMap builder into zk_controller/build
Mirror trino-operator and hdfs-operator by relocating discovery.rs to
zk_controller/build/discovery.rs. The zk_controller `build` module is now
pub(crate) so the znode controller can reach the shared builder; both
controllers build discovery ConfigMaps for a ZookeeperCluster.
Pure move: no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: Simplify build_discovery_configmap params; rename znode validated struct
- build_discovery_configmap drops the redundant `zk` parameter (it was only used
for an error ObjectRef; the owner reference comes from `owner`) and renames
`resolved_product_image` to `image`. It keeps `image` + `zookeeper_security` as
explicit shared params, so both controllers can call it without a shared trait
or a full ValidatedCluster.
- Rename the znode controller's ValidatedInputs to ValidatedZnode and its
resolved_product_image field to image, for naming consistency with
ValidatedCluster.
No behavior change. Build, clippy, the 16 unit tests, and the generated CRD are
all unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: Move logging constants out of crd into their consumers
These constants are no longer used in crd/mod.rs. Following hdfs-operator
(logging constants live in the logging builder) and trino-operator
(MAX_PREPARE_LOG_FILE_SIZE lives in the controller):
- LOGBACK_CONFIG_FILE, LOG4J_CONFIG_FILE, ZOOKEEPER_LOG_FILE and
MAX_ZK_LOG_FILES_SIZE move to zk_controller/build/properties/logging.rs.
- MAX_PREPARE_LOG_FILE_SIZE (the prepare init container) moves to zk_controller.rs.
- config/jvm.rs keeps its own local log-config-file consts (as it already does for
security.properties) to avoid coupling config to the controller build module.
No behavior change. Build, clippy, the 16 unit tests, and the generated CRD are
all unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: clippy
* fix: regenerate hashes
* chore: adapt changelog
* refactor: consume the config-file writer from stackable-operator
Replace the vendored Java-properties writer
(rust/operator-binary/src/framework/writer.rs) with
stackable_operator::v2::config_file_writer (moved there via operator-rs
#1217 on the smooth-operator branch). ZooKeeper's copy was a java-only
subset of the canonical hdfs writer; the upstream module's additional
to_hadoop_xml simply goes unimported. Drop the now-unused java-properties
dependency. The framework module now only contains role_utils.
No behaviour change; rendered .properties output is byte-identical by
construction (same code, new home).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor: add namespace, ui to ValidatedCluster, cleanup raw ZkCluster references
* refactor: use non option properties writer
* fix: remove allow dead code annotation
* refactor: improve discovery configmap
* fix: remove obsolete errors & cleanup
* deps: bumo operator-rs branch
* refactor: switch to EnvVarSet
* fix: update hashes
* use validated cluster in places where raw was used
* refactor: move jvm config to controller/build and remove filename constants
* refactor: use v2 JavaCommonConfig and remove local framework module
* fix(test): shorten rolegroup names
* refactor: move to v2 types for ResourceNames, labels, ownerrefs. Reduce RoleGroupRef usage.
* chore: regenerate
* fix(znode): use v2 types
* refactor: move k8s resources to controller/build/resource
* refactor: switch to v2 cluster resources, cleanup constants
* refactor: move to logging v2. Remove log4j (<=3.7 which we do not support). Add vector tests.
* refactor: add clusteroperation & object overrides to ValidatedCluster / ValidatedZnode
* chore: bump dependencies
* refactor: use v2 ConfigMapName, SecretName, ListenerName
* refactor: make cluster resources for znode infailible
* refactor: use v2 Port, sa name helper
* refactor: consolidate object metadata + identity helpers
* refactor: carry PDB config via ValidatedRoleConfig, drop last raw-CRD read
* refactor: render logback from validated log-config choice
* refactor: adopt upstream RoleGroupConfig, carry optional replicas
* chore: bump dependencies
* refactor: compute zoo.cfg server addresses in build, not validate
* Revert "fix(test): shorten rolegroup names"
This reverts commit 9e5c83f.
* fix: remove v2 / upstream comment refs
* refactor: use VolumeMount type
* fix: improve metrics port helper
* fix: consolidate constants
* test: add unit tests for conditional settings
* bump op-rs to 0.112.0
* fix: replace promethues annotations and label with v2 helper
* bump anyhow to 1.0.103 due to RUSTSEC advisory
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Andrew Kenworthy <andrew.kenworthy@stackable.tech>1 parent 3750916 commit 6c71589
49 files changed
Lines changed: 4822 additions & 5413 deletions
File tree
- deploy
- config-spec
- helm/zookeeper-operator/configs
- docs/modules/zookeeper/pages/reference
- extra
- rust/operator-binary
- src
- config
- crd
- operations
- zk_controller
- build
- properties
- product_logging
- resource
- znode_controller
- tests/templates/kuttl/smoke
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
24 | 26 | | |
25 | 27 | | |
| 28 | + | |
26 | 29 | | |
27 | 30 | | |
28 | 31 | | |
| |||
0 commit comments