Skip to content

Commit 33f4b34

Browse files
committed
chore!: Remove NiFi 1.x configuration fields
1 parent 5cb5bce commit 33f4b34

18 files changed

Lines changed: 53 additions & 988 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,13 @@ All notable changes to this project will be documented in this file.
2121
- Set `nifi.content.repository.archive.max.retention.period` to `3 days` (previously empty, which NiFi interprets as `Long.MAX_VALUE` and effectively disables time-based archive purge). Without a time-based ceiling, the content archive can grow to half the content PVC and accumulate millions of files, which makes the synchronous startup directory scan in `FileSystemRepository.initializeRepository` very slow. Users requiring a longer content-replay window can extend via `configOverrides`. The provenance audit trail is independent of this setting and unaffected ([#936]).
2222
- test: Bump vector-aggregator to 0.55.0, replace /graphql call with gRPC call ([#940]).
2323

24+
### Removed
25+
26+
- BREAKING: Clean up leftover NiFi 1.x code and configuration.
27+
Support for NiFi 1.x was already officially dropped in SDP 26.3, but some now-dead code and CRD fields remained.
28+
This removes the Prometheus reporting-task Job (and its `spec.clusterConfig.createReportingTaskJob` field), the pre-2.x non-rolling upgrade handling, the dedicated metrics port, and the sensitive-properties algorithms that were only supported on NiFi 1.x.
29+
`status.deployed_version` is retained even though it no longer drives the (now removed) non-rolling upgrade state machine, as we don't want a breaking change to the status just for this ([#954]).
30+
2431
### Fixed
2532

2633
- Fix broken link to the NiFi authorization usage guide in the `spec.clusterConfig.authorization` CRD doc (`usage-guide` -> `usage_guide`) ([#924]).
@@ -34,6 +41,7 @@ All notable changes to this project will be documented in this file.
3441
[#935]: https://github.com/stackabletech/nifi-operator/pull/935
3542
[#936]: https://github.com/stackabletech/nifi-operator/pull/936
3643
[#940]: https://github.com/stackabletech/nifi-operator/pull/940
44+
[#954]: https://github.com/stackabletech/nifi-operator/pull/954
3745

3846
## [26.3.0] - 2026-03-16
3947

deploy/helm/nifi-operator/templates/clusterrole-operator.yaml

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -81,17 +81,6 @@ rules:
8181
- list
8282
- patch
8383
- watch
84-
# Optional reporting-task Job (NiFi 1.x only). Applied via SSA and tracked for orphan cleanup.
85-
- apiGroups:
86-
- batch
87-
resources:
88-
- jobs
89-
verbs:
90-
- create
91-
- delete
92-
- get
93-
- list
94-
- patch
9584
# PodDisruptionBudget created per role. Applied via SSA and tracked for orphan cleanup.
9685
- apiGroups:
9786
- policy

docs/modules/nifi/pages/index.adoc

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,8 @@ Every role group is accessible through it's own Service, and there is a Service
3535

3636
== Dependencies
3737

38-
Apache NiFi 1.x depends on Apache ZooKeeper which you can run in Kubernetes with the xref:zookeeper:index.adoc[].
38+
NiFi builds its cluster quorum using Kubernetes by default and needs no additional dependencies.
39+
It can optionally use Apache ZooKeeper instead, which you can run in Kubernetes with the xref:zookeeper:index.adoc[].
3940

4041
== [[demos]]Demos
4142

docs/modules/nifi/pages/usage_guide/clustering.adoc

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,6 @@ CAUTION: The cluster backend of an existing cluster should never be changed. Oth
1414
[#backend-kubernetes]
1515
== Kubernetes
1616

17-
NOTE: The Kubernetes provider is only supported by Apache NiFi 2.0 or newer. When using NiFi 1.x, use the xref:#backend-zookeeper[] backend instead.
18-
1917
The Kubernetes backend is used by default (unless the xref:#backend-zookeeper[] backend is configured), and stores all state in Kubernetes objects, in the same namespace as the `NifiCluster` object.
2018

2119
It takes no configuration.

docs/modules/nifi/pages/usage_guide/monitoring.adoc

Lines changed: 0 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,7 @@
11
= Monitoring
22
:description: The Stackable Operator for Apache NiFi automatically configures NiFi to export Prometheus metrics.
3-
:k8s-job: https://kubernetes.io/docs/concepts/workloads/controllers/job/
4-
:k8s-network-policies: https://kubernetes.io/docs/concepts/services-networking/network-policies/
53
:prometheus-operator: https://prometheus-operator.dev/
64

7-
In November 2024, Apache NiFi released a new major version https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version2.0.0[`2.0.0`,window=_blank].
8-
9-
The NiFi `2.0.0` release changed the way of exposing Prometheus metrics significantly.
10-
The following steps explain on how to expose Metrics in NiFi versions `1.x.x` and `2.x.x`.
11-
12-
== Configure metrics in NiFi `1.x.x`
13-
14-
For NiFi versions `1.x.x`, the operator automatically configures NiFi to export Prometheus metrics.
15-
This is done by creating a {k8s-job}[Job,window=_blank] that connects to NiFi and configures a https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-prometheus-nar/1.26.0/org.apache.nifi.reporting.prometheus.PrometheusReportingTask/index.html[Prometheus Reporting Task,window=_blank].
16-
17-
IMPORTANT: Network access from the Job to NiFi is required.
18-
If you are running a Kubernetes with restrictive {k8s-network-policies}[NetworkPolicies,window=_blank], make sure to allow access from the Job to NiFi.
19-
20-
See xref:operators:monitoring.adoc[] for more details.
21-
22-
== Disabling create-reporting-task Job
23-
24-
It can be helpful to disable the Job, e.g. when you configOverride an authentication mechanism, which the Job currently cannot use to authenticate against NiFi.
25-
26-
To achieve this use the following configuration:
27-
28-
[source,yaml]
29-
----
30-
spec:
31-
clusterConfig:
32-
createReportingTaskJob:
33-
enabled: false
34-
----
35-
365
== Configure metrics in NiFi `2.x.x`
376

387
The Prometheus Reporting Task was removed in NiFi `2.x.x` in https://issues.apache.org/jira/browse/NIFI-13507[NIFI-13507,window=_blank].

docs/modules/nifi/pages/usage_guide/overrides.adoc

Lines changed: 0 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -96,25 +96,6 @@ Pod overrides allow you to configure any attributes that can be configured on a
9696

9797
Read the xref:concepts:overrides.adoc#pod-overrides[Pod overrides concepts page] to learn more.
9898

99-
=== Pod overrides on create-reporting-task Job
100-
101-
In addition to podOverrides on the created StatefulSet we also support podOverrides on the created Kubernetes Job, which enables the export of Prometheus metrics within NiFi.
102-
103-
[source,yaml]
104-
----
105-
spec:
106-
clusterConfig:
107-
createReportingTaskJob:
108-
# enabled: false # You can also turn off the Job entirely
109-
podOverrides: # podOverrides as usual
110-
spec:
111-
tolerations:
112-
- key: "key1"
113-
operator: "Equal"
114-
value: "value1"
115-
effect: "NoSchedule"
116-
----
117-
11899
== JVM argument overrides
119100

120101
Stackable operators automatically determine the set of needed JVM arguments, such as memory settings or trust- and keystores.

docs/modules/nifi/pages/usage_guide/updating.adoc

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -17,16 +17,4 @@ spec:
1717

1818
<1> Change the NiFi version here
1919

20-
[WARNING]
21-
====
22-
NiFi clusters cannot be upgraded or downgraded in a rolling fashion due to a limitation in NiFi prior to version 2.
23-
24-
When upgrading between NiFi 1 versions or from NiFi 1 to NiFi 2, any change to the NiFi version in the CRD triggers a full cluster restart with brief downtime.
25-
However, the Stackable image version can be updated in a rolling manner, provided the NiFi version remains unchanged.
26-
27-
Since NiFi version 2, rolling upgrades are supported.
28-
====
29-
30-
== NiFi 2.0.0
31-
32-
Before you can upgrade to `2.0.0` you https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance[need to update] to at least version 1.27.x!
20+
Rolling upgrades are supported, so changing the NiFi version in the CRD updates the cluster without downtime.

extra/crds.yaml

Lines changed: 1 addition & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -139,33 +139,6 @@ spec:
139139
- accessPolicyProvider
140140
type: object
141141
type: object
142-
createReportingTaskJob:
143-
default:
144-
enabled: true
145-
podOverrides: {}
146-
description: |-
147-
This section creates a `create-reporting-task` Kubernetes Job, which enables the export of
148-
Prometheus metrics within NiFi.
149-
properties:
150-
enabled:
151-
default: true
152-
description: |-
153-
Whether the Kubernetes Job should be created, defaults to true. It can be helpful to disable
154-
the Job, e.g. when you configOverride an authentication mechanism, which the Job currently
155-
can't use to authenticate against NiFi.
156-
type: boolean
157-
podOverrides:
158-
default: {}
159-
description: |-
160-
Here you can define a
161-
[PodTemplateSpec](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#podtemplatespec-v1-core)
162-
to override any property that can be set on the Pod of the create-reporting-task Kubernetes Job.
163-
Read the
164-
[Pod overrides documentation](https://docs.stackable.tech/home/nightly/concepts/overrides#pod-overrides)
165-
for more information.
166-
type: object
167-
x-kubernetes-preserve-unknown-fields: true
168-
type: object
169142
customComponentsGitSync:
170143
default: []
171144
description: |-
@@ -347,26 +320,11 @@ spec:
347320
`nifiPbkdf2AesGcm256` (the default value),
348321
`nifiArgon2AesGcm256`,
349322
350-
The following algorithms are deprecated and will be removed in future versions:
351-
352-
`nifiArgon2AesGcm128`,
353-
`nifiBcryptAesGcm128`,
354-
`nifiBcryptAesGcm256`,
355-
`nifiPbkdf2AesGcm128`,
356-
`nifiScryptAesGcm128`,
357-
`nifiScryptAesGcm256`.
358-
359323
Learn more about the specifics of the algorithm parameters in the
360324
[NiFi documentation](https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#property-encryption-algorithms).
361325
enum:
362326
- nifiPbkdf2AesGcm256
363327
- nifiArgon2AesGcm256
364-
- nifiBcryptAesGcm128
365-
- nifiBcryptAesGcm256
366-
- nifiPbkdf2AesGcm128
367-
- nifiArgon2AesGcm128
368-
- nifiScryptAesGcm128
369-
- nifiScryptAesGcm256
370328
- null
371329
nullable: true
372330
type: string
@@ -413,8 +371,7 @@ spec:
413371
When using the [Stackable operator for Apache ZooKeeper](https://docs.stackable.tech/home/nightly/zookeeper/)
414372
to deploy a ZooKeeper cluster, this will simply be the name of your ZookeeperCluster resource.
415373
416-
The Kubernetes provider will be used if this field is unset. Kubernetes is only supported for NiFi 2.x and newer,
417-
NiFi 1.x requires ZooKeeper.
374+
The Kubernetes provider will be used if this field is unset.
418375
type: string
419376
required:
420377
- authentication

rust/operator-binary/src/config/mod.rs

Lines changed: 5 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ use std::{
55

66
use jvm::build_merged_jvm_config;
77
use product_config::{ProductConfigManager, types::PropertyNameKind};
8-
use snafu::{ResultExt, Snafu, ensure};
8+
use snafu::{ResultExt, Snafu};
99
use stackable_operator::{
1010
commons::resources::Resources,
1111
crd::git_sync,
@@ -20,7 +20,6 @@ use strum::{Display, EnumIter};
2020
use crate::{
2121
crd::{
2222
HTTPS_PORT, NifiConfig, NifiRole, NifiRoleType, NifiStorageConfig, PROTOCOL_PORT,
23-
sensitive_properties,
2423
v1alpha1::{self, NifiClusteringBackend},
2524
},
2625
operations::graceful_shutdown::graceful_shutdown_config_properties,
@@ -101,14 +100,6 @@ pub enum Error {
101100

102101
#[snafu(display("failed to generate OIDC config"))]
103102
GenerateOidcConfig { source: oidc::Error },
104-
105-
#[snafu(display(
106-
"NiFi 1.x requires ZooKeeper (hint: upgrade to NiFi 2.x or set .spec.clusterConfig.zookeeperConfigMapName)"
107-
))]
108-
Nifi1RequiresZookeeper,
109-
110-
#[snafu(display("failed to configure sensitive properties"))]
111-
ConfigureSensitiveProperties { source: sensitive_properties::Error },
112103
}
113104

114105
/// Create the NiFi bootstrap.conf
@@ -156,35 +147,14 @@ pub fn build_nifi_properties(
156147
proxy_hosts: &str,
157148
auth_config: &NifiAuthenticationConfig,
158149
overrides: BTreeMap<String, String>,
159-
product_version: &str,
160150
git_sync_resources: &git_sync::v1alpha2::GitSyncResources,
161151
) -> Result<String, Error> {
162-
// TODO: Remove once we dropped support for all NiFi 1.x versions
163-
let is_nifi_1 = product_version.starts_with("1.");
164-
165152
let mut properties = BTreeMap::new();
166153
// Core Properties
167-
// According to https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance#MigrationGuidance-Migratingto2.0.0-M1
168-
// The nifi.flow.configuration.file property in nifi.properties must be changed to reference
169-
// "flow.json.gz" instead of "flow.xml.gz"
170-
// TODO: Remove once we dropped support for all 1.x.x versions
171-
// TODO(malte): In order to use CLI tools like: ./bin/nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256
172-
// we have to set both "nifi.flow.configuration.file" and "nifi.flow.configuration.json.file" in NiFi 1.x.x.
173-
if is_nifi_1 {
174-
properties.insert(
175-
"nifi.flow.configuration.file".to_string(),
176-
NifiRepository::Database.mount_path() + "/flow.xml.gz",
177-
);
178-
properties.insert(
179-
"nifi.flow.configuration.json.file".to_string(),
180-
NifiRepository::Database.mount_path() + "/flow.json.gz",
181-
);
182-
} else {
183-
properties.insert(
184-
"nifi.flow.configuration.file".to_string(),
185-
NifiRepository::Database.mount_path() + "/flow.json.gz",
186-
);
187-
}
154+
properties.insert(
155+
"nifi.flow.configuration.file".to_string(),
156+
NifiRepository::Database.mount_path() + "/flow.json.gz",
157+
);
188158

189159
properties.insert(
190160
"nifi.flow.configuration.archive.enabled".to_string(),
@@ -535,10 +505,6 @@ pub fn build_nifi_properties(
535505
.clone()
536506
.unwrap_or_default();
537507

538-
sensitive_properties_algorithm
539-
.check_for_nifi_version(spec.image.product_version())
540-
.context(ConfigureSensitivePropertiesSnafu)?;
541-
542508
properties.insert(
543509
"nifi.sensitive.props.algorithm".to_string(),
544510
sensitive_properties_algorithm.to_string(),
@@ -635,8 +601,6 @@ pub fn build_nifi_properties(
635601
}
636602

637603
v1alpha1::NifiClusteringBackend::Kubernetes {} => {
638-
ensure!(!is_nifi_1, Nifi1RequiresZookeeperSnafu);
639-
640604
properties.insert(
641605
"nifi.cluster.leader.election.implementation".to_string(),
642606
"KubernetesLeaderElectionManager".to_string(),
@@ -653,9 +617,6 @@ pub fn build_nifi_properties(
653617
//####################
654618
// Custom components #
655619
//####################
656-
// NiFi 1.x does not support Python components and the Python configuration below is just
657-
// ignored.
658-
659620
// The command used to launch Python.
660621
// This property must be set to enable Python-based processors.
661622
properties.insert("nifi.python.command".to_string(), "python3".to_string());

0 commit comments

Comments
 (0)