Skip to content

Commit 97154f3

Browse files
maltesanderadwk67
andauthored
Fix and document changing sensitive properties algorithm (#799)
* move sensitive properties to own module * remove import * remove return type * add comment * fix flow.configuration.file properties for 1.x.x * add docs * fix warn / error message to be consistent * reference nifi pod instead of node * regenerate charts * adapted changelog * fixes * Apply suggestions from code review Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com> --------- Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com>
1 parent b985f03 commit 97154f3

6 files changed

Lines changed: 209 additions & 81 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ All notable changes to this project will be documented in this file.
3030

3131
- Use `json` file extension for log files ([#774]).
3232
- Fix a bug where changes to ConfigMaps that are referenced in the NifiCluster spec didn't trigger a reconciliation ([#772]).
33+
- The operator now emits a warning (1.x.x) or errors out (2.x.x) if a deprecated or unsupported sensitive properties algorithm is used ([#799]).
3334

3435
### Removed
3536

@@ -45,6 +46,7 @@ All notable changes to this project will be documented in this file.
4546
[#785]: https://github.com/stackabletech/nifi-operator/pull/785
4647
[#787]: https://github.com/stackabletech/nifi-operator/pull/787
4748
[#789]: https://github.com/stackabletech/nifi-operator/pull/789
49+
[#799]: https://github.com/stackabletech/nifi-operator/pull/799
4850

4951
## [25.3.0] - 2025-03-21
5052

deploy/helm/nifi-operator/crds/crds.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -163,12 +163,12 @@ spec:
163163
164164
Learn more about the specifics of the algorithm parameters in the [NiFi documentation](https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#property-encryption-algorithms).
165165
enum:
166-
- nifiArgon2AesGcm128
166+
- nifiPbkdf2AesGcm256
167167
- nifiArgon2AesGcm256
168168
- nifiBcryptAesGcm128
169169
- nifiBcryptAesGcm256
170170
- nifiPbkdf2AesGcm128
171-
- nifiPbkdf2AesGcm256
171+
- nifiArgon2AesGcm128
172172
- nifiScryptAesGcm128
173173
- nifiScryptAesGcm256
174174
nullable: true

docs/modules/nifi/pages/usage_guide/security.adoc

Lines changed: 61 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -374,7 +374,7 @@ If `autoGenerate` is false and no Secret with the given name in `keySecret` is f
374374
The `algorithm` property configures the encryption algorithm used to encrypt the sensitive properties.
375375
Consult the {crd-docs}/nifi.stackable.tech/nificluster/v1alpha1/#spec-clusterConfig-sensitiveProperties-algorithm[reference documentation {external-link-icon}^] for a list of supported algorithms.
376376

377-
=== Autogenerated key example
377+
=== Autogenerated sensitive properties key
378378

379379
Let the operator generate a Secret with the name `nifi-sensitive-property-key`:
380380

@@ -385,7 +385,7 @@ sensitiveProperties:
385385
autoGenerate: true
386386
----
387387

388-
=== Custom key and encryption algorithm example
388+
=== Custom sensitive properties key and sensitive properties algorithm
389389

390390
Create the Secret yourself:
391391

@@ -399,7 +399,7 @@ stringData:
399399
nifiSensitivePropsKey: my-encryption-key
400400
----
401401

402-
Configure the Secret and a different encryption algrithm:
402+
Configure the Secret and a different sensitive properties algorithm:
403403

404404
[source,yaml]
405405
----
@@ -408,6 +408,64 @@ sensitiveProperties:
408408
algorithm: nifiArgon2AesGcm256
409409
----
410410

411+
=== Upgrading sensitive properties algorithm
412+
413+
WARNING: Please make sure to backup any flows before upgrading the sensitive properties algorithm!
414+
415+
The sensitive properties algorithm can be changed via the `nifi.sh` CLI tool as described in the https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#updating-the-sensitive-properties-algorithm[Apache NiFi documentation].
416+
417+
Assuming that you deployed a cluster with this:
418+
419+
[source,yaml]
420+
----
421+
sensitiveProperties:
422+
keySecret: nifi-sensitive-property-key
423+
algorithm: nifiArgon2AesGcm256
424+
----
425+
426+
If you want to change the algorithm to `nifiPbkdf2AesGcm256`, you have to run the following command on each NiFi Pod:
427+
428+
[source,bash]
429+
----
430+
/stackable/nifi/bin/nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256
431+
----
432+
433+
NOTE: Be careful with the notation used in the NiFiCluster `nifiPbkdf2AesGcm256` versus the setting in the NiFi CLI `NIFI_PBKDF2_AES_GCM_256`!
434+
435+
Alternatively, you can use this shell script to automatically execute this in each pod via `kubectl` (make sure to edit the `NAMESPACE` and `STATEFULSET_NAME` accordingly):
436+
437+
[source,bash]
438+
----
439+
NAMESPACE="default"
440+
STATEFULSET_NAME="simple-nifi-node-default"
441+
COMMAND="/stackable/nifi/bin/nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256"
442+
443+
kubectl get pods -n "$NAMESPACE" --no-headers -o custom-columns=":metadata.name" | grep "^$STATEFULSET_NAME" | \
444+
while read pod; do
445+
echo "Running on $pod"
446+
kubectl exec -n "$NAMESPACE" -c "nifi" "$pod" -- sh -c "$COMMAND"
447+
done
448+
----
449+
450+
Afterwards, update your NiFiCluster to the required algorithm `nifiPbkdf2AesGcm256`:
451+
452+
[source,yaml]
453+
----
454+
sensitiveProperties:
455+
keySecret: nifi-sensitive-property-key
456+
algorithm: nifiPbkdf2AesGcm256
457+
----
458+
459+
Finally, apply the updated NiFiCluster and restart / delete the StatefulSet:
460+
461+
[source,bash]
462+
----
463+
kubectl apply -n "$NAMESPACE" -f <nifi-yaml>
464+
kubectl delete -n "$NAMESPACE" statefulsets ${STATEFULSET_NAME}
465+
----
466+
467+
468+
411469
[#host-header-check]
412470
== Host Header Check
413471
NiFi checks the host header of incoming requests and rejects them if they are passing through a proxy that is not on an allow-list configured in the `nifi.web.proxy.host` property.

rust/operator-binary/src/config/mod.rs

Lines changed: 29 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ use strum::{Display, EnumIter};
2020
use crate::{
2121
crd::{
2222
HTTPS_PORT, NifiConfig, NifiConfigFragment, NifiRole, NifiStorageConfig, PROTOCOL_PORT,
23-
v1alpha1::{self, NifiClusteringBackend},
23+
sensitive_properties, v1alpha1, v1alpha1::NifiClusteringBackend,
2424
},
2525
operations::graceful_shutdown::graceful_shutdown_config_properties,
2626
security::{
@@ -101,6 +101,9 @@ pub enum Error {
101101
"NiFi 1.x requires ZooKeeper (hint: upgrade to NiFi 2.x or set .spec.clusterConfig.zookeeperConfigMapName)"
102102
))]
103103
Nifi1RequiresZookeeper,
104+
105+
#[snafu(display("failed to configure sensitive properties"))]
106+
ConfigureSensitiveProperties { source: sensitive_properties::Error },
104107
}
105108

106109
/// Create the NiFi bootstrap.conf
@@ -156,15 +159,25 @@ pub fn build_nifi_properties(
156159
// According to https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance#MigrationGuidance-Migratingto2.0.0-M1
157160
// The nifi.flow.configuration.file property in nifi.properties must be changed to reference
158161
// "flow.json.gz" instead of "flow.xml.gz"
159-
let flow_file_name = if is_nifi_1 {
160-
"flow.xml.gz"
162+
// TODO: Remove once we dropped support for all 1.x.x versions
163+
// TODO(malte): In order to use CLI tools like: ./bin/nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256
164+
// we have to set both "nifi.flow.configuration.file" and "nifi.flow.configuration.json.file" in NiFi 1.x.x.
165+
if is_nifi_1 {
166+
properties.insert(
167+
"nifi.flow.configuration.file".to_string(),
168+
NifiRepository::Database.mount_path() + "/flow.xml.gz",
169+
);
170+
properties.insert(
171+
"nifi.flow.configuration.json.file".to_string(),
172+
NifiRepository::Database.mount_path() + "/flow.json.gz",
173+
);
161174
} else {
162-
"flow.json.gz"
163-
};
164-
properties.insert(
165-
"nifi.flow.configuration.file".to_string(),
166-
NifiRepository::Database.mount_path() + "/" + flow_file_name,
167-
);
175+
properties.insert(
176+
"nifi.flow.configuration.file".to_string(),
177+
NifiRepository::Database.mount_path() + "/flow.json.gz",
178+
);
179+
}
180+
168181
properties.insert(
169182
"nifi.flow.configuration.archive.enabled".to_string(),
170183
"true".to_string(),
@@ -483,15 +496,20 @@ pub fn build_nifi_properties(
483496
"".to_string(),
484497
);
485498

486-
let algorithm = &spec
499+
let sensitive_properties_algorithm = &spec
487500
.cluster_config
488501
.sensitive_properties
489502
.algorithm
490503
.clone()
491504
.unwrap_or_default();
505+
506+
sensitive_properties_algorithm
507+
.check_for_nifi_version(spec.image.product_version())
508+
.context(ConfigureSensitivePropertiesSnafu)?;
509+
492510
properties.insert(
493511
"nifi.sensitive.props.algorithm".to_string(),
494-
algorithm.to_string(),
512+
sensitive_properties_algorithm.to_string(),
495513
);
496514

497515
// key and trust store

rust/operator-binary/src/crd/mod.rs

Lines changed: 2 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
pub mod affinity;
22
pub mod authentication;
3+
pub mod sensitive_properties;
34
pub mod tls;
45

56
use std::collections::BTreeMap;
67

78
use affinity::get_affinity;
9+
use sensitive_properties::NifiSensitivePropertiesConfig;
810
use serde::{Deserialize, Serialize};
911
use snafu::{OptionExt, ResultExt, Snafu};
1012
use stackable_operator::{
@@ -356,71 +358,6 @@ impl CurrentlySupportedListenerClasses {
356358
}
357359
}
358360

359-
/// These settings configure the encryption of sensitive properties in NiFi processors.
360-
/// NiFi supports encrypting sensitive properties in processors as they are written to disk.
361-
/// You can configure the encryption algorithm and the key to use.
362-
/// You can also let the operator generate an encryption key for you.
363-
#[derive(Clone, Debug, Default, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
364-
#[serde(rename_all = "camelCase")]
365-
pub struct NifiSensitivePropertiesConfig {
366-
/// A reference to a Secret. The Secret needs to contain a key `nifiSensitivePropsKey`.
367-
/// If `autoGenerate` is false and this object is missing, the Operator will raise an error.
368-
/// The encryption key needs to be at least 12 characters long.
369-
pub key_secret: String,
370-
371-
/// Whether to generate the `keySecret` if it is missing.
372-
/// Defaults to `false`.
373-
#[serde(default)]
374-
pub auto_generate: bool,
375-
376-
/// This is setting the `nifi.sensitive.props.algorithm` property in NiFi.
377-
/// This setting configures the encryption algorithm to use to encrypt sensitive properties.
378-
/// Valid values are:
379-
///
380-
/// `nifiPbkdf2AesGcm256` (the default value),
381-
/// `nifiArgon2AesGcm256`,
382-
///
383-
/// The following algorithms are deprecated and will be removed in future versions:
384-
///
385-
/// `nifiArgon2AesGcm128`,
386-
/// `nifiBcryptAesGcm128`,
387-
/// `nifiBcryptAesGcm256`,
388-
/// `nifiPbkdf2AesGcm128`,
389-
/// `nifiScryptAesGcm128`,
390-
/// `nifiScryptAesGcm256`.
391-
///
392-
/// Learn more about the specifics of the algorithm parameters in the
393-
/// [NiFi documentation](https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#property-encryption-algorithms).
394-
pub algorithm: Option<NifiSensitiveKeyAlgorithm>,
395-
}
396-
397-
#[derive(strum::Display, Clone, Debug, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
398-
#[serde(rename_all = "camelCase")]
399-
pub enum NifiSensitiveKeyAlgorithm {
400-
#[strum(serialize = "NIFI_ARGON2_AES_GCM_128")]
401-
NifiArgon2AesGcm128,
402-
#[strum(serialize = "NIFI_ARGON2_AES_GCM_256")]
403-
NifiArgon2AesGcm256,
404-
#[strum(serialize = "NIFI_BCRYPT_AES_GCM_128")]
405-
NifiBcryptAesGcm128,
406-
#[strum(serialize = "NIFI_BCRYPT_AES_GCM_256")]
407-
NifiBcryptAesGcm256,
408-
#[strum(serialize = "NIFI_PBKDF2_AES_GCM_128")]
409-
NifiPbkdf2AesGcm128,
410-
#[strum(serialize = "NIFI_PBKDF2_AES_GCM_256")]
411-
NifiPbkdf2AesGcm256,
412-
#[strum(serialize = "NIFI_SCRYPT_AES_GCM_128")]
413-
NifiScryptAesGcm128,
414-
#[strum(serialize = "NIFI_SCRYPT_AES_GCM_256")]
415-
NifiScryptAesGcm256,
416-
}
417-
418-
impl Default for NifiSensitiveKeyAlgorithm {
419-
fn default() -> Self {
420-
Self::NifiArgon2AesGcm256
421-
}
422-
}
423-
424361
#[derive(strum::Display, Clone, Debug, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
425362
#[serde(rename_all = "camelCase")]
426363
pub enum StoreType {
Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
use serde::{Deserialize, Serialize};
2+
use snafu::Snafu;
3+
use stackable_operator::schemars::{self, JsonSchema};
4+
5+
#[derive(Snafu, Debug)]
6+
pub enum Error {
7+
#[snafu(display(
8+
"The sensitive properties algorithm '{algorithm}' is not supported in NiFi 2.X.X. Please see https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#updating-the-sensitive-properties-algorithm on how to upgrade the algorithm."
9+
))]
10+
UnsupportedSensitivePropertiesAlgorithm { algorithm: String },
11+
}
12+
13+
/// These settings configure the encryption of sensitive properties in NiFi processors.
14+
/// NiFi supports encrypting sensitive properties in processors as they are written to disk.
15+
/// You can configure the encryption algorithm and the key to use.
16+
/// You can also let the operator generate an encryption key for you.
17+
#[derive(Clone, Debug, Default, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
18+
#[serde(rename_all = "camelCase")]
19+
pub struct NifiSensitivePropertiesConfig {
20+
/// A reference to a Secret. The Secret needs to contain a key `nifiSensitivePropsKey`.
21+
/// If `autoGenerate` is false and this object is missing, the Operator will raise an error.
22+
/// The encryption key needs to be at least 12 characters long.
23+
pub key_secret: String,
24+
25+
/// Whether to generate the `keySecret` if it is missing.
26+
/// Defaults to `false`.
27+
#[serde(default)]
28+
pub auto_generate: bool,
29+
30+
/// This is setting the `nifi.sensitive.props.algorithm` property in NiFi.
31+
/// This setting configures the encryption algorithm to use to encrypt sensitive properties.
32+
/// Valid values are:
33+
///
34+
/// `nifiPbkdf2AesGcm256` (the default value),
35+
/// `nifiArgon2AesGcm256`,
36+
///
37+
/// The following algorithms are deprecated and will be removed in future versions:
38+
///
39+
/// `nifiArgon2AesGcm128`,
40+
/// `nifiBcryptAesGcm128`,
41+
/// `nifiBcryptAesGcm256`,
42+
/// `nifiPbkdf2AesGcm128`,
43+
/// `nifiScryptAesGcm128`,
44+
/// `nifiScryptAesGcm256`.
45+
///
46+
/// Learn more about the specifics of the algorithm parameters in the
47+
/// [NiFi documentation](https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#property-encryption-algorithms).
48+
pub algorithm: Option<NifiSensitiveKeyAlgorithm>,
49+
}
50+
51+
#[derive(strum::Display, Clone, Debug, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
52+
#[serde(rename_all = "camelCase")]
53+
pub enum NifiSensitiveKeyAlgorithm {
54+
// supported in v2
55+
#[strum(serialize = "NIFI_PBKDF2_AES_GCM_256")]
56+
NifiPbkdf2AesGcm256,
57+
// supported in v2
58+
#[strum(serialize = "NIFI_ARGON2_AES_GCM_256")]
59+
NifiArgon2AesGcm256,
60+
// Deprecated in v1 -> can be removed when 1.x.x is no longer supported
61+
#[strum(serialize = "NIFI_BCRYPT_AES_GCM_128")]
62+
NifiBcryptAesGcm128,
63+
// Deprecated in v1 -> can be removed when 1.x.x is no longer supported
64+
#[strum(serialize = "NIFI_BCRYPT_AES_GCM_256")]
65+
NifiBcryptAesGcm256,
66+
// Deprecated in v1 -> can be removed when 1.x.x is no longer supported
67+
#[strum(serialize = "NIFI_PBKDF2_AES_GCM_128")]
68+
NifiPbkdf2AesGcm128,
69+
// Deprecated in v1 -> can be removed when 1.x.x is no longer supported
70+
#[strum(serialize = "NIFI_ARGON2_AES_GCM_128")]
71+
NifiArgon2AesGcm128,
72+
// Deprecated in v1 -> can be removed when 1.x.x is no longer supported
73+
#[strum(serialize = "NIFI_SCRYPT_AES_GCM_128")]
74+
NifiScryptAesGcm128,
75+
// Deprecated in v1 -> can be removed when 1.x.x is no longer supported
76+
#[strum(serialize = "NIFI_SCRYPT_AES_GCM_256")]
77+
NifiScryptAesGcm256,
78+
}
79+
80+
impl NifiSensitiveKeyAlgorithm {
81+
/// Checks if the used encryption algorithm is supported or deprecated.
82+
/// Will warn for deprecation and error out for missing support.
83+
pub fn check_for_nifi_version(&self, product_version: &str) -> Result<(), Error> {
84+
let algorithm = self.to_string();
85+
86+
match self {
87+
// Allowed and supported in NiFi 1.x.x and 2.x.x
88+
NifiSensitiveKeyAlgorithm::NifiPbkdf2AesGcm256
89+
| NifiSensitiveKeyAlgorithm::NifiArgon2AesGcm256 => {}
90+
// All others are deprecated in 1.x.x and removed in 2.x.x
91+
// see https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#property-encryption-algorithms
92+
_ => {
93+
if product_version.starts_with("1.") {
94+
tracing::warn!(
95+
"You are using a deprecated sensitive properties algorithm '{algorithm}'. Please update to '{pbkd}' or '{argon}'.",
96+
pbkd = NifiSensitiveKeyAlgorithm::NifiPbkdf2AesGcm256,
97+
argon = NifiSensitiveKeyAlgorithm::NifiArgon2AesGcm256
98+
)
99+
} else {
100+
return Err(Error::UnsupportedSensitivePropertiesAlgorithm { algorithm });
101+
}
102+
}
103+
}
104+
105+
Ok(())
106+
}
107+
}
108+
109+
impl Default for NifiSensitiveKeyAlgorithm {
110+
fn default() -> Self {
111+
Self::NifiArgon2AesGcm256
112+
}
113+
}

0 commit comments

Comments
 (0)