Skip to content

Commit 3a4dc15

Browse files
maltesandermaltesander
committed
use storage config for filebased authorization PVC
1 parent 2adad2c commit 3a4dc15

4 files changed

Lines changed: 122 additions & 39 deletions

File tree

‎deploy/helm/nifi-operator/crds/crds.yaml‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -602,6 +602,8 @@ spec:
602602
capacity: null
603603
databaseRepo:
604604
capacity: null
605+
fileBasedRepo:
606+
capacity: null
605607
flowfileRepo:
606608
capacity: null
607609
provenanceRepo:
@@ -745,6 +747,52 @@ spec:
745747
nullable: true
746748
type: string
747749
type: object
750+
fileBasedRepo:
751+
default:
752+
capacity: null
753+
description: |-
754+
Used as persistance for file-based authorization.
755+
Default size: 16MB
756+
properties:
757+
capacity:
758+
description: "Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and AsInt64() accessors.\n\nThe serialization format is:\n\n``` <quantity> ::= <signedNumber><suffix>\n\n\t(Note that <suffix> may be empty, from the \"\" case in <decimalSI>.)\n\n<digit> ::= 0 | 1 | ... | 9 <digits> ::= <digit> | <digit><digits> <number> ::= <digits> | <digits>.<digits> | <digits>. | .<digits> <sign> ::= \"+\" | \"-\" <signedNumber> ::= <number> | <sign><number> <suffix> ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI> ::= Ki | Mi | Gi | Ti | Pi | Ei\n\n\t(International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)\n\n<decimalSI> ::= m | \"\" | k | M | G | T | P | E\n\n\t(Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)\n\n<decimalExponent> ::= \"e\" <signedNumber> | \"E\" <signedNumber> ```\n\nNo matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities.\n\nWhen a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized.\n\nBefore serializing, Quantity will be put in \"canonical form\". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that:\n\n- No precision is lost - No fractional digits will be emitted - The exponent (or suffix) is as large as possible.\n\nThe sign will be omitted unless the number is negative.\n\nExamples:\n\n- 1.5 will be serialized as \"1500m\" - 1.5Gi will be serialized as \"1536Mi\"\n\nNote that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise.\n\nNon-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.)\n\nThis format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation."
759+
nullable: true
760+
type: string
761+
selectors:
762+
description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
763+
nullable: true
764+
properties:
765+
matchExpressions:
766+
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
767+
items:
768+
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
769+
properties:
770+
key:
771+
description: key is the label key that the selector applies to.
772+
type: string
773+
operator:
774+
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
775+
type: string
776+
values:
777+
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
778+
items:
779+
type: string
780+
type: array
781+
required:
782+
- key
783+
- operator
784+
type: object
785+
type: array
786+
matchLabels:
787+
additionalProperties:
788+
type: string
789+
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
790+
type: object
791+
type: object
792+
storageClass:
793+
nullable: true
794+
type: string
795+
type: object
748796
flowfileRepo:
749797
default:
750798
capacity: null
@@ -1154,6 +1202,8 @@ spec:
11541202
capacity: null
11551203
databaseRepo:
11561204
capacity: null
1205+
fileBasedRepo:
1206+
capacity: null
11571207
flowfileRepo:
11581208
capacity: null
11591209
provenanceRepo:
@@ -1297,6 +1347,52 @@ spec:
12971347
nullable: true
12981348
type: string
12991349
type: object
1350+
fileBasedRepo:
1351+
default:
1352+
capacity: null
1353+
description: |-
1354+
Used as persistance for file-based authorization.
1355+
Default size: 16MB
1356+
properties:
1357+
capacity:
1358+
description: "Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and AsInt64() accessors.\n\nThe serialization format is:\n\n``` <quantity> ::= <signedNumber><suffix>\n\n\t(Note that <suffix> may be empty, from the \"\" case in <decimalSI>.)\n\n<digit> ::= 0 | 1 | ... | 9 <digits> ::= <digit> | <digit><digits> <number> ::= <digits> | <digits>.<digits> | <digits>. | .<digits> <sign> ::= \"+\" | \"-\" <signedNumber> ::= <number> | <sign><number> <suffix> ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI> ::= Ki | Mi | Gi | Ti | Pi | Ei\n\n\t(International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)\n\n<decimalSI> ::= m | \"\" | k | M | G | T | P | E\n\n\t(Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)\n\n<decimalExponent> ::= \"e\" <signedNumber> | \"E\" <signedNumber> ```\n\nNo matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities.\n\nWhen a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized.\n\nBefore serializing, Quantity will be put in \"canonical form\". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that:\n\n- No precision is lost - No fractional digits will be emitted - The exponent (or suffix) is as large as possible.\n\nThe sign will be omitted unless the number is negative.\n\nExamples:\n\n- 1.5 will be serialized as \"1500m\" - 1.5Gi will be serialized as \"1536Mi\"\n\nNote that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise.\n\nNon-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.)\n\nThis format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation."
1359+
nullable: true
1360+
type: string
1361+
selectors:
1362+
description: A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
1363+
nullable: true
1364+
properties:
1365+
matchExpressions:
1366+
description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
1367+
items:
1368+
description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
1369+
properties:
1370+
key:
1371+
description: key is the label key that the selector applies to.
1372+
type: string
1373+
operator:
1374+
description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
1375+
type: string
1376+
values:
1377+
description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
1378+
items:
1379+
type: string
1380+
type: array
1381+
required:
1382+
- key
1383+
- operator
1384+
type: object
1385+
type: array
1386+
matchLabels:
1387+
additionalProperties:
1388+
type: string
1389+
description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
1390+
type: object
1391+
type: object
1392+
storageClass:
1393+
nullable: true
1394+
type: string
1395+
type: object
13001396
flowfileRepo:
13011397
default:
13021398
capacity: null

‎rust/operator-binary/src/controller.rs‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,8 @@ use crate::{
8585
APP_NAME, BALANCE_PORT, BALANCE_PORT_NAME, Container, HTTPS_PORT, HTTPS_PORT_NAME,
8686
METRICS_PORT, METRICS_PORT_NAME, NifiConfig, NifiConfigFragment, NifiNodeRoleConfig,
8787
NifiRole, NifiStatus, PROTOCOL_PORT, PROTOCOL_PORT_NAME, STACKABLE_LOG_CONFIG_DIR,
88-
STACKABLE_LOG_DIR, authentication::AuthenticationClassResolved, v1alpha1,
88+
STACKABLE_LOG_DIR, authentication::AuthenticationClassResolved,
89+
authorization::NifiAccessPolicyProvider, v1alpha1,
8990
},
9091
listener::{
9192
LISTENER_VOLUME_DIR, LISTENER_VOLUME_NAME, build_group_listener, build_group_listener_pvc,
@@ -1492,6 +1493,7 @@ fn get_volume_claim_templates(
14921493
&rolegroup_ref.role_group,
14931494
))
14941495
.context(LabelBuildSnafu)?;
1496+
14951497
// listener endpoints will use persistent volumes
14961498
// so that load balancers can hard-code the target addresses and
14971499
// that it is possible to connect to a consistent address
@@ -1503,7 +1505,16 @@ fn get_volume_claim_templates(
15031505
.context(ListenerConfigurationSnafu)?,
15041506
);
15051507

1506-
pvcs.extend(authorization_config.get_pvcs());
1508+
// Add file-based PVC if required
1509+
if let ResolvedNifiAuthorizationConfig::Standard {
1510+
access_policy_provider: NifiAccessPolicyProvider::FileBased { .. },
1511+
} = authorization_config
1512+
{
1513+
pvcs.push(merged_config.resources.storage.file_based_repo.build_pvc(
1514+
&NifiRepository::State.repository(),
1515+
Some(vec!["ReadWriteOnce"]),
1516+
))
1517+
}
15071518

15081519
Ok(pvcs)
15091520
}

‎rust/operator-binary/src/crd/mod.rs‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -442,6 +442,11 @@ impl NifiConfig {
442442
storage_class: None,
443443
selectors: None,
444444
},
445+
file_based_repo: PvcConfigFragment {
446+
capacity: Some(Quantity("16Mi".to_string())),
447+
storage_class: None,
448+
selectors: None,
449+
},
445450
},
446451
},
447452
affinity: get_affinity(cluster_name, role),
@@ -525,6 +530,11 @@ pub struct NifiStorageConfig {
525530
/// Default size: 1GB
526531
#[fragment_attrs(serde(default))]
527532
pub state_repo: PvcConfig,
533+
534+
/// Used as persistance for file-based authorization.
535+
/// Default size: 16MB
536+
#[fragment_attrs(serde(default))]
537+
pub file_based_repo: PvcConfig,
528538
}
529539

530540
#[derive(Clone, Debug, Deserialize, JsonSchema, PartialEq, Serialize)]

‎rust/operator-binary/src/security/authorization.rs‎

Lines changed: 3 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,13 @@
1-
use std::collections::BTreeMap;
2-
31
use indoc::{formatdoc, indoc};
42
use snafu::{ResultExt, Snafu};
53
use stackable_operator::{
64
builder::pod::volume::{SecretOperatorVolumeSourceBuilder, VolumeBuilder},
75
client::Client,
86
commons::opa::OpaConfig,
9-
k8s_openapi::{
10-
api::core::v1::{
11-
ConfigMap, ConfigMapKeySelector, EnvVar, EnvVarSource, PersistentVolumeClaim,
12-
PersistentVolumeClaimSpec, Volume, VolumeMount, VolumeResourceRequirements,
13-
},
14-
apimachinery::pkg::api::resource::Quantity,
7+
k8s_openapi::api::core::v1::{
8+
ConfigMap, ConfigMapKeySelector, EnvVar, EnvVarSource, Volume, VolumeMount,
159
},
16-
kube::{ResourceExt, api::ObjectMeta},
10+
kube::ResourceExt,
1711
};
1812

1913
use crate::{
@@ -258,34 +252,6 @@ impl ResolvedNifiAuthorizationConfig {
258252
Ok(volumes)
259253
}
260254

261-
pub fn get_pvcs(&self) -> Vec<PersistentVolumeClaim> {
262-
let mut pvcs = vec![];
263-
264-
if let ResolvedNifiAuthorizationConfig::Standard { .. } = self {
265-
pvcs.push(PersistentVolumeClaim {
266-
metadata: ObjectMeta {
267-
name: Some(FILE_BASED_MOUNT_NAME.to_owned()),
268-
..ObjectMeta::default()
269-
},
270-
spec: Some(PersistentVolumeClaimSpec {
271-
access_modes: Some(vec!["ReadWriteOnce".to_owned()]),
272-
resources: Some(VolumeResourceRequirements {
273-
requests: Some({
274-
let mut map = BTreeMap::new();
275-
map.insert("storage".to_string(), Quantity("16Mi".to_owned()));
276-
map
277-
}),
278-
..Default::default()
279-
}),
280-
..PersistentVolumeClaimSpec::default()
281-
}),
282-
..PersistentVolumeClaim::default()
283-
})
284-
};
285-
286-
pvcs
287-
}
288-
289255
pub fn has_opa_tls(&self) -> bool {
290256
matches!(
291257
self,

0 commit comments

Comments
 (0)