Skip to content

Commit 42e7392

Browse files
xliuqqxliuqq
authored
Extend CacheRuntime phase 2.2: support dataset secret mount options (#5810)
* support secret mount options Signed-off-by: xliuqq <xlzq1992@gmail.com> * fix openapi Signed-off-by: xliuqq <xlzq1992@gmail.com> * remove extra parameters Signed-off-by: xliuqq <xlzq1992@gmail.com> * improve transformEncryptOptionsToComponentVolumes implemetation Signed-off-by: xliuqq <xlzq1992@gmail.com> * not return error Signed-off-by: xliuqq <xlzq1992@gmail.com> * fix secret volume limit and nil panic Signed-off-by: xliuqq <xlzq1992@gmail.com> * fix nil panic Signed-off-by: xliuqq <xlzq1992@gmail.com> * component secret mount should be configurable per runtime implementation. Signed-off-by: xliuqq <xlzq1992@gmail.com> * fix openapi Signed-off-by: xliuqq <xlzq1992@gmail.com> --------- Signed-off-by: xliuqq <xlzq1992@gmail.com>
1 parent 4fca3ab commit 42e7392

25 files changed

Lines changed: 1676 additions & 138 deletions

‎api/v1alpha1/cacheruntimeclass_types.go‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -81,10 +81,6 @@ type ExecutionCommonEntry struct {
8181
TimeoutSeconds int32 `json:"timeout,omitempty"`
8282
}
8383

84-
// EncryptOptionComponentDependency defines the configuration for encrypt option dependency
85-
type EncryptOptionComponentDependency struct {
86-
}
87-
8884
// ExtraResourcesComponentDependency defines the extra resources configuration for component dependencies
8985
type ExtraResourcesComponentDependency struct {
9086
// ConfigMaps is a list of ConfigMaps in the same namespace to mount into the component
@@ -94,15 +90,23 @@ type ExtraResourcesComponentDependency struct {
9490

9591
// RuntimeComponentDependencies defines the dependencies required by a CacheRuntime component
9692
type RuntimeComponentDependencies struct {
97-
// EncryptOption is the configuration for encrypt option secret mount
93+
// SecretMount controls whether dataset encrypt-option secrets are mounted into this component pod.
94+
// Defaults to true for Master/Worker, false for Client unless explicitly enabled.
9895
// +optional
99-
EncryptOption *EncryptOptionComponentDependency `json:"encryptOption,omitempty"`
96+
SecretMount *SecretMountComponentDependency `json:"secretMount,omitempty"`
10097

10198
// ExtraResources specifies the usage of extra resources such as ConfigMaps
10299
// +optional
103100
ExtraResources *ExtraResourcesComponentDependency `json:"extraResources,omitempty"`
104101
}
105102

103+
// SecretMountComponentDependency defines the secret mount configuration for component dependencies
104+
type SecretMountComponentDependency struct {
105+
// Enabled indicates whether dataset encrypt-option secrets should be mounted into this component pod.
106+
// +optional
107+
Enabled bool `json:"enabled,omitempty"`
108+
}
109+
106110
// HeadlessRuntimeComponentService defines the configuration for headless service
107111
type HeadlessRuntimeComponentService struct {
108112
}

‎api/v1alpha1/openapi_generated.go‎

Lines changed: 25 additions & 16 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/v1alpha1/zz_generated.deepcopy.go‎

Lines changed: 50 additions & 18 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎charts/fluid/fluid/crds/data.fluid.io_cacheruntimeclasses.yaml‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -72,8 +72,6 @@ spec:
7272
properties:
7373
dependencies:
7474
properties:
75-
encryptOption:
76-
type: object
7775
extraResources:
7876
properties:
7977
configMaps:
@@ -86,6 +84,11 @@ spec:
8684
type: object
8785
type: array
8886
type: object
87+
secretMount:
88+
properties:
89+
enabled:
90+
type: boolean
91+
type: object
8992
type: object
9093
executionEntries:
9194
properties:
@@ -3489,8 +3492,6 @@ spec:
34893492
properties:
34903493
dependencies:
34913494
properties:
3492-
encryptOption:
3493-
type: object
34943495
extraResources:
34953496
properties:
34963497
configMaps:
@@ -3503,6 +3504,11 @@ spec:
35033504
type: object
35043505
type: array
35053506
type: object
3507+
secretMount:
3508+
properties:
3509+
enabled:
3510+
type: boolean
3511+
type: object
35063512
type: object
35073513
executionEntries:
35083514
properties:
@@ -6906,8 +6912,6 @@ spec:
69066912
properties:
69076913
dependencies:
69086914
properties:
6909-
encryptOption:
6910-
type: object
69116915
extraResources:
69126916
properties:
69136917
configMaps:
@@ -6920,6 +6924,11 @@ spec:
69206924
type: object
69216925
type: array
69226926
type: object
6927+
secretMount:
6928+
properties:
6929+
enabled:
6930+
type: boolean
6931+
type: object
69236932
type: object
69246933
executionEntries:
69256934
properties:

‎config/crd/bases/data.fluid.io_cacheruntimeclasses.yaml‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -72,8 +72,6 @@ spec:
7272
properties:
7373
dependencies:
7474
properties:
75-
encryptOption:
76-
type: object
7775
extraResources:
7876
properties:
7977
configMaps:
@@ -86,6 +84,11 @@ spec:
8684
type: object
8785
type: array
8886
type: object
87+
secretMount:
88+
properties:
89+
enabled:
90+
type: boolean
91+
type: object
8992
type: object
9093
executionEntries:
9194
properties:
@@ -3489,8 +3492,6 @@ spec:
34893492
properties:
34903493
dependencies:
34913494
properties:
3492-
encryptOption:
3493-
type: object
34943495
extraResources:
34953496
properties:
34963497
configMaps:
@@ -3503,6 +3504,11 @@ spec:
35033504
type: object
35043505
type: array
35053506
type: object
3507+
secretMount:
3508+
properties:
3509+
enabled:
3510+
type: boolean
3511+
type: object
35063512
type: object
35073513
executionEntries:
35083514
properties:
@@ -6906,8 +6912,6 @@ spec:
69066912
properties:
69076913
dependencies:
69086914
properties:
6909-
encryptOption:
6910-
type: object
69116915
extraResources:
69126916
properties:
69136917
configMaps:
@@ -6920,6 +6924,11 @@ spec:
69206924
type: object
69216925
type: array
69226926
type: object
6927+
secretMount:
6928+
properties:
6929+
enabled:
6930+
type: boolean
6931+
type: object
69236932
type: object
69246933
executionEntries:
69256934
properties:

‎docs/en/dev/generic_cache_runtime_integration.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,8 +81,7 @@ The component in Topology mainly contains the following content:
8181
| Options | Default options, will be overridden by user settings | |
8282
| Template | PodTemplateSpec native field | |
8383
| Service | Currently only supports Headless | |
84-
| Dependencies | EncryptOption | Whether this component needs Fluid to mount the access keys defined in Dataset for accessing data sources [Not supported in current version], using the keys defined in Dataset for access. |
85-
| | ExtraResources | Whether this component needs to mount additional ConfigMaps (the dependent ConfigMap information is defined in the ExtraResources field of CacheRuntimeClass). |
84+
| Dependencies | ExtraResources | Whether this component needs to mount additional ConfigMaps (the dependent ConfigMap information is defined in the ExtraResources field of CacheRuntimeClass). |
8685
| ExecutionEntries| MountUFS | For Master-Worker architecture, when Master is Ready, the underlying file system mount operation needs to be executed. |
8786
| ExecutionEntries| ReportSummary | How the cache system defines operations to obtain cache information metrics [Not supported in current version]. |
8887

@@ -260,7 +259,7 @@ spec:
260259
In cacheruntime, all control plane processes are handled by Fluid. However, as a data caching engine, when providing services, the entire cache system requires **topology**, **data source**, **authentication**, and **cache information**. Fluid will provide this information to components through configuration files based on different Component roles. The component's internal process is responsible for parsing this configuration to perform environment variable configuration, data engine configuration file generation, and other operations. After preparation is complete, the data engine process can be started. For specific parsing details, please refer to the table below:
261260

262261
* Taking the above resources as an example, the Config examples mounted by Master/Worker/Client and maintained by Fluid are as follows:
263-
262+
the `mounts`, `accessModes`, and `targetPath` fields in the JSON are all derived from the Dataset's Spec definition.
264263

265264
```json
266265
{
@@ -274,6 +273,10 @@ In cacheruntime, all control plane processes are handled by Fluid. However, as a
274273
"region_name": "us-east-1",
275274
"secret": "minioadmin"
276275
},
276+
"encryptOptions": {
277+
"access-key": "/etc/fluid/secrets/minio-secret/access-key",
278+
"secret-key": "/etc/fluid/secrets/minio-secret/secret-key"
279+
},
277280
"name": "minio",
278281
"path": "/minio"
279282
}

0 commit comments

Comments
 (0)