You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
=Templates for Common Files in Operator Repositories
1
+
#Templates for Common Files in Operator Repositories
2
2
3
3
Stackable develops and maintains a growing number of operators for open source software.
4
4
5
5
The high level structure of the repository is consistent across all repositories, which in the past has led to repetitive maintenance tasks in many repositories even for small changes.
6
6
7
7
This repository is intended to help with these changes by keeping common files as templates and offering tooling to roll these out to all repositories that are under management by this tool.
8
8
9
-
==Structure of this repository
9
+
##Structure of this repository
10
10
11
-
=== GitHub Actions
11
+
###GitHub Actions
12
12
13
13
The definition files for GitHub actions that in turn execute the playbooks doing the actual work are kept in `.github`
14
14
15
-
=== Playbook
15
+
###Playbook
16
16
17
17
`playbook` contains the ansible playbook which is executed to perform the needed changes.
18
18
19
-
=== Templates
19
+
###Templates
20
20
21
21
Everything under the top level folder `template` is replicated to the target repositories.
22
22
The folder and file structure under `template` is considered as source structure which will be synchronized across all repositories in the following manner:
23
23
24
-
* Regular files and folders are simply copied
25
-
* Files with an extension of `.j2` will be processed as jinja2 templates and copied to the target repositories with the `.j2` extension removed. The default jinja2 variable delimiter has been replaced with `{[ }]` since a lot of the template files contain `{{ }}` and caused issues.
26
-
* File and directory names which contain `\[[product]]` will have this substituted for the value of the `product_string` variable. For example: "stackable-\[[product]]-operator.service" would become `stackable-kafka-operator.service`.
24
+
- Regular files and folders are simply copied
25
+
- Files with an extension of `.j2` will be processed as jinja2 templates and copied to the target repositories with the `.j2` extension removed. The default jinja2 variable delimiter has been replaced with `{[ }]` since a lot of the template files contain `{{ }}` and caused issues.
26
+
- File and directory names which contain `\[[product]]` will have this substituted for the value of the `product_string` variable. For example: "stackable-\[[product]]-operator.service" would become `stackable-kafka-operator.service`.
27
27
28
28
To remove files or directories that already exist in the target repositories these need to be configured in `repositories.yaml` under the `retired_files` key.
29
29
30
30
Anything that is listed here will be deleted from the target repositories.
31
31
32
-
NOTE: Deletion is the last step that is performed, so if there is an overlap between files existing in the template folder and this setting, the files would not be rolled out, since they'd get deleted before creating the pull request.
32
+
> [!NOTE]
33
+
> Deletion is the last step that is performed, so if there is an overlap between files existing in the template folder and this setting, the files would not be rolled out, since they'd get deleted before creating the pull request.
33
34
34
-
=== Configuration
35
+
###Configuration
35
36
36
37
All user-facing configuration is kept in `repositories.yaml`.
37
38
@@ -41,62 +42,51 @@ So in principle it is as simple as pushing any commit to `main` which will trigg
41
42
42
43
Target repositories are configured in `repositories.yaml` in the following form:
43
44
44
-
----
45
+
```yaml
45
46
- name: kafka-operator
46
47
url: stackabletech/kafka-operator.git
47
48
product_string: kafka
48
49
pretty_string: Kafka
49
-
----
50
-
51
-
|===
52
-
|Field |Description
53
-
54
-
|name
55
-
|This is only used internally to name working directories and the like.
56
-
57
-
|url
58
-
|The github repository for this operator. Need to be in the form of `<org>/<repo>.git`
59
-
60
-
|product_string
61
-
| A lower case string to use in config files, file names and the like. Should not contain whitespaces. This can sometimes be a shortened version of the full name, for example for Open Policy Agent this would be "opa"
62
-
63
-
|pretty_string
64
-
| The actual name of the product, including whitespaces and proper capitalization. This is intended to be used in doc or man files or similar things.
65
-
66
-
|include_productconfig
67
-
| Whether to include files from the `deploy/config-spec` folder into the os package.
68
-
69
-
*Default*: true
70
-
|===
71
-
50
+
```
51
+
52
+
| Field | Description |
53
+
| ----- | ----------- |
54
+
| `name` | This is only used internally to name working directories and the like. |
55
+
| `url` | The github repository for this operator. Need to be in the form of `<org>/<repo>.git`. |
56
+
| `product_string` | A lower case string to use in config files, file names and the like. Should not contain whitespaces. This can sometimes be a shortened version of the full name, for example for Open Policy Agent this would be "opa" |
57
+
| `pretty_string` | The actual name of the product, including whitespaces and proper capitalization. This is intended to be used in doc or man files or similar things. |
58
+
| `config.include_productconfig` | Whether to include files from the `deploy/config-spec` folder into the os package. *Default*: true |
59
+
| `config.has_product` | Indicates that the operator manages a product. This is particularly useful to differentiate between core and product operators. *Default*: true |
60
+
| `config.run_as` | Whether to run the operator as a default Deployment, or something custom. |
72
61
73
62
These are the only variables currently being used on the playbooks, but can be extended easily as more are needed.
74
63
75
-
NOTE: If a new variable is introduced, it needs to be added to all repository objects!
64
+
> [!NOTE]
65
+
> If a new variable is introduced, it needs to be added to all repository objects!
76
66
77
67
Additional settings can be found in `playbook/group_vars/all`, but these are not intended to be freely changed and should be treated with care.
78
68
79
-
80
-
== Making changes to the template
69
+
## Making changes to the template
81
70
82
71
If you want to make a change that should be rolled out to all operators, make the change in the `template` directory.
83
72
Consult the section above to learn more about the structure of the template.
84
73
85
-
=== Test changes locally
74
+
### Test changes locally
86
75
87
-
1. Run the `test.sh` script. To limit the run to a single operator, add `--extra-vars "shard_repositories=['nifi-operator']"` (or your choice of operator).
76
+
1. Run the `test.sh` script. To limit the run to a single operator, add `--extra-vars "shard_repositories#['nifi-operator']"` (or your choice of operator).
88
77
It will automatically delete and recreate a `work` directory.
89
78
2. The changes can be examined with `git status`.
90
79
When the pull request is later merged into the `main` branch then pull requests with these changes will be created automatically.
91
80
3. Depending on the change, it makes sense to run the integration tests for all changed operators.
92
81
If the tests are not run in this stage and if there is even just one integration test failing in the subsequential generated pull requests then the operator-templating must be adapted which creates again pull requests for all operators.
93
82
Changes in the GitHub workflow actions cannot be tested until finally merged.
94
83
95
-
== Deploying changes
84
+
## Deploying changes
96
85
97
86
Changes are rolled out via GitHub actions.
98
87
99
-
=== Authentication
88
+
### Authentication
89
+
100
90
Since this tool needs to authenticate itself in order to push commits it needs credentials of a user.
101
91
This is currently solved via a personal access token that needs to be provided as a repository (or org) secret with the name `GH_ACCESS_TOKEN`.
102
92
@@ -106,9 +96,11 @@ The personal access token needs to have the following permissions for this to wo
106
96
- org:read
107
97
- workflow
108
98
109
-
== Limitations
99
+
## Limitations
100
+
110
101
There is currently no synchronization with existing PRs on the target repositories whatsoever. A new pull request will be created for every commit made to this repository.
111
102
112
103
To update a PR that was created via this tool, it will have to be closed and necessary changes pushed here, which will result in a new PR.
113
104
114
-
WARNING: The Helm Chart files that are rolled out by the templates in their current form do not include a ClusterRole object which may be needed for this to work with RBAC.
105
+
> [!WARNING]
106
+
> The Helm Chart files that are rolled out by the templates in their current form do not include a ClusterRole object which may be needed for this to work with RBAC.
0 commit comments