Skip to content

Commit cc3b6b0

Browse files
maxstackmaxstack
committed
Move deployment/deploy-openbao-kayobe-automation.yml to secret-store/secret-store-deploy-openbao-runners.yml
This is inline with the playbooks installing vault/openbao for the seed and overcloud. Fix deployment issues with GitHub runners: * determine network interface to use, docker0 or lo * protocol missing from api_uri * general fixes so playbook's more inline with other secret-store-deploy playbooks
1 parent 396402a commit cc3b6b0

File tree

3 files changed

+156
-67
lines changed

3 files changed

+156
-67
lines changed

‎doc/source/configuration/ci-cd.rst‎

Lines changed: 120 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -140,41 +140,116 @@ Runner Deployment
140140
10. Repeat the above steps for each environment you intend to deploy runners within.
141141
You can share the fine-grained access token between environments.
142142

143+
OpenBao Deployment
144+
------------------
145+
146+
OpenBao is optional when deploying GitHub workflows but recommended, instead of storing numerous secrets within GitHub, including the SSH private key, just the OpenBao token will be stored instead.
147+
148+
OpenBao must be installed on the same host as the runners.
149+
If you have multiple environments that each have their own runners then OpenBao must be installed on each host.
150+
However, if you have a single host that is shared between environments then OpenBao only needs to be installed once which can be achieved by running the following playbook.
151+
152+
.. code-block:: bash
153+
154+
kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store/secret-store-deploy-openbao-runners.yml
155+
156+
.. note::
157+
158+
This playbook configures OpenBao to listen on the Docker network bridge `docker0` which will be accessible to the containers running the Kayobe Automation.
159+
If running OpenBao between environments care should be taken to ensure OpenBao listens on an interface which is accessible to all runners.
160+
161+
.. note::
162+
163+
If you are sharing OpenBao between environments then you will need to rerun the playbook under each environment to ensure that the correct secrets are available to the runners.
164+
You may use :code:`--tags add_secrets` to skip the deployment within other environments.
165+
For this to work you will need to copy :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json` from the deployment environment to the other environments in addition to copying the host definition of the gitlab runner add network IP.
166+
167+
Once the above playbook has been applied you need to grab the root token from :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json` as you will need this for the GitHub `BAO_TOKEN` secret.
168+
169+
This would also be an opportune time to encrypt the :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json` to protect the contents.
170+
171+
.. code-block:: bash
172+
173+
ansible-vault encrypt $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json --vault-password-file ~/.vault.password
174+
143175
Workflow Deployment
144176
-------------------
145177

146178
1. Edit :code:`$KAYOBE_CONFIG_PATH/inventory/group_vars/github-writer/writer.yml` in the base configuration making the appropriate changes to your deployments specific needs. See documentation for `stackhpc.kayobe_workflows.github <https://github.com/stackhpc/ansible-collection-kayobe-workflows/tree/main/roles/github>`__.
147179

180+
If using OpenBao then make sure to configure the :code:`github_checkout_hook` and :code:`github_kayobe_arguments` variables to include the appropriate steps to retrieve secrets from OpenBao and export them as environment variables for use within the workflows.
181+
182+
.. code-block:: yaml
183+
184+
github_checkout_hook: !unsafe |
185+
- name: Import secrets via OpenBao
186+
id: secrets
187+
uses: hashicorp/vault-action@v3.0.0
188+
with:
189+
# Access OpenBao via the docker bridge
190+
url: http://172.17.0.1:8200
191+
token: ${{ secrets[format('{0}_BAO_TOKEN', inputs.kayobe_environment || matrix.environment)] }}
192+
secrets: |
193+
kayobe-automation/${{ inputs.kayobe_environment || matrix.environment }} kayobe_vault_password | KAYOBE_VAULT_PASSWORD ;
194+
kayobe-automation/${{ inputs.kayobe_environment || matrix.environment }} kayobe_automation_ssh_private_key | KAYOBE_AUTOMATION_SSH_PRIVATE_KEY ;
195+
kayobe-automation/${{ inputs.kayobe_environment || matrix.environment }} kayobe_tempest_openrc | TEMPEST_OPENRC ;
196+
197+
github_kayobe_arguments:
198+
KAYOBE_VAULT_PASSWORD: !unsafe "${{ env.KAYOBE_VAULT_PASSWORD }}"
199+
KAYOBE_AUTOMATION_SSH_PRIVATE_KEY: !unsafe "${{ env.KAYOBE_AUTOMATION_SSH_PRIVATE_KEY }}"
200+
TEMPEST_OPENRC: !unsafe "${{ env.TEMPEST_OPENRC }}"
201+
148202
2. Run :code:`kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/deployment/write-github-workflows.yml`
149203

150204
3. Add all required secrets and variables to repository either via the GitHub UI or GitHub CLI (may require repository owner)
151205

152-
+----------------------------------------------------------------------------------+
153-
| Secrets |
154-
+===================================+==============================================+
155-
| Single Environment | Multiple Environments |
156-
+-----------------------------------+----------------------------------------------+
157-
| KAYOBE_AUTOMATION_SSH_PRIVATE_KEY | <ENV_NAME>_KAYOBE_AUTOMATION_SSH_PRIVATE_KEY |
158-
+-----------------------------------+----------------------------------------------+
159-
| KAYOBE_VAULT_PASSWORD | <ENV_NAME>_KAYOBE_VAULT_PASSWORD |
160-
+-----------------------------------+----------------------------------------------+
161-
| REGISTRY_PASSWORD | <ENV_NAME>_REGISTRY_PASSWORD |
162-
+-----------------------------------+----------------------------------------------+
163-
| TEMPEST_OPENRC | <ENV_NAME>_TEMPEST_OPENRC |
164-
+-----------------------------------+----------------------------------------------+
165-
166-
+-------------------------------------------------------+
167-
| VARIABLES |
168-
+====================+==================================+
169-
| Single Environment | Multiple Environments |
170-
+--------------------+----------------------------------+
171-
| REGISTRY_URL | <ENV_NAME>_REGISTRY_URL |
172-
+--------------------+----------------------------------+
173-
| REGISTRY_USERNAME | <ENV_NAME>_REGISTRY_USERNAME |
174-
+--------------------+----------------------------------+
175-
176-
Note the above tables shows the secrets and variables one may need to add to GitHub for a successful deployment.
177-
When adding secrets and variables make sure to adhere to the naming standards and ensure the :code:`<ENV_NAME>` is replaced with all supported kayobe environments in uppercase.
206+
3.1 The following secrets and variables are always required for the :code:`pulp registry` to be used within the workflows
207+
208+
+----------------------------------------------------------------------------------+
209+
| Secrets |
210+
+===================================+==============================================+
211+
| Single Environment | Multiple Environments |
212+
+-----------------------------------+----------------------------------------------+
213+
| REGISTRY_PASSWORD | <ENV_NAME>_REGISTRY_PASSWORD |
214+
+-----------------------------------+----------------------------------------------+
215+
216+
+-------------------------------------------------------+
217+
| VARIABLES |
218+
+====================+==================================+
219+
| Single Environment | Multiple Environments |
220+
+--------------------+----------------------------------+
221+
| REGISTRY_URL | <ENV_NAME>_REGISTRY_URL |
222+
+--------------------+----------------------------------+
223+
| REGISTRY_USERNAME | <ENV_NAME>_REGISTRY_USERNAME |
224+
+--------------------+----------------------------------+
225+
226+
3.2 **If OpenBao is being used** then the following secret is required to allow workflows access to OpenBao's secrets
227+
228+
This is where you use the root token previously grabbed from :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json`.
229+
230+
+-----------------------------------------------+
231+
| Secrets |
232+
+====================+==========================+
233+
| Single Environment | Multiple Environments |
234+
+--------------------+--------------------------+
235+
| BAO_TOKEN | <ENV_NAME>_BAO_TOKEN |
236+
+--------------------+--------------------------+
237+
238+
3.3 **If OpenBao is not being used** then the following secrets are required
239+
240+
+----------------------------------------------------------------------------------+
241+
| Secrets |
242+
+===================================+==============================================+
243+
| Single Environment | Multiple Environments |
244+
+-----------------------------------+----------------------------------------------+
245+
| KAYOBE_AUTOMATION_SSH_PRIVATE_KEY | <ENV_NAME>_KAYOBE_AUTOMATION_SSH_PRIVATE_KEY |
246+
+-----------------------------------+----------------------------------------------+
247+
| REGISTRY_PASSWORD | <ENV_NAME>_REGISTRY_PASSWORD |
248+
+-----------------------------------+----------------------------------------------+
249+
| TEMPEST_OPENRC | <ENV_NAME>_TEMPEST_OPENRC |
250+
+-----------------------------------+----------------------------------------------+
251+
252+
When adding secrets and variables make sure to adhere to the naming standards and ensure the :code:`<ENV_NAME>` is replaced with all supported kayobe environments in uppercase.
178253

179254
4. Commit and push all newly generated workflows found under :code:`.github/workflows`
180255

@@ -209,19 +284,19 @@ Runner Deployment
209284
Whether the host is in an infra-vm or not it will need access to the :code:`admin_network` or :code:`provision_oc_network`, :code:`public_network` and the :code:`pulp registry` on the seed.
210285
The steps will assume that an infra-vm will be used for the purpose of hosting the runners.
211286

212-
2. Edit the environment's :code:`${KAYOBE_CONFIG_PATH}/environments/${KAYOBE_ENVIRONMENT}/inventory/hosts` to define the host(s) that will host the runners.
287+
2. Edit the environment's :code:`$KAYOBE_CONFIG_PATH/environments/${KAYOBE_ENVIRONMENT}/inventory/hosts` to define the host(s) that will host the runners.
213288

214289
.. code-block:: ini
215290
216291
[gitlab-runners]
217292
gitlab-runner-01
218293
219-
4. Provide all the relevant Kayobe :code:`group_vars` for :code:`gitlab-runners` under :code:`${KAYOBE_CONFIG_PATH}/environments/${KAYOBE_ENVIRONMENT}/inventory/group_vars/gitlab-runners`
294+
4. Provide all the relevant Kayobe :code:`group_vars` for :code:`gitlab-runners` under :code:`$KAYOBE_CONFIG_PATH/environments/${KAYOBE_ENVIRONMENT}/inventory/group_vars/gitlab-runners`
220295
* `infra-vms` ensuring all required `infra_vm_extra_network_interfaces` are defined
221296
* `network-interfaces`
222297
* `allocated IPs`
223298

224-
5. Edit the ``${KAYOBE_CONFIG_PATH}/inventory/group_vars/gitlab-runners/runners.yml`` file which will contain the variables required to deploy a series of runners.
299+
5. Edit the ``$KAYOBE_CONFIG_PATH/inventory/group_vars/gitlab-runners/runners.yml`` file which will contain the variables required to deploy a series of runners.
225300
Below is an example of how GitLab runners can be configured for deployment.
226301
In this example we have two runners, one for production and one for staging and will both be deployed on the same host.
227302
This might not be possible for all deployments as multiple environments may require different runners as no single runner can serve all environments.
@@ -287,7 +362,7 @@ Runner Deployment
287362
288363
kayobe infra vm host configure --limit gitlab-runner-01
289364
290-
9. Run :code:`kayobe playbook run ${KAYOBE_CONFIG_PATH}/ansible/deployment/deploy-gitlab-runner.yml`
365+
9. Run :code:`kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/deployment/deploy-gitlab-runner.yml`
291366

292367
10. Check runners have registered properly by visiting the repository's :code:`CI/CD` tab -> :code:`Runners`
293368

@@ -299,25 +374,31 @@ OpenBao Deployment
299374
------------------
300375

301376
OpenBao must be installed on the same host as the runners.
302-
If you have multiple environments that each have the own runners then OpenBao must be installed on each host.
303-
However, if you have a single host that is shared between environments then OpenBao only needs to be installed once and can be achieved by running the following playbook.
377+
If you have multiple environments that each have their own runners then OpenBao must be installed on each host.
378+
However, if you have a single host that is shared between environments then OpenBao only needs to be installed once which can be achieved by running the following playbook.
304379

305380
.. code-block:: bash
306381
307-
kayobe playbook run ${KAYOBE_CONFIG_PATH}/ansible/deployment/deploy-openbao-kayobe-automation.yml
382+
kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store/secret-store-deploy-openbao-runners.yml
383+
384+
.. note::
385+
386+
This playbook configures OpenBao to listen on localhost using the loopback interface `lo` which will be accessible to the containers running the Kayobe Automation.
387+
If running OpenBao between environments care should be taken to ensure OpenBao listens on an interface which is accessible to all runners.
308388

309389
.. note::
310390

311391
If you are sharing OpenBao between environments then you will need to rerun the playbook under each environment to ensure that the correct secrets are available to the runners.
312392
You may use :code:`--tags add_secrets` to skip the deployment within other environments.
313-
For this to work you will need to copy :code:`vault/kayobe-automation-keys.json` from the first environment to the other environments in addition to copying the host definition of the gitlab runner add network IP.
393+
For this to work you will need to copy :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json` from the deployment environment to the other environments in addition to copying the host definition of the gitlab runner add network IP.
394+
395+
Once the above playbook has been applied you need to grab the root token from :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json` as you will need this to enable JWT support.
314396

315-
Once the above playbook has been applied you need to grab the root token from :code:`vault/kayobe-automation-keys.json` as you will need this to enable JWT support.
316-
This would also be an opportune time to encrypt the :code:`vault/kayobe-automation-keys.json` to protect the contents.
397+
This would also be an opportune time to encrypt the :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json` to protect the contents.
317398

318399
.. code-block:: bash
319400
320-
ansible-vault encrypt vault/kayobe-automation-keys.json --vault-password-file ~/.vault.password
401+
ansible-vault encrypt $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/kayobe-automation-keys.json --vault-password-file ~/.vault.password
321402
322403
In order to enable JWT support the following steps must be carried out within the openbao container on the runner host.
323404

@@ -364,13 +445,13 @@ In order to enable JWT support the following steps must be carried out within th
364445
GitLab Pipelines
365446
----------------
366447
367-
1. Edit :code:`${KAYOBE_CONFIG_PATH}/inventory/group_vars/gitlab-writer/writer.yml` or environment equivalent the appropriate changes to your deployments specific needs.
448+
1. Edit :code:`$KAYOBE_CONFIG_PATH/inventory/group_vars/gitlab-writer/writer.yml` or environment equivalent the appropriate changes to your deployments specific needs.
368449
See documentation for `stackhpc.kayobe_workflows.gitlab <https://github.com/stackhpc/ansible-collection-kayobe-workflows/tree/main/roles/gitlab>`__.
369450
Following the instructions in the documentation will allow you to customise the workflows to fit within your deployment.
370451
If using multiple environments ensure that :code:`gitlab_kayobe_environments` is updated to reflect all environments present in the deployment.
371452
Also consider the impact runbooks might have as the runbooks are designed with a particular cloud in mind and may not be suitable for all deployments such as hyperconverged deployments with Ceph on hypervisors.
372453
373-
2. Run :code:`kayobe playbook run ${KAYOBE_CONFIG_PATH}/ansible/deployment/write-gitlab-pipelines.yml`
454+
2. Run :code:`kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/deployment/write-gitlab-pipelines.yml`
374455
375456
3. Commit and push all newly generated pipelines found under root of the repository.
376457

0 commit comments

Comments
 (0)