Skip to content

Commit 374db2c

Browse files
TyskaioharstaInes Duitscrosmullerbaszoetekouw
authored
merge main in this release branch (#616)
Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Okke Harsta <oharsta@zilverline.com> Co-authored-by: Ines Duits <ines@deploy.mgnt.ams.surfconext.nl> Co-authored-by: crosmuller <chantal.rosmuller@surf.nl> Co-authored-by: Bas Zoetekouw <bas.zoetekouw@surf.nl> Co-authored-by: Peter Havekes <peter.havekes@surf.nl> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pieter van der Meulen <pieter.vandermeulen@surf.nl> Co-authored-by: Peter Havekes <peter@havekes.eu> Co-authored-by: Leroy <3416288+Liemine@users.noreply.github.com> Co-authored-by: Ricardo van der Heijden <ricardo.vanderheijden@surf.nl> Co-authored-by: Ricardo van der Heijden <20791917+ricardovdheijden@users.noreply.github.com>
1 parent d340264 commit 374db2c

File tree

122 files changed

+2141
-1442
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

122 files changed

+2141
-1442
lines changed

.ansible-lint

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
---
2+
profile: "production"
3+
offline: false

.github/workflows/molecule-loadbalancer.yml

Lines changed: 19 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2,31 +2,32 @@
22

33
name: loadbalancer
44
on:
5-
push:
6-
paths:
7-
- 'roles/haproxy/**'
8-
- 'roles/keepalived/**'
9-
- 'roles/bind/**'
10-
- 'molecule/loadbalancer/**'
11-
- 'roles/selfsigned_certs/**'
12-
- '.github/workflows/molecule-loadbalancer.yml'
13-
pull_request:
14-
paths:
15-
- 'roles/haproxy/**'
16-
- 'roles/keepalived/**'
17-
- 'roles/bind/**'
18-
- 'molecule/loadbalancer/**'
19-
- 'roles/selfsigned_certs/**'
20-
- '.github/workflows/molecule-loadbalancer.yml'
5+
workflow_dispatch:
6+
# push:
7+
# paths:
8+
# - 'roles/haproxy/**'
9+
# - 'roles/keepalived/**'
10+
# - 'roles/bind/**'
11+
# - 'molecule/loadbalancer/**'
12+
# - 'roles/selfsigned_certs/**'
13+
# - '.github/workflows/molecule-loadbalancer.yml'
14+
# pull_request:
15+
# paths:
16+
# - 'roles/haproxy/**'
17+
# - 'roles/keepalived/**'
18+
# - 'roles/bind/**'
19+
# - 'molecule/loadbalancer/**'
20+
# - 'roles/selfsigned_certs/**'
21+
# - '.github/workflows/molecule-loadbalancer.yml'
2122

2223
jobs:
2324
build:
2425
runs-on: ubuntu-20.04
2526
steps:
26-
- uses: actions/checkout@v4
27+
- uses: actions/checkout@v6
2728

2829
- name: Set up Python 3.8
29-
uses: actions/setup-python@v5
30+
uses: actions/setup-python@v6
3031
with:
3132
python-version: 3.8
3233

.github/workflows/molecule-mongo.yml

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,25 @@
22

33
name: mongo
44
on:
5-
push:
6-
paths:
7-
- 'roles/mongo/**'
8-
- 'molecule/mongo/**'
9-
- '.github/workflows/molecule-mongo.yml'
10-
pull_request:
11-
paths:
12-
- 'roles/mongo/**'
13-
- 'molecule/mongo/**'
14-
- '.github/workflows/molecule-mongo.yml'
5+
workflow_dispatch:
6+
# push:
7+
# paths:
8+
# - 'roles/mongo/**'
9+
# - 'molecule/mongo/**'
10+
# - '.github/workflows/molecule-mongo.yml'
11+
# pull_request:
12+
# paths:
13+
# - 'roles/mongo/**'
14+
# - 'molecule/mongo/**'
15+
# - '.github/workflows/molecule-mongo.yml'
1516

1617
jobs:
1718
build:
18-
runs-on: ubuntu-20.04
19+
runs-on: ubuntu-24.04
1920
steps:
20-
- uses: actions/checkout@v4
21-
21+
- uses: actions/checkout@v6
2222
- name: Set up Python 3.8
23-
uses: actions/setup-python@v5
23+
uses: actions/setup-python@v6
2424
with:
2525
python-version: 3.8
2626

.github/workflows/syntax.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@ jobs:
1919
build:
2020
runs-on: ubuntu-latest
2121
steps:
22-
- uses: actions/checkout@v4
22+
- uses: actions/checkout@v6
2323

2424
- name: Set up Python 3.8
25-
uses: actions/setup-python@v5
25+
uses: actions/setup-python@v6
2626
with:
2727
python-version: 3.8
2828

CHANGELOG.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
### Changed
6+
- All group_var files are moved to the environment example template, more information about where to save group_vars in the [README](https://github.com/OpenConext/OpenConext-deploy/blob/main/README.md)
7+
- separate plays for separate roles in the provision.yml playbook
8+
- seperate groups are defined for separate apps, dividing apps across the container servers should be set in the inventory not in the playbook, this way you can easily change it for different environments. This also makes it impossible to use the wrong tag and deploy something you did not intend to, instead nothing will happen.
9+
- mysql_standalone group replaces storage group
10+
11+
### Removed
12+
- selfsigned_certs role is deprecated and removed from the provision.yml playbook
13+
- environment/playbook inclusion in provision.yml
14+
15+
### Todo
16+
- [ ] Complete environments/template

README.md

Lines changed: 75 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -39,12 +39,12 @@ Every application has a seperate role to install it. The following roles can be
3939
| stepupra | Stepup ra interface |
4040
| stepupselfservice | Stepup selfservice interface |
4141

42-
All these applications run in Docker. You can use the "docker" role to install docker and Traefik. The result is a Docker application server, with port 443 open. Applications are served by Traefik and recognized on basis of a Host: header. If you run a small installation, you can add a https certificate to Traefik and run a single node application server.
42+
All these applications run in Docker. You can use the "docker" role to install docker and Traefik. The result is a Docker application server, with port 443 open. Applications are served by Traefik and recognized on basis of a Host: header. If you run a small installation, you can add a https certificate to Traefik and run a single node application server.
4343

44-
For a fully functioning environment you also need a MariaDB database server and a Mongo database server.
44+
For a fully functioning environment you also need a MariaDB database server and a Mongo database server.
4545

4646
## Infra roles
47-
This repository is used for deployment of SURFconext, and several roles that the SURFconext teams uses to provision our infrastructure are provided here as well. You can use them for your own infrastructure or use them as inspiration.
47+
This repository is used for deployment of SURFconext, and several roles that the SURFconext teams uses to provision our infrastructure are provided here as well. You can use them for your own infrastructure or use them as inspiration.
4848
| name | remarks |
4949
| --- | --- |
5050
| bind | DNS server for high availability. Very specific for SURFconext |
@@ -59,40 +59,93 @@ This repository is used for deployment of SURFconext, and several roles that the
5959
| mongo | Install a mongo cluster (has its own README) |
6060
| manage_provision_entities|Provision entities to Manage |
6161

62-
# Environment specific variables
63-
Many variables can be overridden to create a setup suitable for your needs. The environment should be placed in the directory environments_external.
62+
# Setting up your environment
63+
Many variables can be overridden to create a setup suitable for your needs. We will explain the setup here for one environment or for a multi-environment (OTAP for example) setup.
6464

65-
A script is available to provision a new environment. It will create a new environment directory under environments-external/ and it will create all necessary passwords and (self-signed) certificates. Replace <environment> with the name of the target. Replace <domain> with the domain of the target.
65+
The setup descibed below should work, but when using ansible many paths lead to Rome. If you want to know more about variables and where to save them, this can be helpfull: https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable
6666

67+
## Inventory
68+
You need an inventory file for your environment or multiple inventory files if you have multiple environments. An example can be found in environments/template
6769

70+
## Playbook
71+
You can use the provision.yml script to deploy all infra and application roles. Every play has a tag so you can deploy your environment one application at a time by using the specific tag. You can also use your own playbooks if you prefer.
72+
73+
## First steps
74+
Clone the repository with git.
75+
76+
```bash
77+
cd yourdir
78+
git clone https://github.com/OpenConext/OpenConext-deploy.git
6879
```
69-
/prep-env <environment> <domain>
70-
```
71-
Then run
72-
```
73-
cp environments-external/<environment>/host_vars/template.yml environments-external/<environment>/host_vars/<target_ip>.yml
74-
```
75-
(where <target_ip> is the ip address or hostname of your target machine, whatever is set in your inventory file)
7680

77-
Change in environments-external/<environment>/inventory:
78-
Change all references from %target_host% to <target_ip>
81+
Create ansible.cfg in your directory and add Openconext-deploy/roles to your roles_path
7982

83+
```bash
84+
[defaults]
85+
diff = true
86+
roles_path = OpenConext-deploy/roles # Add your own roles directory if you want
8087
```
81-
Please note that this has not been tested in quite a while. You will need a lot of manual work to get this environment working
88+
89+
## One environment
90+
Copy the inventory, host and group files from environment/template to your directory and adjust them according to your preferences:
91+
92+
```bash
93+
cp -R OpenConext-deploy/environments/template/* .
8294
```
8395

96+
Edit your inventory file
97+
Edit group_var and host_var files if necessary
98+
99+
Create an ansible vault in secrets and name it secrets.yml, an unencrypted example can be found in secrets/secret_example.yml
100+
More information about vaults: https://docs.ansible.com/projects/ansible/latest/vault_guide/index.html
101+
The final setup will look like this:
84102

85-
# Playbooks, tags and the provision wrapper script
103+
- group_vars/all.yml
104+
- group_vars/\<GROUPNAME\>.yml
105+
- secrets/secrets.yml
106+
- host_vars/\<HOSTNAME\>/yml
107+
- inventory
108+
- Openconext-deploy/provision.yml
109+
- Openconext-deploy/roles
110+
- \<YOUROWNOPTIONALPLAYBOOKS\>.yml
111+
- ansible.cfg
86112

87-
Two playbooks exist in this repository: provision.yml and playbook_haproxy.yml. The latter can be used to do red/blue deployments if you also use our haproxy role.
88-
The main playbook is provision.yml. It contains series of plays to install every role on the right node. All roles are tagged, so you can use the [Ansible tag mechanism](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_tags.html) to deploy a specific role.
113+
You can use the provision playbook now:
89114

90-
If you would like to deploy manage to your test environment, you would run:
115+
```bash
116+
ansible-playbook OpenConext-deploy/provision.yml -i inventory -t <TAG> --ask-vault-password
91117
```
92-
ansible-playbook -i environments-external/test/inventory --tags manage -u THE_REMOTE_SSH_USER_WITH_SUDO_PERMISSIONS
118+
119+
## Multi-environment
120+
Copy the inventory and group files from environment/template to your directory and adjust them according to your preferences:
121+
122+
```bash
123+
mkdir <ENVIRONMENT> # test for example
124+
cp -R OpenConext-deploy/environments/template/* <ENVIRONMENT>
125+
# etc...
93126
```
127+
Edit your inventory files
128+
Edit group_var and host_var files if necessary
129+
130+
For each environment create an ansible vault in secrets and name it secrets.yml, an unencrypted example can be found in secrets/secret_example.yml
131+
More information about vaults: https://docs.ansible.com/projects/ansible/latest/vault_guide/index.html
132+
133+
The final setup will look like this:
94134

95-
A wrapper script which enables you to use your own roles can be used as well. That is documented here: https://github.com/OpenConext/OpenConext-deploy/wiki/Add-your-own-roles-and-playbooks
135+
- \<ENVIRONMENT\>/group_vars/all.yml
136+
- \<ENVIRONMENT\>/group_vars/\<GROUPNAME\>.yml
137+
- \<ENVIRONMENT\>/host_vars/\<HOSTNAME\>/yml
138+
- \<ENVIRONMENT\>/inventory
139+
- Openconext-deploy/provision.yml
140+
- Openconext-deploy/roles
141+
- \<YOUROWNOPTIONALPLAYBOOKS\>.yml
142+
- ansible.cfg
143+
144+
You can use the provision playbook now:
145+
146+
```bash
147+
ansible-playbook OpenConext-deploy/provision.yml -i <ENVIRONMENT>/inventory -t <TAG> --ask-vault-password
148+
```
96149

97150
# License
98151

0 commit comments

Comments
 (0)