Skip to content

Commit 1c3fccb

Browse files
ocaisaocaisa
committed
More details on CI
1 parent bc2c18c commit 1c3fccb

7 files changed

Lines changed: 258 additions & 7 deletions

File tree

‎docs/building-on-top.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -237,8 +237,6 @@ make # (3)!
237237

238238
If we try this, we get output similar to
239239
``` { .output .no-copy}
240-
{EESSI/2025.06} $ module load CMake
241-
242240
{EESSI/2025.06} $ mkdir build
243241
244242
{EESSI/2025.06} $ cd build

‎docs/eessi-in-ci.md‎

Lines changed: 204 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,20 +47,219 @@ and GitLab to make sure this is the case:
4747
[https://gitlab.com/explore/catalog/eessi/gitlab-eessi](https://gitlab.com/explore/catalog/eessi/gitlab-eessi). It
4848
is possible to use the [EESSI GitLab Component](https://gitlab.com/explore/catalog/eessi/gitlab-eessi) in a
4949
self-hosted GitLab instance, documentation on how to do this is available at
50-
<https://docs.gitlab.com/ci/components/#use-a-gitlabcom-component-on-gitlab-self-managed>.
50+
<https://docs.gitlab.com/ci/components/#use-a-gitlabcom-component-on-gitlab-self-managed>.
5151

5252

53-
## Using the EESSI GitHub Action
53+
## Adding CI based on EESSI to our software project
5454

5555
??? question "But I use GitLab not GitHub :disappointed:"
5656

5757
The [EESSI GitLab Component](https://gitlab.com/explore/catalog/eessi/gitlab-eessi) also exists, and this allows
5858
you to follow a very similar approach to that described
59-
here for GitHub. At the end of the episode we will provide the equivalent file needed to enable GitLab CI.
59+
here for GitHub. At the end of the episode we will provide the equivalent file needed to enable GitLab CI.
6060

61-
## Adding CI based on EESSI to our software project
61+
The first thing we need to do is to translate our workflow into something that the workflow tool, GitHub Actions in
62+
this case, can understand. GitHub itself has
63+
[extensive documentation on using GitHub Actions](https://docs.github.com/en/actions) which we are not going to
64+
reproduce, instead we will jump straight to the documentation for the
65+
[EESSI GitHub Action](https://github.com/marketplace/actions/eessi#instructions) and try to adapt it for our use case.
66+
An interesting example jumps out, which the documentation says
67+
> will run on both `x86_64` and `Arm` architectures
68+
69+
``` { .yaml .copy }
70+
jobs:
71+
ubuntu-minimal:
72+
runs-on: ${{ matrix.os }}
73+
strategy:
74+
matrix:
75+
os:
76+
- ubuntu-24.04-arm
77+
- ubuntu-24.04
78+
steps:
79+
- uses: actions/checkout@v6
80+
- uses: eessi/github-action-eessi@v3
81+
with:
82+
eessi_stack_version: '2025.06'
83+
- name: Test EESSI
84+
run: |
85+
module avail
86+
shell: bash
87+
```
88+
If you've never seen a GitHub Action file before, there is lot to pick apart here. Some key parts are
89+
90+
* It runs the CI on both `ubuntu-24.04-arm` and `ubuntu-24.04`
91+
* It "checks out" the code repository where the file is stored
92+
* It uses `eessi/github-action-eessi` to configure the environment for `EESSI/2025.06`
93+
* The test it runs is a basic `module avail`
94+
95+
!!! warning "I need more information than that!"
96+
Unfortunately, we won't get into the details here of the syntax being used other than to say it is
97+
[YAML](https://yaml.org/). If you would like a detailed explanation, we recommend that you talk to an LLM to have
98+
it explain it to you. They are very powerful when it comes to helping with CI, and we will use this later to help
99+
us solve problems.
100+
101+
What does our workflow look like? Well, we can summarise it in a few commands once `EESSI/2025.06` is loaded and
102+
available and we are in the base directory of our repository:
103+
``` { .bash .copy}
104+
module load CMake/4.0.3-GCCcore-14.3.0 # (1)!
105+
module load HDF5/1.14.6-gompi-2025b # (2)!
106+
module load buildenv/default-foss-2025b # (3)!
107+
mkdir build
108+
cd build
109+
cmake .. # (4)!
110+
make -j # (5)!
111+
ctest --output-on-failure --verbose # (6)!
112+
```
113+
114+
1. Load our build dependency on `CMake`
115+
2. Load our runtime dependency on `HDF5` (and, implicitly, `MPI`)
116+
3. Load the `buildenv` module so we use the `RPATH` wrappers
117+
4. Run `cmake` for our project
118+
5. Build the project
119+
6. Run our tests
120+
121+
One thing we can do then is to just add our workflow to the example from the documentation. We need to be a little
122+
careful to respect the YAML syntax, but in the end we would have something like
123+
``` { .yaml .copy }
124+
jobs:
125+
ubuntu-minimal:
126+
runs-on: ${{ matrix.os }}
127+
strategy:
128+
matrix:
129+
os:
130+
- ubuntu-24.04-arm
131+
- ubuntu-24.04
132+
steps:
133+
- uses: actions/checkout@v6
134+
- uses: eessi/github-action-eessi@v3
135+
with:
136+
eessi_stack_version: '2025.06'
137+
- name: Build and test our package
138+
run: |
139+
module load CMake/4.0.3-GCCcore-14.3.0
140+
module load HDF5/1.14.6-gompi-2025b
141+
module load buildenv/default-foss-2025b
142+
mkdir build
143+
cd build
144+
cmake ..
145+
make -j
146+
ctest --output-on-failure --verbose
147+
shell: bash
148+
```
149+
150+
### How do we actually enable our CI?
151+
152+
In GitHub Actions, workflow files are stored in the `.github/workflows/` directory of your repository. You can either
153+
use `.yml` or `.yaml` as the file extension for your workflow file, and GitHub will recognize them as an intended
154+
workflow.
155+
156+
Let's give it a try. First, we need to enter the directory of **our** repository (which we created and cloned in the
157+
last episode). Then let's make a
158+
[feature branch](https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow) and create the
159+
folder structure for our CI
160+
161+
``` { .bash .copy }
162+
cd cicd-demo # (1)!
163+
git checkout -b add_ci_to_project # (2)!
164+
mkdir --parents .github/workflows/ # (3)!
165+
```
166+
167+
1. Enter our repository
168+
2. Checkout a branch (copy) of our repository with a specific name
169+
3. Make the directory structure to store our CI
170+
171+
Now, let's add the CI file (that we are hoping will work) to that directory
172+
```yaml title="ci.yml"
173+
--8<-- "scripts/ci-incomplete.yml"
174+
```
175+
176+
We now add that to git and push the branch to GitHub to try to trigger GitHub Actions.
177+
178+
``` { .bash .copy }
179+
git add .github/workflows/ci.yml # (1)!
180+
git commit -m "Add CI to our application" # (2)!
181+
git push origin add_ci_to_project # (3)!
182+
```
183+
184+
1. Add the new file to our git repository.
185+
2. Commit the file with a helpful comment.
186+
3. Push the new branch to our repository on GitHub.
187+
188+
After completing the final command we see something like
189+
``` { .output .no-copy}
190+
...
191+
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
192+
remote:
193+
remote: Create a pull request for 'add_ci_to_project' on GitHub by visiting:
194+
remote: https://github.com/vlad/cicd-demo/pull/new/add_ci_to_project
195+
remote:
196+
To github.com:vlad/cicd-demo.git
197+
* [new branch] add_ci_to_project -> add_ci_to_project
198+
...
199+
```
200+
(where `vlad` is replaced by your own GitHub user handle). Let's go ahead and create the pull request lile the text
201+
suggested by visiting the URL. Once we open the link, we can modify the title/description if we wish, but otherwise
202+
we can just go ahead and click "Create pull request".
203+
204+
We see a new Pull Request opened, but there doesn't seem to be anything else obvious going on. The PR has a few tabs
205+
though, one of which is "Checks" which sounds a little interesting. Let's click on that and see what happens.
206+
207+
<p align="center"><img src="img/ci-broken.png" alt="Broken CI" width="600px"/></p>
208+
209+
Hmm, there is a big red X there for some reason, and our workflow file does appear there. Let's explore why by clicking
210+
on our file name. That brings us to another page where the URL looks like
211+
`https://github.com/vlad/cicd-demo/actions/runs/27020476886`, so it definitely has something to do with Actions. Under
212+
"Annotations" we see another big X and an error
213+
``` { .output .no-copy }
214+
Error
215+
No event triggers defined in `on`
216+
```
217+
but what does that mean?
218+
219+
To figure that out, let's use an LLM. We can say that we have this error in GitHub Actions and paste the error and the
220+
current contents of our YAML file in for context. For ChatGPT, this returns some very useful information
221+
222+
> The error:
223+
> > Error: No event triggers defined in on
224+
>
225+
> means your GitHub Actions workflow file is missing the required top-level `on:` section. Every workflow must specify
226+
> what events trigger it.
227+
228+
and goes to explain that we need to add additional lines to the top of the file to indicate our triggers:
229+
230+
``` { .yaml .copy}
231+
name: CI
232+
233+
on:
234+
push:
235+
pull_request:
236+
```
237+
which tells GitHub Actions that it should be triggered for every `push` and every Pull Request.
238+
239+
Let's add that content to our `ci.yml` file
240+
```yaml title="ci.yml"
241+
--8<-- "scripts/ci.yml"
242+
```
243+
and push the updated file back to GitHub
244+
``` { .bash .copy }
245+
git add .github/workflows/ci.yml
246+
git commit -m "Add a trigger for our CI"
247+
git push origin add_ci_to_project
248+
```
249+
250+
Now, if we browse to the Pull Requests of our repository, we can see that something is happening.
251+
252+
<p align="center"><img src="img/running_ci.png" alt="Running CI" width="600px"/></p>
253+
254+
A new little brown button has appeared beside our commit, and "Checks" now has 4 items. If we click on the brown
255+
button a new box pops up that indicates that 4 different CI runs are executing.
256+
257+
<p align="center"><img src="img/details_ci.png" alt="Detailed CI runs" width="600px"/></p>
258+
259+
2 of these are for the push, one for each architecture, and two more for the Pull Request, again one for each
260+
architecture. If we click on `Details` for one of the runs, we get taken to where the workflow is actually running,
261+
and the output it is generating.
62262

63-
### Where to find help?
64263

65264
### What happens when CI fails?
66265

‎docs/img/ci-broken.png‎

45.5 KB
Loading

‎docs/img/details_ci.png‎

45.3 KB
Loading

‎docs/img/running_ci.png‎

53 KB
Loading

‎scripts/ci-incomplete.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
jobs:
2+
ubuntu-minimal:
3+
runs-on: ${{ matrix.os }}
4+
strategy:
5+
matrix:
6+
os:
7+
- ubuntu-24.04-arm
8+
- ubuntu-24.04
9+
steps:
10+
- uses: actions/checkout@v6
11+
- uses: eessi/github-action-eessi@v3
12+
with:
13+
eessi_stack_version: '2025.06'
14+
- name: Build and test our package
15+
run: |
16+
module load CMake/4.0.3-GCCcore-14.3.0
17+
module load HDF5/1.14.6-gompi-2025b
18+
module load buildenv/default-foss-2025b
19+
mkdir build
20+
cd build
21+
cmake ..
22+
make -j
23+
ctest --output-on-failure --verbose
24+
shell: bash

‎scripts/ci.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
jobs:
8+
ubuntu-minimal:
9+
runs-on: ${{ matrix.os }}
10+
strategy:
11+
matrix:
12+
os:
13+
- ubuntu-24.04-arm
14+
- ubuntu-24.04
15+
steps:
16+
- uses: actions/checkout@v6
17+
- uses: eessi/github-action-eessi@v3
18+
with:
19+
eessi_stack_version: '2025.06'
20+
- name: Build and test our package
21+
run: |
22+
module load CMake/4.0.3-GCCcore-14.3.0
23+
module load HDF5/1.14.6-gompi-2025b
24+
module load buildenv/default-foss-2025b
25+
mkdir build
26+
cd build
27+
cmake ..
28+
make -j
29+
ctest --output-on-failure --verbose
30+
shell: bash

0 commit comments

Comments
 (0)