Skip to content

Commit 2050903

Browse files
ocaisaocaisa
committed
Initial commit
0 parents  commit 2050903

9 files changed

Lines changed: 1149 additions & 0 deletions

File tree

‎.github/workflows/docs.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: Documentation
2+
on:
3+
push:
4+
branches:
5+
- master
6+
- main
7+
permissions:
8+
contents: read
9+
pages: write
10+
id-token: write
11+
jobs:
12+
deploy:
13+
environment:
14+
name: github-pages
15+
url: ${{ steps.deployment.outputs.page_url }}
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/configure-pages@v6
19+
- uses: actions/checkout@v6
20+
- uses: actions/setup-python@v6
21+
with:
22+
python-version: 3.x
23+
- run: pip install zensical
24+
- run: zensical build --clean
25+
- uses: actions/upload-pages-artifact@v5
26+
with:
27+
path: site
28+
- uses: actions/deploy-pages@v5
29+
id: deployment

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.venv

‎docs/eessi-getting-access.md‎

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
# Getting Access to EESSI
2+
3+
To get access to EESSI, it suffices to have [CernVM-FS](https://cernvm.cern.ch/fs/) installed, and make it aware
4+
of the EESSI repository.
5+
6+
## Is EESSI accessible?
7+
8+
EESSI can be accessed via [a native (CernVM-FS) installation](#native-installation),
9+
or via [a container that includes CernVM-FS](#eessi-container).
10+
11+
Before you look into these options, check if EESSI is already accessible on your system.
12+
13+
Run the following command:
14+
``` { .bash .copy }
15+
ls /cvmfs/pilot.eessi-hpc.org
16+
```
17+
18+
!!! note
19+
20+
This ``ls`` command may take a couple of seconds to finish, since CernVM-FS may need to download
21+
or update the metadata for that directory.
22+
23+
If you see output like shown below, **you already have access to EESSI on your system**. :tada:
24+
```
25+
host_injections latest versions
26+
```
27+
28+
For starting to use EESSI, continue reading about
29+
[Setting up environment](eessi-usage.md#setting-up-environment).
30+
31+
If you see an error message as shown below, **EESSI is not yet accessible on your
32+
system**.
33+
```
34+
ls: /cvmfs/pilot.eessi-hpc.org: No such file or directory
35+
```
36+
No worries, you don't need to be a :mage: to get access to EESSI.
37+
38+
Continue reading about the [Native installation](#native-installation) of EESSI,
39+
or access via the [EESSI container](#eessi-container).
40+
41+
## Native installation
42+
43+
Setting up native access to EESSI, that is a system-wide deployment that does not require workarounds like
44+
[using a container](../eessi_container), requires the installation and configuration of [CernVM-FS](https://cernvm.cern.ch/fs).
45+
46+
This requires **admin privileges**, since you need to install CernVM-FS as an OS package.
47+
48+
The following actions must be taken for a (basic) native installation of EESSI:
49+
50+
* Installing CernVM-FS itself, ideally using the OS packages provided by the CernVM-FS project
51+
(although installing from source is also possible);
52+
* Installing the EESSI configuration for CernVM-FS, which can be done by installing the ``cvmfs-config-eessi``
53+
package that we provide for the most popular Linux distributions
54+
(more information available [here](https://github.com/EESSI/filesystem-layer/));
55+
* Creating a small client configuration file for CernVM-FS (``/etc/cvmfs/default.local``);
56+
see also the [CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-quickstart.html#create-default-local).
57+
58+
The good news is that all of this only requires a handful commands :astonished: :
59+
60+
=== "RHEL-based Linux distributions"
61+
62+
``` { .bash .copy }
63+
# Installation commands for RHEL-based distros like CentOS, Rocky Linux, Almalinux, Fedora, ...
64+
65+
# install CernVM-FS
66+
sudo yum install -y https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm
67+
sudo yum install -y cvmfs
68+
69+
# install EESSI configuration for CernVM-FS
70+
sudo yum install -y https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi-latest.noarch.rpm
71+
72+
# create client configuration file for CernVM-FS (no squid proxy, 10GB local CernVM-FS client cache)
73+
sudo bash -c "echo 'CVMFS_CLIENT_PROFILE="single"' > /etc/cvmfs/default.local"
74+
sudo bash -c "echo 'CVMFS_QUOTA_LIMIT=10000' >> /etc/cvmfs/default.local"
75+
76+
# make sure that EESSI CernVM-FS repository is accessible
77+
sudo cvmfs_config setup
78+
```
79+
80+
=== "Debian-based Linux distributions"
81+
82+
``` { .bash .copy }
83+
# Installation commands for Debian-based distros like Ubuntu, ...
84+
85+
# install CernVM-FS
86+
sudo apt-get install lsb-release
87+
wget https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest_all.deb
88+
sudo dpkg -i cvmfs-release-latest_all.deb
89+
rm -f cvmfs-release-latest_all.deb
90+
sudo apt-get update
91+
sudo apt-get install -y cvmfs
92+
93+
# install EESSI configuration for CernVM-FS
94+
wget https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi_latest_all.deb
95+
sudo dpkg -i cvmfs-config-eessi_latest_all.deb
96+
97+
# create client configuration file for CernVM-FS (no squid proxy, 10GB local CernVM-FS client cache)
98+
sudo bash -c "echo 'CVMFS_CLIENT_PROFILE="single"' > /etc/cvmfs/default.local"
99+
sudo bash -c "echo 'CVMFS_QUOTA_LIMIT=10000' >> /etc/cvmfs/default.local"
100+
101+
# make sure that EESSI CernVM-FS repository is accessible
102+
sudo cvmfs_config setup
103+
```
104+
105+
!!! note
106+
107+
:point_up: The commands above only cover the basic installation of EESSI.
108+
109+
This is good enough for an individual client, or for testing purposes,
110+
but for a production-quality setup you should also set up a Squid proxy cache.
111+
112+
For large-scale systems, like an HPC cluster, you should also consider setting up your own CernVM-FS Stratum-1 mirror server.
113+
114+
For more details on this, please refer to the
115+
[*Stratum 1 and proxies section* of the CernVM-FS tutorial](https://cvmfs-contrib.github.io/cvmfs-tutorial-2021/03_stratum1_proxies/).
116+
117+
## EESSI client container
118+
119+
The `eessi_container.sh` script provides a very easy yet versatile means
120+
to access EESSI.
121+
122+
This page guides you through several example scenarios
123+
illustrating the use of the script.
124+
125+
### Prerequisites
126+
127+
- Apptainer 1.0.0 (_or newer_), or Singularity 3.7.x
128+
- Check with `apptainer --version` or `singularity --version`
129+
- Support for the `--fusemount` option in the ``shell`` and ``run`` subcommands is required
130+
- Git
131+
- Check with `git --version`
132+
133+
### Preparation
134+
135+
Clone the [`EESSI/software-layer`](https://github.com/EESSI/software-layer.git)
136+
repository and change into the `software-layer` directory by running these commands:
137+
138+
``` { .bash .copy }
139+
git clone https://github.com/EESSI/software-layer.git
140+
cd software-layer
141+
```
142+
143+
### Quickstart
144+
145+
Run the `eessi_container` script (from the ``software-layer`` directory) to start a shell session in the EESSI container:
146+
147+
``` { .bash .copy }
148+
./eessi_container.sh
149+
```
150+
151+
!!! Note
152+
Startup will take a bit longer the first time you run this because the container image is downloaded and converted.
153+
154+
You should see output like
155+
```
156+
Using /tmp/eessi.abc123defg as tmp storage (add '--resume /tmp/eessi.abc123defg' to resume where this session ended).
157+
Pulling container image from docker://ghcr.io/eessi/build-node:debian11 to /tmp/eessi.abc123defg/ghcr.io_eessi_build_node_debian11.sif
158+
Launching container with command (next line):
159+
singularity -q shell --fusemount container:cvmfs2 pilot.eessi-hpc.org /cvmfs/pilot.eessi-hpc.org /tmp/eessi.abc123defg/ghcr.io_eessi_build_node_debian11.sif
160+
CernVM-FS: pre-mounted on file descriptor 3
161+
Apptainer> CernVM-FS: loading Fuse module... done
162+
fuse: failed to clone device fd: Inappropriate ioctl for device
163+
fuse: trying to continue without -o clone_fd.
164+
165+
Apptainer>
166+
```
167+
!!! Note
168+
You may have to press enter to clearly see the prompt as some messages
169+
beginning with `CernVM-FS: ` have been printed after the first prompt
170+
`Apptainer> ` was shown.
171+
172+
In this environment, you should be able to access the EESSI pilot repository:
173+
174+
``` { .bash .copy }
175+
ls /cvmfs/pilot.eessi-hpc.org
176+
```
177+
178+
More information on using the `eessi_container` script is available in the [EESSI documentation](https://eessi.github.io/docs/getting_access/eessi_container/).
179+
180+
181+
---
182+
183+
To start using EESSI, see [Using EESSI](eessi-usage.md).
184+
185+
186+
187+
[*next: Using EESSI*](eessi-usage.md) - [*(back to overview page)*](index.md)

‎docs/eessi-introduction.md‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# Introduction to EESSI
2+
3+
<p align="center"><a href="https://eessi.github.io/docs"><img src="../EESSI_logo_horizontal_transparant.png" alt="EESSI logo" width="600px"/></a></p>
4+
5+
## What is EESSI?
6+
7+
The European Environment for Scientific Software Installations (EESSI) project is a collaboration between various
8+
(mostly) European HPC sites and partners, to build a **shared stack of scientific software installations**.
9+
10+
The main idea is to work together and avoid duplicate work across HPC sites, even more so than we already do via
11+
EasyBuild.
12+
13+
For end users, we want to provide a **uniform user experience** with respect to available scientific software, regardless of which system they use.
14+
15+
EESSI should work on laptops, personal workstations, HPC clusters and in the cloud, which means we will need to support different CPUs, networks, GPUs, and so on.
16+
We hope to make this work for any Linux distribution, and maybe even macOS and Windows via [WSL](https://docs.microsoft.com/en-us/windows/wsl/),
17+
and a wide variety of CPU architectures (Intel, AMD, ARM, POWER, RISC-V).
18+
19+
We focus point on the **performance** of the provided software installations, but also on automating the workflow
20+
for maintaining the software stack, thoroughly testing the installations, and collaborating efficiently.
21+
22+
## Inspiration
23+
24+
The EESSI concept is heavily inspired by Compute Canada (now [Digital Research Alliance of Canada](https://alliancecan.ca/en)) software stack,
25+
which is a shared software stack used on all 5 major national systems (and a bunch of smaller ones) in Canada.
26+
27+
The design of the Compute Canada software stack is discussed in detail
28+
in the PEARC'19 paper [*"Providing a Unified Software Environment for Canada’s
29+
National Advanced Computing Centers"*](https://ssl.linklings.net/conferences/pearc/pearc19_program/views/includes/files/pap139s3-file1.pdf).
30+
31+
It has also been presented at the 5th EasyBuild User Meetings ([slides](https://users.ugent.be/~kehoste/eum20/eum20_03_maxime_computecanada.pdf), [recorded talk](https://www.youtube.com/watch?v=_0j5Shuf2uE&list=PLhnGtSmEGEQidEM8MZKkOaVutgt9WmqI0)), and is [well documented](https://docs.computecanada.ca/wiki/Accessing_CVMFS).
32+
33+
## High-level design
34+
35+
The EESSI project consists of 3 layers.
36+
37+
![EESSI overview](EESSI-overview-layers.png)
38+
39+
The bottom layer is the **filesystem layer**,
40+
which is responsible for distributing the software stack across clients.
41+
42+
The middle layer is a **compatibility layer**,
43+
which ensures that the software stack is compatible with multiple different
44+
client operating systems.
45+
46+
The top layer is the **software layer**, which contains the actual scientific software applications and their dependencies.
47+
48+
The host OS still provides a couple of things, like drivers for network and GPU, support for shared filesystems like GPFS and Lustre, a resource manager like Slurm, and so on.
49+
50+
### Filesystem layer
51+
52+
<img src="../EESSI-filesystem-layer.png" alt="EESSI filesystem layer" width="400px"/>
53+
54+
The bottom layer of the EESSI project is the **filesystem layer**,
55+
which is responsible for distributing the software stack.
56+
57+
For this we rely on [CernVM-FS](https://cernvm.cern.ch/portal/filesystem) (or CVMFS for short), a network file system used to distribute the software to the clients in a fast, reliable and scalable way.
58+
59+
CVMFS was created by CERN over a decade ago, specifically for the purpose of globally distributing a large software stack. For the experiments at the Large Hadron Collider, it hosts several hundred million files and directories that are distributed to the order of a 100,000 client computers.
60+
61+
<img src="../EESSI-cvmfs-hierarchy.png" alt="CernVM-FS hierarchy" width="600px"/>
62+
63+
The hierarchical structure with multiple caching layers (Stratum-0, Stratum-1's located at partner sites, and local caching proxies) ensures good performance with limited resources. Redundancy is provided by using multiple Stratum-1's at various sites. Since CVMFS is based on the HTTP protocol, the ubiquitous [Squid caching proxy](http://www.squid-cache.org/) can be leveraged to reduce server loads and improve performance at large installations (such as HPC clusters). Clients can easily mount the file system (read-only) via a [FUSE](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) (Filesystem in Userspace) module.
64+
65+
For a (basic) introduction to CernVM-FS, see [this presentation](https://www.youtube.com/watch?v=MyYx-xaL36k).
66+
67+
Detailed information about how we configure CVMFS is available at
68+
[https://github.com/EESSI/filesystem-layer](https://github.com/EESSI/filesystem-layer) .
69+
70+
### Compatibility layer
71+
72+
<img src="../EESSI-compat-layer.png" alt="EESSI compatibility layer" width="400px"/>
73+
74+
The middle layer of the EESSI project is the **compatibility layer**,
75+
which ensures that our scientific software stack is compatible with
76+
different (versions of) client operating systems (different Linux distributions,
77+
macOS and even Windows via [WSL](https://docs.microsoft.com/en-us/windows/wsl/)).
78+
79+
For this we rely on [Gentoo Prefix](https://wiki.gentoo.org/wiki/Project:Prefix),
80+
by installing a limited set of Gentoo Linux packages in a non-standard location
81+
(a "prefix"), using Gentoo's package manager [Portage](https://wiki.gentoo.org/wiki/Portage).
82+
83+
The compatible layer is maintained via our [https://github.com/EESSI/compatibility-layer](https://github.com/EESSI/compatibility-layer) GitHub repository.
84+
85+
### Software layer
86+
87+
<img src="../EESSI-software-layer.png" alt="EESSI software layer" width="400px"/>
88+
89+
The top layer of the EESSI project is the **software layer**,
90+
which provides the actual scientific software installations.
91+
92+
To install the software we include in our stack,
93+
we use [**EasyBuild**](https://easybuild.io),
94+
a framework for installing scientific software on HPC systems.
95+
These installations are optimized for a particular system architecture
96+
(specific CPU and GPU generation).
97+
98+
To access these software installation we provide *environment module files*
99+
and use [**Lmod**](https://lmod.readthedocs.io), a modern environment modules tool which has been widely
100+
adopted in the HPC community in recent years.
101+
102+
We leverage the [**archspec**](https://archspec.readthedocs.io) Python library
103+
to automatically select the best suited part of the software stack for
104+
a particular host, based on its system architecture.
105+
106+
The software layer is maintained through our [https://github.com/EESSI/software-layer](https://github.com/EESSI/software-layer) GitHub repository.
107+
108+
109+
## Current status
110+
111+
The EESSI project was started mid 2020, as a loose collaboration between various Dutch universities including
112+
the [University of Groningen](https://www.rug.nl/society-business/centre-for-information-technology), [VU
113+
Amsterdam](https://vu.nl/en), [TU Delft](https://www.tudelft.nl/en/), [TU Eindhoven](https://www.tue.nl/en/), and
114+
[HPC-UGent](https://www.ugent.be/hpc/en), as a follow-up of a meeting organised in Delft in early March 2020
115+
under the impuls of [Dell Technologies](https://www.dell.com).
116+
117+
*European Environment for Scientific Software Installations (EESSI)* was chosen as project name, and a GitHub
118+
organisation ([https://github.com/EESSI](https://github.com/EESSI)) and Slack were set up to kickstart the
119+
collaboration. Since April 2020, [online meetings](https://github.com/EESSI/meetings/wiki) have been held to share
120+
progress updates and discuss next steps.
121+
122+
In July 2020 a proof-of-concept [EESSI pilot repository](https://eessi.github.io/docs/pilot/) was set up,
123+
to explore technical aspects, automate the process of setting up the EESSI layers and installing software into EESSI.
124+
125+
In February 2022, an open access paper on EESSI was published (https://doi.org/10.1002/spe.3075).
126+
127+
During the first half of 2022, the most active partners in the EESSI project worked together on a project proposal
128+
to start a EuroHPC Centre-of-Excellence in which EESSI could be developed further. The proposal was accepted,
129+
and hence the [MultiXscale](https://www.multixscale.eu/) EuroHPC Centre-of-Excellence was started. This is a 4-year
130+
project (2023-2026) that is a collaboration between EESSI and [CECAM](https://www.cecam.org/), with goals
131+
that include making EESSI ready for production, and supporting community contributions.
132+
133+
[*next: Getting access to EESSI*](eessi-getting-access.md) - [*(back to overview page)*](index.md)

0 commit comments

Comments
 (0)