You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The HPC software landscape offers powerful tools for managing scientific software, such as [EasyBuild](https://easybuild.io) and [Spack](https://spack.io).
14
14
15
-
[EESSI](https://www.eessi.io) provides a ready-to-use software stack with thousands of optimized packages built with EasyBuild.
15
+
[EESSI](https://www.eessi.io) provides a ready-to-use software stack with thousands of optimized software installations that were built with EasyBuild.
16
16
17
-
Imagine you are working on an HPC system with EESSI already available. You have access to a wealth of optimized scientific software packages, libraries, tools, as well as compilers. But you need to install a new tool or a specific version of a package that's not in EESSI yet.
17
+
Imagine you are working on an HPC system with EESSI already available. You have access to a wealth of optimized installations for scientific software packages, libraries, tools, as well as compilers. But you need to install a new tool or a specific version of a package that's not in EESSI yet.
18
18
19
-
You can already extend EESSI with new software through the [EESSI-extend](https://www.eessi.io/docs/using_eessi/building_on_eessi/#building-software-on-top-of-eessi-with-easybuild) module.
20
-
This utility provides you with a pre-configured EasyBuild installation that you can use to build packages from [EasyConfig files](https://docs.easybuild.io/terminology/#easyconfig_files).
19
+
You can already extend EESSI with new software through the [EESSI-extend](../../../../using_eessi/building_on_eessi.md) module.
20
+
This utility provides you with a pre-configured EasyBuild installation that you can use to build packages from [easyconfig files](https://docs.easybuild.io/terminology/#easyconfig_files).
21
21
22
22
Like EasyBuild, [Spack](https://spack.io) is a flexible build tool that also offers a vast repository of build recipes maintained by a large and active community, making it a familiar tool for many HPC users.
23
-
By enabling Spack to leverage packages already available in EESSI as dependencies, we can offer users the best of both worlds: the convenience of a pre-built, optimized software stack combined with the flexibility to quickly build new packages using tools they already know.
23
+
By enabling Spack to leverage software installations already available in EESSI as dependencies, we can offer users the best of both worlds: the convenience of a pre-built, optimized software stack combined with the flexibility to quickly build new packages using tools they already know.
24
24
25
25
<!-- more -->
26
26
27
27
28
-
In a previous [blog post](https://www.eessi.io/docs/blog/2026/01/09/Spack-on-top-of-EESSI-PoC/), we presented a first proof-of-concept implementation of this vision.
28
+
In a previous [blog post](../01/Spack-on-top-of-EESSI-PoC.md), we presented a first proof-of-concept implementation of this vision.
29
29
We used a custom-built upstream database to make Spack aware of EESSI's software stack and managed to build a new Quantum ESPRESSO installation that reused dependencies from EESSI.
30
30
31
-
Thanks to recent updates to Spack (in particular, treating externals as concrete specs and allowing the definition of dependencies – see [this PR](https://github.com/spack/spack/pull/51118)) and an active collaboration with its developers, it is now possible to connect Spack and EESSI in a *seamless* and *Spack-native* way.
31
+
Thanks to recent updates to Spack (in particular, treating externals as concrete specs and allowing the definition of dependencies – see [this pull request in Spack](https://github.com/spack/spack/pull/51118)) and an active collaboration with its developers, it is now possible to connect Spack and EESSI in a *seamless* and *Spack-native* way.
32
32
33
33
## How it works
34
34
35
-
Spack 1.1+ introduces external packages treated as concrete specs (i.e. like any other spec), with dependency support, allowing us to expose EESSI packages as externals along with their dependencies.
35
+
Spack 1.1+ introduces external packages treated as concrete specs (i.e. like any other spec), with dependency support, allowing us to expose software installed in EESSI as externals along with their dependencies.
36
36
This is the proper, *Spack-onic* way to achieve our goal, and it does not require any modification of Spack's source code, nor the creation of a custom database.
37
37
38
38
!!! note
39
39
40
40
Currently we need to add a small patch to Spack to make sure its compiler wrapper works correctly
41
41
with the unusual sysroot configuration of EESSI. This is a known
42
-
[issue](https://github.com/spack/spack/issues/51582) that will be addressed in future Spack releases.)
42
+
[issue](https://github.com/spack/spack/issues/51582) that will be addressed in future Spack releases.
43
43
44
44
We implement the following workflow to connect Spack to EESSI, and build a new Quantum ESPRESSO with Spack reusing EESSI packages as dependencies.
45
45
46
46
47
-
### Step 1 – EESSI software-layer builds
48
-
49
-
Declare EESSI software-layer builds as [external packages](https://spack.readthedocs.io/en/latest/packages_yaml.html#external-packages) in a `packages.yaml` configuration file with [external dependencies](https://spack.readthedocs.io/en/latest/packages_yaml.html#specifying-dependencies-among-external-packages).
50
-
51
-

47
+
### Step 1 – External packages for EESSI software layer
48
+
49
+
Declare software installations in EESSI software layer as [external packages](https://spack.readthedocs.io/en/latest/packages_yaml.html#external-packages) in a `packages.yaml` configuration file with [external dependencies](https://spack.readthedocs.io/en/latest/packages_yaml.html#specifying-dependencies-among-external-packages).
50
+
51
+
```yaml
52
+
# Example of packages.yaml file for EESSI software stack without compat-layer packages (to be added by detection tool)
*Variants* (e.g. `~mpi+openmp`) should be specified to the best of our knowledge. By default, Spack will fill in the gaps with the package's default variants.
54
135
55
-
*Link* and *runtime dependencies* should be specified whenever possible. Pure build dependencies are not needed (the only exception being the *compiler*, Spack can reuse this information). A dependency spec should unambiguously point to another declared external package. If an ambiguity exists, Spack will throw an error.
136
+
*Link* and *runtime dependencies* should be specified whenever possible. Pure build dependencies are not needed (the only exception being the *compiler*, since Spack can reuse this information). A dependency spec should unambiguously point to another declared external package. If an ambiguity exists, Spack will throw an error.
56
137
57
138
Many packages in EESSI depends on packages of the compatibility layer (e.g. `zlib`), that were filtered out during the build process. To detect these packages, see the [next step](#step-2-optional-suggested-eessi-compat-layer).
58
139
59
-
Anyway, software packages in EESSI are always RPATH-ed to their dependencies, so in most real-case scenarios linking will likely work even if you forget to declare some dependencies.
140
+
Since software installations in EESSI are always RPATH'ed to their dependencies, linking will likely work in most real-case scenarios even if you forget to declare some dependencies.
It is possible to detect OS packages available under the EESSI compat-layer and configure Spack to use them as externals.
65
-
In EESSI, these packages are often dependencies of software-layer packages (see previous example), so it is suggested to include them.
145
+
It is possible to detect software installations available in the EESSI compatibility layer and configure Spack to use them as externals.
146
+
In EESSI, these are often dependencies of software installations in the software layer (see previous example), so it is suggested to include them.
66
147
67
148
This can be done automatically via `spack external find`:
68
149
@@ -145,8 +226,8 @@ The future of HPC software management is not about choosing between distribution
145
226
146
227
### Learn more and get involved
147
228
148
-
We encourage you to join the discussion on the `#spack` channel of [EESSI Slack](https://join.slack.com/t/eessi-hpc/shared_invite/zt-2wg10p26d-m_CnRB89xQq3zk9qxf1k3g) and share your experiences and suggestions!
229
+
We encourage you to join the discussion on the `#spack` channel of [EESSI Slack]({{ config.extra.slack_invite_url }}), as well as our weekly [EESSI Happy Hour sessions](https://www.eessi.io/docs/training-events/happy-hours-sessions/), and share your experiences and suggestions!
149
230
150
231
### Acknowledgements
151
232
152
-
This work was made possible thanks to the collaboration with the Spack development team, in particular Todd Gamblin and Massimiliano Culpo, as well as EESSI team members Kenneth Hoste and Alan O'Cais.
233
+
This work was made possible thanks to the collaboration with the Spack development team, in particular [Todd Gamblin](https://github.com/tgamblin) and [Massimiliano Culpo](https://github.com/alalazo), as well as EESSI team members [Kenneth Hoste](https://github.com/boegel) and [Alan O'Cais](https://github.com/ocaisa).
0 commit comments