Skip to content

Commit 9b80ac9

Browse files
committed
Address some comments
1 parent c695add commit 9b80ac9

2 files changed

Lines changed: 102 additions & 21 deletions

File tree

docs/blog/posts/2026/02/Spack-on-top-of-EESSI-best-of-both-worlds.md

Lines changed: 102 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -12,57 +12,138 @@ slug: Spack-on-top-of-EESSI-best-of-both-worlds
1212

1313
The HPC software landscape offers powerful tools for managing scientific software, such as [EasyBuild](https://easybuild.io) and [Spack](https://spack.io).
1414

15-
[EESSI](https://www.eessi.io) provides a ready-to-use software stack with thousands of optimized packages built with EasyBuild.
15+
[EESSI](https://www.eessi.io) provides a ready-to-use software stack with thousands of optimized software installations that were built with EasyBuild.
1616

17-
Imagine you are working on an HPC system with EESSI already available. You have access to a wealth of optimized scientific software packages, libraries, tools, as well as compilers. But you need to install a new tool or a specific version of a package that's not in EESSI yet.
17+
Imagine you are working on an HPC system with EESSI already available. You have access to a wealth of optimized installations for scientific software packages, libraries, tools, as well as compilers. But you need to install a new tool or a specific version of a package that's not in EESSI yet.
1818

19-
You can already extend EESSI with new software through the [EESSI-extend](https://www.eessi.io/docs/using_eessi/building_on_eessi/#building-software-on-top-of-eessi-with-easybuild) module.
20-
This utility provides you with a pre-configured EasyBuild installation that you can use to build packages from [EasyConfig files](https://docs.easybuild.io/terminology/#easyconfig_files).
19+
You can already extend EESSI with new software through the [EESSI-extend](../../../../using_eessi/building_on_eessi.md) module.
20+
This utility provides you with a pre-configured EasyBuild installation that you can use to build packages from [easyconfig files](https://docs.easybuild.io/terminology/#easyconfig_files).
2121

2222
Like EasyBuild, [Spack](https://spack.io) is a flexible build tool that also offers a vast repository of build recipes maintained by a large and active community, making it a familiar tool for many HPC users.
23-
By enabling Spack to leverage packages already available in EESSI as dependencies, we can offer users the best of both worlds: the convenience of a pre-built, optimized software stack combined with the flexibility to quickly build new packages using tools they already know.
23+
By enabling Spack to leverage software installations already available in EESSI as dependencies, we can offer users the best of both worlds: the convenience of a pre-built, optimized software stack combined with the flexibility to quickly build new packages using tools they already know.
2424

2525
<!-- more -->
2626

2727

28-
In a previous [blog post](https://www.eessi.io/docs/blog/2026/01/09/Spack-on-top-of-EESSI-PoC/), we presented a first proof-of-concept implementation of this vision.
28+
In a previous [blog post](../01/Spack-on-top-of-EESSI-PoC.md), we presented a first proof-of-concept implementation of this vision.
2929
We used a custom-built upstream database to make Spack aware of EESSI's software stack and managed to build a new Quantum ESPRESSO installation that reused dependencies from EESSI.
3030

31-
Thanks to recent updates to Spack (in particular, treating externals as concrete specs and allowing the definition of dependencies – see [this PR](https://github.com/spack/spack/pull/51118)) and an active collaboration with its developers, it is now possible to connect Spack and EESSI in a *seamless* and *Spack-native* way.
31+
Thanks to recent updates to Spack (in particular, treating externals as concrete specs and allowing the definition of dependencies – see [this pull request in Spack](https://github.com/spack/spack/pull/51118)) and an active collaboration with its developers, it is now possible to connect Spack and EESSI in a *seamless* and *Spack-native* way.
3232

3333
## How it works
3434

35-
Spack 1.1+ introduces external packages treated as concrete specs (i.e. like any other spec), with dependency support, allowing us to expose EESSI packages as externals along with their dependencies.
35+
Spack 1.1+ introduces external packages treated as concrete specs (i.e. like any other spec), with dependency support, allowing us to expose software installed in EESSI as externals along with their dependencies.
3636
This is the proper, *Spack-onic* way to achieve our goal, and it does not require any modification of Spack's source code, nor the creation of a custom database.
3737

3838
!!! note
3939

4040
Currently we need to add a small patch to Spack to make sure its compiler wrapper works correctly
4141
with the unusual sysroot configuration of EESSI. This is a known
42-
[issue](https://github.com/spack/spack/issues/51582) that will be addressed in future Spack releases.)
42+
[issue](https://github.com/spack/spack/issues/51582) that will be addressed in future Spack releases.
4343

4444
We implement the following workflow to connect Spack to EESSI, and build a new Quantum ESPRESSO with Spack reusing EESSI packages as dependencies.
4545

4646

47-
### Step 1 – EESSI software-layer builds
48-
49-
Declare EESSI software-layer builds as [external packages](https://spack.readthedocs.io/en/latest/packages_yaml.html#external-packages) in a `packages.yaml` configuration file with [external dependencies](https://spack.readthedocs.io/en/latest/packages_yaml.html#specifying-dependencies-among-external-packages).
50-
51-
![Example packages.yaml of software installations included in EESSI](spack-eessi-20260205-001.webp)
47+
### Step 1 – External packages for EESSI software layer
48+
49+
Declare software installations in EESSI software layer as [external packages](https://spack.readthedocs.io/en/latest/packages_yaml.html#external-packages) in a `packages.yaml` configuration file with [external dependencies](https://spack.readthedocs.io/en/latest/packages_yaml.html#specifying-dependencies-among-external-packages).
50+
51+
```yaml
52+
# Example of packages.yaml file for EESSI software stack without compat-layer packages (to be added by detection tool)
53+
# EESSI detected microarchitecture: haswell
54+
packages:
55+
gcc:
56+
externals:
57+
- spec: gcc@13.2.0 languages:='c,c++,fortran' target=haswell
58+
prefix: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/GCCcore/13.2.0
59+
dependencies: []
60+
extra_attributes:
61+
compilers:
62+
c: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/GCCcore/13.2.0/bin/gcc
63+
cxx: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/GCCcore/13.2.0/bin/g++
64+
fortran: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/GCCcore/13.2.0/bin/gfortran
65+
gmake:
66+
externals:
67+
- spec: gmake@4.4.1 target=haswell
68+
prefix: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/make/4.4.1-GCCcore-13.2.0
69+
dependencies:
70+
- spec: gcc@13.2.0 target=haswell
71+
openblas:
72+
externals:
73+
- spec: openblas@0.3.24~ilp64 threads=openmp target=haswell
74+
prefix: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/OpenBLAS/0.3.24-GCC-13.2.0
75+
dependencies:
76+
- spec: gcc@13.2.0 target=haswell
77+
fftw:
78+
externals:
79+
- spec: fftw@3.3.10~mpi+openmp+shared precision=float,double,long_double,quad target=haswell
80+
prefix: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/FFTW/3.3.10-GCC-13.2.0
81+
dependencies:
82+
- spec: gcc@13.2.0 target=haswell
83+
curl:
84+
externals:
85+
- spec: curl@8.3.0+nghttp2 target=haswell
86+
prefix: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/cURL/8.3.0-GCCcore-13.2.0
87+
dependencies:
88+
- spec: gcc@13.2.0 target=haswell
89+
- spec: zlib@1.2.13 # compat
90+
deptypes:
91+
- link
92+
- spec: openssl@1.1.1w # compat
93+
deptypes:
94+
- build
95+
- link
96+
libarchive:
97+
externals:
98+
- spec: libarchive@3.7.2 target=haswell
99+
prefix: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/libarchive/3.7.2-GCCcore-13.2.0
100+
dependencies:
101+
- spec: gcc@13.2.0 target=haswell
102+
cmake:
103+
externals:
104+
- spec: cmake@3.31.8 target=haswell
105+
prefix: /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/haswell/software/CMake/3.27.6-GCCcore-13.2.0
106+
dependencies:
107+
- spec: gcc@13.2.0 target=haswell
108+
- spec: curl@8.3.0 target=haswell
109+
deptypes:
110+
- build
111+
- link
112+
- spec: ncurses@6.4.20230401 # compat
113+
deptypes:
114+
- build
115+
- link
116+
- spec: zlib@1.2.13 # compat
117+
deptypes:
118+
- build
119+
- link
120+
- spec: libarchive@3.7.2 target=haswell
121+
deptypes:
122+
- build
123+
- link
124+
- spec: bzip2@1.0.8 # compat
125+
deptypes:
126+
- link
127+
- spec: openssl@1.1.1w # compat
128+
deptypes:
129+
- build
130+
- link
131+
132+
```
52133

53134
*Variants* (e.g. `~mpi+openmp`) should be specified to the best of our knowledge. By default, Spack will fill in the gaps with the package's default variants.
54135

55-
*Link* and *runtime dependencies* should be specified whenever possible. Pure build dependencies are not needed (the only exception being the *compiler*, Spack can reuse this information). A dependency spec should unambiguously point to another declared external package. If an ambiguity exists, Spack will throw an error.
136+
*Link* and *runtime dependencies* should be specified whenever possible. Pure build dependencies are not needed (the only exception being the *compiler*, since Spack can reuse this information). A dependency spec should unambiguously point to another declared external package. If an ambiguity exists, Spack will throw an error.
56137

57138
Many packages in EESSI depends on packages of the compatibility layer (e.g. `zlib`), that were filtered out during the build process. To detect these packages, see the [next step](#step-2-optional-suggested-eessi-compat-layer).
58139

59-
Anyway, software packages in EESSI are always RPATH-ed to their dependencies, so in most real-case scenarios linking will likely work even if you forget to declare some dependencies.
140+
Since software installations in EESSI are always RPATH'ed to their dependencies, linking will likely work in most real-case scenarios even if you forget to declare some dependencies.
60141

61142

62-
### Step 2 – (optional, suggested) EESSI compat-layer
143+
### Step 2 – (optional, suggested) EESSI compatibility layer
63144

64-
It is possible to detect OS packages available under the EESSI compat-layer and configure Spack to use them as externals.
65-
In EESSI, these packages are often dependencies of software-layer packages (see previous example), so it is suggested to include them.
145+
It is possible to detect software installations available in the EESSI compatibility layer and configure Spack to use them as externals.
146+
In EESSI, these are often dependencies of software installations in the software layer (see previous example), so it is suggested to include them.
66147

67148
This can be done automatically via `spack external find`:
68149

@@ -145,8 +226,8 @@ The future of HPC software management is not about choosing between distribution
145226

146227
### Learn more and get involved
147228

148-
We encourage you to join the discussion on the `#spack` channel of [EESSI Slack](https://join.slack.com/t/eessi-hpc/shared_invite/zt-2wg10p26d-m_CnRB89xQq3zk9qxf1k3g) and share your experiences and suggestions!
229+
We encourage you to join the discussion on the `#spack` channel of [EESSI Slack]({{ config.extra.slack_invite_url }}), as well as our weekly [EESSI Happy Hour sessions](https://www.eessi.io/docs/training-events/happy-hours-sessions/), and share your experiences and suggestions!
149230

150231
### Acknowledgements
151232

152-
This work was made possible thanks to the collaboration with the Spack development team, in particular Todd Gamblin and Massimiliano Culpo, as well as EESSI team members Kenneth Hoste and Alan O'Cais.
233+
This work was made possible thanks to the collaboration with the Spack development team, in particular [Todd Gamblin](https://github.com/tgamblin) and [Massimiliano Culpo](https://github.com/alalazo), as well as EESSI team members [Kenneth Hoste](https://github.com/boegel) and [Alan O'Cais](https://github.com/ocaisa).
-203 KB
Binary file not shown.

0 commit comments

Comments
 (0)