Skip to content

Commit 107ebc8

Browse files
committed
tidy up docs a wee bit
1 parent 840d85a commit 107ebc8

1 file changed

Lines changed: 38 additions & 29 deletions

File tree

Lines changed: 38 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
# Mirrors and Build Caches
22

33
Spack can use *mirrors* and *caches* to speed up image builds and to build on systems with limited or no internet access.
4-
They are configured in a single `mirrors.yaml` file that you supply on the command line:
4+
5+
They are configured in a single YAML file passed to `stack-config` using the `--mirror` flag:
56

67
```bash
78
stack-config -b $build -r $recipe -s $system --mirror mirrors.yaml
89
```
910

1011
The file is not part of the [system configuration](cluster-config.md): mirror locations are usually specific to the person running the build, so each invocation provides its own.
11-
Paths inside the file (such as relative gpg key paths) are resolved relative to the directory containing the `mirrors.yaml`, so a self-contained mirror directory (the `mirrors.yaml` plus its keys) can be moved around freely.
1212

1313
A `mirrors.yaml` can describe five kinds of entry, each optional and each documented below:
1414

@@ -20,23 +20,29 @@ A `mirrors.yaml` can describe five kinds of entry, each optional and each docume
2020
| [`sourcecache`](#source-cache) | one | writable local cache that fills with sources as you build |
2121
| [`concretizer`](#concretizer-cache) | one | writable local cache that persists concretization results |
2222

23-
A complete example:
23+
!!! example
24+
```yaml title="mirrors.yaml"
25+
buildcache:
26+
url: file:///capstor/scratch/team/uenv-cache
27+
private_key: /capstor/scratch/bobsmith/.keys/spack-push-key.gpg
28+
mount_specific: true
29+
bootstrap:
30+
url: https://bootstrap.spack.io
31+
sourcemirror:
32+
# more than one source mirror can be configred.
33+
netmirror:
34+
url: https://example.com/spack-sources
35+
localmirror:
36+
url: file://scratch/group15/spack-sources
37+
sourcecache:
38+
path: /capstor/scratch/bobsmith/spack-sources
39+
concretizer:
40+
path: /capstor/scratch/bobsmith/spack-concretizer
41+
```
42+
43+
!!! note
44+
Paths inside the file (such as relative gpg key paths) are resolved relative to the directory containing the `mirrors.yaml`, so a self-contained mirror directory (the `mirrors.yaml` plus its keys) can be moved around freely.
2445

25-
```yaml title="mirrors.yaml"
26-
buildcache:
27-
url: file:///capstor/scratch/team/uenv-cache
28-
private_key: $SCRATCH/.keys/spack-push-key.gpg
29-
mount_specific: true
30-
bootstrap:
31-
url: https://bootstrap.spack.io
32-
sourcemirror:
33-
mirror1:
34-
url: https://example.com/spack-sources
35-
sourcecache:
36-
path: /capstor/scratch/$USER/spack-sources
37-
concretizer:
38-
path: /capstor/scratch/$USER/spack-concretizer
39-
```
4046

4147
To stop using any entry, remove (or comment out) it from `mirrors.yaml`.
4248

@@ -50,7 +56,7 @@ During a build Spack fetches packages from the cache when it can, and signs and
5056
```yaml title="mirrors.yaml"
5157
buildcache:
5258
url: file:///capstor/scratch/team/uenv-cache
53-
private_key: $SCRATCH/.keys/spack-push-key.gpg
59+
private_key: /capstor/scratch/bobsmith/.keys/spack-push-key.gpg
5460
```
5561
5662
| Field | Required | Description |
@@ -79,7 +85,7 @@ Set `mount_specific: true` to append the mount point to the cache URL, giving ea
7985
```yaml
8086
buildcache:
8187
url: file:///capstor/scratch/team/uenv-cache
82-
private_key: $SCRATCH/.keys/spack-push-key.gpg
88+
private_key: /capstor/scratch/bobsmith/.keys/spack-push-key.gpg
8389
mount_specific: true # packages stored under .../uenv-cache/user-environment
8490
```
8591

@@ -89,11 +95,11 @@ A build cache needs an empty directory and a PGP signing key:
8995

9096
```bash
9197
# 1. create the cache directory
92-
mkdir -p $SCRATCH/uenv-cache
98+
mkdir -p /capstor/scratch/bobsmith/uenv-cache
9399
94100
# 2. generate and export a signing key
95101
spack gpg create <name> <e-mail>
96-
spack gpg export --secret $SCRATCH/.keys/spack-push-key.gpg
102+
spack gpg export --secret /capstor/scratch/bobsmith/.keys/spack-push-key.gpg
97103
```
98104

99105
See [Keys](#keys) for where to store the key.
@@ -177,7 +183,7 @@ Unlike a source mirror it is written to automatically, and is created on demand.
177183

178184
```yaml title="mirrors.yaml"
179185
sourcecache:
180-
path: /capstor/scratch/$USER/spack-sources
186+
path: /capstor/scratch/bobsmith/spack-sources
181187
```
182188

183189
| Field | Required | Description |
@@ -191,7 +197,7 @@ Concretization can be a large fraction of build time, so pointing this at a pers
191197

192198
```yaml title="mirrors.yaml"
193199
concretizer:
194-
path: /capstor/scratch/$USER/spack-concretizer
200+
path: /capstor/scratch/bobsmith/spack-concretizer
195201
```
196202

197203
| Field | Required | Description |
@@ -201,14 +207,17 @@ concretizer:
201207
This emits a `concretizer.yaml` that sets `concretizer:concretization_cache:{enable: true, url}`.
202208
The cache is keyed by the hash of the solver inputs, so it can be reused safely across builds — stale entries simply miss.
203209

204-
!!! warning "It does not eliminate concretization time"
205-
The cache stores only the *result of the solve* for a given set of inputs. Before it can be consulted, Spack still has to rebuild the full concretization problem on every run — loading the package recipes and enumerating the reusable packages — and that setup work is often the larger part of concretization. So a cache hit skips the solver but not the setup: concretization gets faster, not free.
210+
!!! info "concretizer cache is not a silver bullet"
211+
The cache stores only the *result of the solve* for a given set of inputs.
212+
Before it can be consulted, Spack still has to rebuild the full concretization problem on every run, loading the package recipes and enumerating the reusable packages, and that setup work is often the larger part of concretization.
213+
So a cache hit skips the solver but not the setup: concretization gets faster, not free.
206214

207-
The win is therefore largest for repeated builds of the same stack against a stable build cache (the previous solve is replayed), and smallest when the bulk of concretization time is in setup. The cache never makes concretization slower.
215+
The win is therefore largest for repeated builds of the same stack against a stable build cache (the previous solve is replayed), and smallest when the bulk of concretization time is in setup.
208216

209217
!!! note "Requires Spack ≥ 1.1"
210218
The `concretizer:concretization_cache` config key was introduced in Spack 1.1, and Spack 1.0 rejects it.
211-
Stackinator infers the Spack version from the `spack.commit` in `config.yaml` (defaulting to a supported version when the commit is a branch or arbitrary SHA that cannot be pinned). When it detects Spack 1.0 it skips the concretizer cache with a warning rather than producing a config that would fail the build.
219+
Stackinator infers the Spack version from the `spack.commit` in `config.yaml` (defaulting to a supported version when the commit is a branch or arbitrary SHA that cannot be pinned).
220+
When it detects Spack 1.0 it skips the concretizer cache with a warning rather than producing a config that would fail the build.
212221

213222
## Keys
214223

@@ -235,7 +244,7 @@ chmod 600 $SCRATCH/.keys/spack-push-key.gpg
235244

236245
See the [Spack documentation](https://spack.readthedocs.io/en/latest/getting_started.html#gpg-signing) for more on GPG keys.
237246

238-
!!! failure "Don't use `$HOME`"
247+
!!! warning "Don't use `$HOME`"
239248
The build remounts `~` as a tmpfs, so keys under `$HOME` are not visible during the build and Spack will fail to read them. Use scratch storage instead.
240249

241250
## Legacy `--cache` option

0 commit comments

Comments
 (0)