Skip to content

Commit bb48a44

Browse files
cailmdaleyclaude
andcommitted
Move user-facing docs to the docs-rework PR (#739)
The README front door, the container.md 'Running on a cluster' section, and the basic_execution.md MPI docs are relocated to #739, which owns the full docs story (cluster docs now live in a dedicated clusters.md, so keeping the walkthrough here too would duplicate it). This PR keeps only the code/infra and the CLAUDE.md build-loop note that the container changes here introduce. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 981c8e0 commit bb48a44

3 files changed

Lines changed: 10 additions & 103 deletions

File tree

README.rst

Lines changed: 6 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,55 +1,22 @@
11
ShapePipe
22
=========
33

4-
|CI| |CD| |python312| |release|
4+
|CI| |CD| |python39| |release|
55

66
.. |CI| image:: https://github.com/CosmoStat/shapepipe/workflows/CI/badge.svg
77
:target: https://github.com/CosmoStat/shapepipe/actions?query=workflow%3ACI
88

99
.. |CD| image:: https://github.com/CosmoStat/shapepipe/actions/workflows/pages/pages-build-deployment/badge.svg
1010
:target: https://github.com/CosmoStat/shapepipe/actions/workflows/pages/pages-build-deployment
1111

12-
.. |python312| image:: https://img.shields.io/badge/python-3.12-green.svg
13-
:target: https://www.python.org/
12+
.. |python39| image:: https://img.shields.io/badge/python-3.9-green.svg
13+
:target: https://www.python.org/
1414

1515
.. |release| image:: https://img.shields.io/github/v/release/CosmoStat/shapepipe
1616
:target: https://github.com/CosmoStat/shapepipe/releases/latest
1717

1818
ShapePipe is a galaxy shape measurement pipeline developed within the
19-
CosmoStat lab at CEA Paris-Saclay. It runs the full chain from raw survey
20-
images to calibrated shear catalogues — object detection, PSF modelling, and
21-
shape measurement — and produced the first UNIONS cosmic-shear release.
19+
CosmoStat lab at CEA Paris-Saclay.
2220

23-
Quickstart
24-
----------
25-
26-
ShapePipe ships as a container image, so you can run the bundled example
27-
pipeline — a single CFIS tile through the full chain — without installing
28-
anything:
29-
30-
.. code-block:: bash
31-
32-
# Apptainer (HPC, no root needed):
33-
apptainer exec docker://ghcr.io/cosmostat/shapepipe:develop-runtime shapepipe_run_example
34-
35-
# ...or Docker:
36-
docker run --rm ghcr.io/cosmostat/shapepipe:develop-runtime shapepipe_run_example
37-
38-
The image is published on every push to the `GitHub Container Registry
39-
<https://github.com/CosmoStat/shapepipe/pkgs/container/shapepipe>`_:
40-
``:develop`` tracks the integration branch, release tags (e.g. ``:v1.1.0``) a
41-
stable cut, and the ``-runtime`` suffix selects the slim batch image over the
42-
full interactive one.
43-
44-
Documentation
45-
-------------
46-
47-
Full documentation lives at https://cosmostat.github.io/shapepipe. Good places
48-
to start:
49-
50-
- `Installation <https://cosmostat.github.io/shapepipe/installation.html>`_ — getting ShapePipe onto your machine or cluster.
51-
- `Basic execution <https://cosmostat.github.io/shapepipe/basic_execution.html>`_ and `configuration <https://cosmostat.github.io/shapepipe/configuration.html>`_ — running ``shapepipe_run`` and writing pipeline configs.
52-
- `Container workflow <https://cosmostat.github.io/shapepipe/container.html>`_ — the two image targets, the ``pyproject.toml`` / ``uv.lock`` / ``Dockerfile`` layers, and how to run on a SLURM cluster (with a worked candide example).
53-
54-
If you use ShapePipe in academic work, please cite Guinot et al. (2022) and
55-
Farrens et al. (2022).
21+
See the `documentation <https://cosmostat.github.io/shapepipe>`_ for details
22+
on how to install and run ShapePipe.

docs/source/basic_execution.md

Lines changed: 4 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -37,33 +37,11 @@ shapepipe_run -c <PATH TO CONFIG FILE>
3737
## Running the Pipeline with MPI
3838

3939
ShapePipe can also use [mpi4py](https://mpi4py.readthedocs.io/en/stable/)
40-
to spread work across multiple nodes of a cluster. Set `MODE = mpi` in the
41-
`[EXECUTION]` section of the config and launch with an MPI runner:
40+
for managing parallel processes on clusters with multiple nodes.
41+
The `shapepipe_run` script can be run with MPI as follows
4242

4343
```bash
44-
mpiexec -n <NUMBER OF RANKS> shapepipe_run -c <PATH TO CONFIG FILE>
44+
mpiexec -n <NUMBER OF CORES> shapepipe_run
4545
```
4646

47-
where `<NUMBER OF RANKS>` is the number of MPI processes to start.
48-
49-
### Through the container (the supported way on a cluster)
50-
51-
On a cluster you run ShapePipe from the published image as a standard Apptainer
52-
*hybrid* MPI job: the **host** `mpirun`/`mpiexec` launches one container rank per
53-
slot, and the OpenMPI bundled in the image wires the ranks together.
54-
55-
```bash
56-
# one-time: pull the runtime image
57-
apptainer pull shapepipe.sif docker://ghcr.io/cosmostat/shapepipe:develop-runtime
58-
59-
# load a host MPI in the same family as the image's OpenMPI (5.0.x), then launch
60-
module load openmpi
61-
mpirun -n <NUMBER OF RANKS> \
62-
apptainer exec --bind "$PWD:$PWD" shapepipe.sif \
63-
shapepipe_run -c <PATH TO CONFIG FILE>
64-
```
65-
66-
The image ships **OpenMPI 5.0.x** so that its PMIx matches modern cluster
67-
launchers. The host and container MPI must be compatible: if you see *N* copies
68-
of `rank 0 of 1` instead of one *N*-rank job, load a host OpenMPI in the 5.0.x
69-
family. See `example/pbs/candide_mpi.sh` for a complete SLURM batch script.
47+
where `<NUMBER OF CORES>` is the number of cores to allocate to the run.

docs/source/container.md

Lines changed: 0 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -91,44 +91,6 @@ in, all because something tries to write under `/app` or `$HOME`:
9191
If you bypass `/tmp` (e.g. with a custom apptainer profile) you may need
9292
to override these manually.
9393

94-
## Running on a cluster (SLURM)
95-
96-
On a batch cluster you pull the slim `runtime` image once to a SIF, then
97-
submit a job that runs `shapepipe_run` through it. The repo ships ready
98-
SLURM scripts for the **candide** cluster in `example/pbs/`
99-
`candide_smp.sh` (single node) and `candide_mpi.sh` (multi-node hybrid
100-
MPI) — that you can copy and adapt. The example below runs the bundled
101-
single-tile pipeline end to end:
102-
103-
```bash
104-
# 1. Keep the SIF and Apptainer's scratch off the quota-limited $HOME.
105-
# On candide a pull under $HOME fails with "disk quota exceeded";
106-
# point both at a roomy data partition instead.
107-
export DATA=/n17data/$USER # adjust to your data partition
108-
export APPTAINER_CACHEDIR=$DATA/.apptainer
109-
110-
# 2. Pull the runtime image (~850 MB).
111-
apptainer pull "$DATA/shapepipe-runtime.sif" \
112-
docker://ghcr.io/cosmostat/shapepipe:develop-runtime
113-
114-
# 3. Submit the example pipeline. SPDIR is your local clone; it is
115-
# bind-mounted at the same path inside the container so the config's
116-
# $SPDIR-relative input/output directories resolve identically in and
117-
# out of the container.
118-
SP_IMAGE="$DATA/shapepipe-runtime.sif" SPDIR="/path/to/shapepipe" \
119-
sbatch example/pbs/candide_smp.sh
120-
```
121-
122-
Both job scripts read `SP_IMAGE` (the SIF) and `SPDIR` (the clone) from
123-
the environment, so the same script serves the example and a real run —
124-
point the config inside the script at your own pipeline. The MPI script
125-
additionally needs the host's OpenMPI to match the container's PMIx wire
126-
protocol; it `module load`s a compatible OpenMPI (the image ships the
127-
5.0.x series), and the script's header comments explain the contract.
128-
129-
Adapting to another SLURM cluster is mostly the `#SBATCH` directives and
130-
the `module load` line — the `apptainer exec` invocation carries over.
131-
13294
## Three configuration layers
13395

13496
Three files determine what the image contains. Each has a clear role; the

0 commit comments

Comments
 (0)