From 7e815cae5680998f103ab2a41830a01147074723 Mon Sep 17 00:00:00 2001 From: Alexander Sinn Date: Mon, 4 May 2026 11:47:21 +0200 Subject: [PATCH 1/5] Add Documentation to run on JSC Jupiter Booster --- docs/source/building/hpc.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/source/building/hpc.rst b/docs/source/building/hpc.rst index d3ad10b59a..b0887fb153 100644 --- a/docs/source/building/hpc.rst +++ b/docs/source/building/hpc.rst @@ -33,6 +33,7 @@ This section documents quick-start guides for a selection of supercomputers that :maxdepth: 1 platforms/booster_jsc + platforms/jupyter_jsc platforms/lumi_csc platforms/maxwell_desy platforms/perlmutter_nersc From 04d06c931e34894929d42e6adfaa7e0590141d54 Mon Sep 17 00:00:00 2001 From: Alexander Sinn Date: Mon, 4 May 2026 11:48:43 +0200 Subject: [PATCH 2/5] add doc file --- .../source/building/platforms/jupiter_jsc.rst | 75 +++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 docs/source/building/platforms/jupiter_jsc.rst diff --git a/docs/source/building/platforms/jupiter_jsc.rst b/docs/source/building/platforms/jupiter_jsc.rst new file mode 100644 index 0000000000..535e0307c4 --- /dev/null +++ b/docs/source/building/platforms/jupiter_jsc.rst @@ -0,0 +1,75 @@ +Jupiter Booster @ JSC +===================== + +This page only provides HiPACE++ specific instructions. +For more information please visit the `JSC documentation `__. + +Log in with ``@login.jupiter.fz-juelich.de``. Note that you will need to log into +JuDoor to be added to a compute project, set up an SSH key, and set up 2FA. + +Running on GPU +-------------- + +Create a file ``profile.hipace`` and ``source`` it whenever you log in and want to work with HiPACE++: + +.. code-block:: bash + + # please set your project account + export proj= + # required dependencies + module load CMake + module load GCC + module load OpenMPI + module load CUDA + module load HDF5 + module load ccache # optional, accelerates recompilation + # optimize CUDA compilation for GH200 + export AMREX_CUDA_ARCH=9.0 + +Install HiPACE++ (the first time, and whenever you want the latest version): + +.. code-block:: bash + + source profile.hipace + git clone https://github.com/Hi-PACE/hipace.git $HOME/src/hipace # only the first time + cd $HOME/src/hipace + rm -rf build + cmake -S . -B build -DHiPACE_COMPUTE=CUDA + cmake --build build -j 16 + +You can get familiar with the HiPACE++ input file format in our :doc:`../../run/get_started` section, to prepare an input file that suits your needs. +You can then create your directory in your ``$SCRATCH_``, where you can put your input file and adapt the following submission script: + +.. code-block:: bash + + #!/bin/bash -l + #SBATCH -A $proj + #SBATCH --partition=booster + #SBATCH --nodes=2 + #SBATCH --ntasks=8 + #SBATCH --ntasks-per-node=4 + #SBATCH --gres=gpu:4 + #SBATCH --time=00:05:00 + #SBATCH --job-name=hipace + #SBATCH --output=hipace-%j-%N.txt + #SBATCH --error=hipace-%j-%N.err + export OMP_NUM_THREADS=1 + module load GCC + module load OpenMPI + module load CUDA + module load HDF5 + # fix issues with MPI + export UCX_CUDA_COPY_REG_WHOLE_ALLOC=on + export UCX_CUDA_COPY_MAX_REG_RATIO=0 + export UCX_MEMTYPE_REG_WHOLE_ALLOC_TYPES=cuda + export UCX_MEMTYPE_CACHE=n + srun -n 8 --cpu_bind=sockets $HOME/src/hipace/build/bin/hipace inputs + +and use it to submit a simulation. + +.. tip:: + Parallel simulations can be largely accelerated by using GPU-aware MPI. + To utilize GPU-aware MPI, the input parameter ``comms_buffer.on_gpu = 1`` must be set. + Additionally ``comms_buffer.pre_register_memory = 1`` can help improve startup time. + + Note that using GPU-aware MPI may require more GPU memory. From 102af58ed58231fc7df5979400b54614a6c37294 Mon Sep 17 00:00:00 2001 From: Alexander Sinn Date: Mon, 4 May 2026 11:49:27 +0200 Subject: [PATCH 3/5] fix --- docs/source/building/hpc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/building/hpc.rst b/docs/source/building/hpc.rst index b0887fb153..76e68aed97 100644 --- a/docs/source/building/hpc.rst +++ b/docs/source/building/hpc.rst @@ -33,7 +33,7 @@ This section documents quick-start guides for a selection of supercomputers that :maxdepth: 1 platforms/booster_jsc - platforms/jupyter_jsc + platforms/jupiter_jsc platforms/lumi_csc platforms/maxwell_desy platforms/perlmutter_nersc From 004fb6fec0034e99c1cb3a5ac0b2533db222242e Mon Sep 17 00:00:00 2001 From: Alexander Sinn Date: Mon, 4 May 2026 11:52:24 +0200 Subject: [PATCH 4/5] improve wording --- docs/source/building/platforms/jupiter_jsc.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/building/platforms/jupiter_jsc.rst b/docs/source/building/platforms/jupiter_jsc.rst index 535e0307c4..90f08bccce 100644 --- a/docs/source/building/platforms/jupiter_jsc.rst +++ b/docs/source/building/platforms/jupiter_jsc.rst @@ -4,8 +4,8 @@ Jupiter Booster @ JSC This page only provides HiPACE++ specific instructions. For more information please visit the `JSC documentation `__. -Log in with ``@login.jupiter.fz-juelich.de``. Note that you will need to log into -JuDoor to be added to a compute project, set up an SSH key, and set up 2FA. +Log in with ``@login.jupiter.fz-juelich.de``. Note that you first need to log into JuDoor +to be added to a compute project, set up an SSH key and set up 2FA. Running on GPU -------------- From 938488ba5d18638b8928a8e0b497e7a9af19addc Mon Sep 17 00:00:00 2001 From: Alexander Sinn Date: Mon, 4 May 2026 16:08:15 +0200 Subject: [PATCH 5/5] update sbatch command --- docs/source/building/platforms/jupiter_jsc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/building/platforms/jupiter_jsc.rst b/docs/source/building/platforms/jupiter_jsc.rst index 90f08bccce..55dcc37e21 100644 --- a/docs/source/building/platforms/jupiter_jsc.rst +++ b/docs/source/building/platforms/jupiter_jsc.rst @@ -43,7 +43,7 @@ You can then create your directory in your ``$SCRATCH_``, where you .. code-block:: bash #!/bin/bash -l - #SBATCH -A $proj + #SBATCH -A #SBATCH --partition=booster #SBATCH --nodes=2 #SBATCH --ntasks=8