Skip to content

Commit 13f39ae

Browse files
authored
Enhance Apptainer tutorial for Torch usage
Expanded the tutorial on using Apptainer with Torch, detailing commands for running containers, differences between 'apptainer run' and 'apptainer exec', opening interactive shells, and pulling images from Docker registries.
1 parent a77be4e commit 13f39ae

1 file changed

Lines changed: 164 additions & 2 deletions

File tree

docs/hpc/13_tutorial_intro_hpc/11_apptainer on Torch

Lines changed: 164 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,175 @@ Researchers often rely on complex software stacks that include programming langu
66
Containers provide a way to package software together with the environment it needs to run. Instead of manually installing every dependency on a system, users can run software inside a container that already includes the required libraries and tools.
77

88
## What is Apptainer?
9-
10-
Apptainer is a container platform designed for Linux and HPC environments. It allows users to run software inside isolated environments without requiring administrator privileges on the host system.
9+
Apptainer is a container platform that allows users to run software inside isolated environments. It allows users to run software inside isolated environments without requiring administrator privileges on the host system.
1110

1211
Apptainer is the continuation of the Singularity project. The open-source Singularity project was renamed to Apptainer and continues to be developed under the Linux Foundation.
1312

1413
Unlike Docker, Apptainer does not require a privileged daemon running on the system. This makes it well suited for shared HPC environments where security and multi-user access are important considerations.
1514

1615
Torch uses Apptainer as its supported container platform. Many container images distributed through Docker registries can be used directly with Apptainer, allowing researchers to take advantage of existing software environments while working on the cluster.
1716

17+
# Using Apptainer to run commands on Torch
18+
## Running Specific Commands Within a Container
19+
20+
In the previous episode, we launched a container using a prebuilt Ubuntu image available on Torch. By default, a container may be configured to run a specific application when it starts. What if we want to run a different command inside the container?
21+
22+
If we know the path of an executable we want to run, we can use the `apptainer exec` command. For example, using the Ubuntu container image provided on Torch:
23+
24+
```bash
25+
apptainer exec /share/apps/images/ubuntu-24.04.3.sif /bin/echo "Hello World!"
26+
```
27+
28+
Output:
29+
30+
```text
31+
Hello World!
32+
```
33+
34+
Here, Apptainer starts a container from the Ubuntu image and executes the `/bin/echo` command inside the container. Once the command finishes, the container exits.
35+
36+
We can also run other commands inside the container. For example:
37+
38+
```bash
39+
apptainer exec /share/apps/images/ubuntu-24.04.3.sif cat /etc/os-release
40+
```
41+
42+
Example output:
43+
44+
```text
45+
PRETTY_NAME="Ubuntu 24.04.3 LTS"
46+
NAME="Ubuntu"
47+
VERSION_ID="24.04"
48+
...
49+
```
50+
51+
This confirms that the command is being executed within the Ubuntu container environment.
52+
53+
## The Difference Between `apptainer run` and `apptainer exec`
54+
55+
Above we used the `apptainer exec` command. In the previous episode we used `apptainer run`. To clarify, the difference between these two commands is:
56+
57+
### `apptainer run`
58+
59+
This starts a container and executes the default application configured within the image. You do not specify a command. Instead, Apptainer runs whatever application the image creator configured as the default.
60+
61+
```bash
62+
apptainer run image.sif
63+
```
64+
65+
### `apptainer exec`
66+
67+
This starts a container and runs a command that you provide on the command line.
68+
69+
```bash
70+
apptainer exec image.sif command
71+
```
72+
73+
Any default application configured within the image is ignored, and the command you specify is executed instead.
74+
75+
## Opening an Interactive Shell Within a Container
76+
77+
Sometimes it is useful to explore a container interactively. Apptainer provides the `apptainer shell` command for this purpose.
78+
79+
Launch a shell inside the Ubuntu container:
80+
81+
```bash
82+
apptainer shell /share/apps/images/ubuntu-24.04.3.sif
83+
```
84+
85+
You should see a prompt similar to:
86+
87+
```text
88+
Apptainer>
89+
```
90+
91+
You can now run commands inside the container:
92+
93+
```bash
94+
whoami
95+
pwd
96+
cat /etc/os-release
97+
```
98+
99+
Example output:
100+
101+
```text
102+
PRETTY_NAME="Ubuntu 24.04.3 LTS"
103+
NAME="Ubuntu"
104+
VERSION_ID="24.04"
105+
...
106+
```
107+
108+
Notice that the prompt changes to indicate that you are working inside the container environment.
109+
110+
When you are finished, leave the container with:
111+
112+
```bash
113+
exit
114+
```
115+
116+
This returns you to your normal shell on Torch.
117+
118+
## Pulling Container Images
119+
120+
So far, we have used container images that are already available on Torch under `/share/apps/images`.
121+
122+
In practice, you may also want to run software that is not provided by the cluster. Apptainer can pull images directly from Docker registries and convert them into the Apptainer SIF format.
123+
124+
For example, we can pull the official Ubuntu Docker image:
125+
126+
```bash
127+
apptainer pull ubuntu-24.04.sif docker://ubuntu:24.04
128+
```
129+
130+
During the pull process, Apptainer downloads the Docker image layers and converts them into a single SIF image:
131+
132+
```text
133+
INFO: Converting OCI blobs to SIF format
134+
INFO: Starting build...
135+
...
136+
INFO: Creating SIF file...
137+
```
138+
139+
When the command completes, a new image named `ubuntu-24.04.sif` will be created in the current directory.
140+
141+
You can verify that the image exists:
142+
143+
```bash
144+
ls -lh ubuntu-24.04.sif
145+
```
146+
147+
The image can now be used like any other Apptainer image.
148+
149+
For example:
150+
151+
```bash
152+
apptainer exec ubuntu-24.04.sif cat /etc/os-release
153+
```
154+
155+
or
156+
157+
```bash
158+
apptainer shell ubuntu-24.04.sif
159+
```
160+
161+
## Docker Images and Apptainer
162+
163+
Although Apptainer can pull images directly from Docker registries, it does not run Docker containers directly.
164+
165+
Instead, Apptainer downloads the Docker image layers and converts them into a single Apptainer image file (`.sif`). Once converted, the image can be used without Docker.
166+
167+
This allows researchers to take advantage of the large software ecosystem available through Docker Hub while continuing to use Apptainer on shared HPC systems.
168+
169+
## Torch Images vs Docker Images
170+
171+
Torch already provides many prebuilt container images:
172+
173+
```bash
174+
ls /share/apps/images/
175+
```
176+
177+
For commonly used software, it is often easier to use a cluster-provided image.
178+
179+
If the software you need is not available, you can pull an image directly from Docker Hub and run it with Apptainer.
18180

0 commit comments

Comments
 (0)