Skip to content

Commit 8cbf593

Browse files
Spring school 2026 (#140)
* update best practices python/r * update data transfer manuals * update running ollama manual * include spring school page * Apply suggestions from code review Co-authored-by: Jelle Treep <40466121+jelletreep@users.noreply.github.com> * revise workshop page * update python environments --------- Co-authored-by: Jelle Treep <40466121+jelletreep@users.noreply.github.com> Co-authored-by: Jelle Treep <h.j.treep@uu.nl>
1 parent c35ced0 commit 8cbf593

7 files changed

Lines changed: 310 additions & 51 deletions

File tree

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
---
2+
title: "HPC Spring School June 2026: SURF Research Cloud Workshop"
3+
---
4+
5+
## Overview
6+
7+
Welcome to the SURF Research Cloud workshop for the UU HPC Spring School 2026!
8+
9+
During this hands-on workshop, you'll learn how to work effectively with SURF Research Cloud. There are several tutorials available for you to explore:
10+
11+
| Topic | Section | Estimated time |
12+
| --- | --- | --- |
13+
| **Working with R/Python environments** | [Go to section](#working-with-r-python-environments) | 15-45 minutes |
14+
| **Data transfer methods** | [Go to section](#data-transfer-methods) | 15-45 minutes |
15+
| **Working with Ollama** | [Go to section](#working-with-ollama) | 20-40 minutes |
16+
| **Running long jobs** | [Go to section](#running-long-jobs-via-the-command-line) | 10-30 minutes |
17+
| **VS Code Remote-SSH** | [Go to section](#connecting-vs-code-to-research-cloud) | 30 minutes |
18+
19+
Each tutorial includes step-by-step instructions. You can choose the ones most relevant to your research needs or interests. While there are several workspaces available that you can use to follow the tutorials, we recommend using the [VRE Lab](../workspaces/programming/vre-lab.qmd) workspace for most of them, as it comes with many relevant tools pre-installed.
20+
21+
## Prerequisites
22+
23+
**Required**
24+
25+
Before starting the tutorials, you need:
26+
27+
1. **SURF Research Cloud account**: You can login here [**Research Cloud portal**](https://portal.live.surfresearchcloud.nl/)
28+
2. **Active workspace**: You will need to create a workspace, depending on the tutorial you wish to follow. See [How to Create a Workspace](../first-steps.qmd#create-a-workspace) for guidance.
29+
30+
**Recommended**
31+
32+
Familiarity with the command line (terminal), git, GitHub and basic knowledge of Python/R will be helpful for some tutorials.
33+
34+
## Workshop Tutorials
35+
36+
### 1. Working with R/Python Environments {#working-with-r-python-environments}
37+
38+
Learn how to create, manage, and use Python and R environments in your workspace. Working with environments is good practice for creating robust and reproducible projects, and makes it easier to install the exact same package versions that you are using currentlyon SURF Research Cloud.
39+
40+
**What you'll learn:**
41+
42+
- Creating virtual environments and managing packages
43+
- Creating requirements files for Python and R
44+
- Installing packages on SURF Research Cloud using `uv`, `conda`, and `renv`
45+
46+
**Tutorials:**
47+
48+
- [Working with Python Environments and Best Practices](../manuals/python-environment.qmd)
49+
- [Working with R Environments and Best Practices](../manuals/r-environment.qmd)
50+
51+
We recommend using the [VRE Lab](../workspaces/programming/vre-lab.qmd) workspace to follow these tutorial. Select `Python tools` (for working with Python Environments) and/or `RStudioUU` (for working with R Environments) under optional components in the last step of creating your workspace.
52+
53+
**Estimated time:** 15-45 minutes (depending on any previous experience with environment management tools)
54+
55+
---
56+
57+
### 2. Data Transfer Methods {#data-transfer-methods}
58+
59+
Transfer data to your Research Cloud workspace from your local computer, cloud storage, Yoda and/or GitHub. Learn which method works best for your data source and file sizes.
60+
61+
**What you'll learn:**
62+
63+
- Transferring files from your PC/laptop
64+
- Configuring connections to cloud storage such as Surfdrive, Researchdrive and/or Yoda
65+
- Choosing the right transfer method for your needs
66+
67+
**Tutorials:**
68+
69+
To find the best method to transfer your data, check out this [Interactive Data Transfer Tool.](../manuals/data-transfer-options.qmd) It will help you determine the best method based on your data source, workspace, and file sizes.
70+
71+
You can explore the following tutorials for specific transfer methods:
72+
73+
- [Data Transfer to and from Yoda using iBridges](../manuals/ibridges.qmd)
74+
- [Data Transfer to and from Yoda using iCommands](../manuals/icommands.qmd)
75+
- [Data Transfer from your computer using scp and rsync](../manuals/ssh-data-transfer-methods.qmd)
76+
- [Data Transfer between Research Cloud and Online Storage (Surfdrive, Researchdrive) using Rclone](../manuals/rclone-researchcloud.qmd)
77+
- [Data Transfer from GitHub/GitLab using Git](../manuals/git-clone.qmd)
78+
79+
**Estimated time:** 15-45 minutes (depending on tutorial selected)
80+
81+
---
82+
83+
### 3. Working with Ollama {#working-with-ollama}
84+
85+
Set up and use [Ollama](https://ollama.com/) directly in your Jupyter notebooks, e.g. for data extraction from documents or image analysis.
86+
87+
**What you'll learn:**
88+
89+
- Installing and configuring Ollama
90+
- Downloading models
91+
- Data extraction and Image description in Jupyter notebooks
92+
93+
**Relevant Workspaces:** [VRE Lab](../workspaces/programming/vre-lab.qmd) with `Python tools` as optional component, [Python Workbench Desktop](../workspaces/programming/python-workbench.qmd)
94+
95+
**Tutorial:** [Setting up and Using Ollama](../manuals/ollama.qmd)
96+
97+
**Estimated time:** 20-40 minutes
98+
99+
---
100+
101+
### 4. Running Long Jobs via the Command Line {#running-long-jobs-via-the-command-line}
102+
103+
Keep computational tasks running even after (accidentally) closing your terminal or SSH connection. Essential for long-running analyses and simulations. It is also possible to automate pausing of the workspace to save costs.
104+
105+
**What you'll learn:**
106+
107+
- Running jobs in the background
108+
- Checking job status and output
109+
- Automating workspace pausing
110+
111+
**Tutorial:**
112+
113+
- [Running Long Jobs](../manuals/long-jobs.qmd)
114+
This tutorial is relevant if you have an analysis script or simulation that takes a long time to execute.
115+
116+
**Estimated time:** 10-30 minutes
117+
118+
---
119+
120+
### 5. Connecting VS Code to Research Cloud {#connecting-vs-code-to-research-cloud}
121+
122+
Connect Visual Studio Code on your PC to your SURF Research Cloud workspace. This is useful e.g. if you want to use the VS Code extensions that you have installed.
123+
124+
**What you'll learn:**
125+
126+
- Setting up VS Code Remote-SSH extension
127+
- Connecting to your workspace
128+
- Editing files remotely
129+
- Using VS Code extensions on the remote workspace
130+
131+
Works on any workspace with SSH access, however [VRE Lab](../workspaces/programming/vre-lab.qmd) is recommended.
132+
133+
**Tutorial:**
134+
135+
- [VS Code Remote-SSH](../manuals/vscode-to-researchcloud.qmd)
136+
137+
**Estimated time:** 30 minutes
138+
139+
---

docs/manuals/ibridges.qmd

Lines changed: 35 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: Using iBridges on SURF Research Cloud
33
---
44

5-
The instructions below describe the how to setup iBridges to transfer data between SURF Research Cloud and Yoda or any other iRODS instance. It is possible to use iBridges in three different ways:
5+
The instructions below describe how to setup iBridges to transfer data between SURF Research Cloud and Yoda or any other iRODS instance. It is possible to use iBridges in three different ways:
66

77
- as a python package
88
- via the command line interface (CLI)
@@ -18,14 +18,42 @@ The python package and the command line interface are available (or can be made
1818

1919
### Prerequisites
2020

21-
Access to Yoda or another iRODS instance which hosts your data.
21+
Access to [Yoda](https://www.uu.nl/en/research/yoda) or another iRODS instance which hosts your data.
2222

23-
iBridges needs to be installed on your workspace. Many SURF ResearchCloud workspaces developed by Utrecht University (such as Python and R workbenches) have iBridges preinstalled.
23+
iBridges needs to be installed on your workspace. Most workspaces developed by Utrecht University (such as [Python](../workspaces/programming/python-workbench.qmd) and [R](../workspaces/programming/r-workbench.qmd) workbenches, [VRE Lab](../workspaces/programming/vre-lab.qmd)) have iBridges preinstalled. On workspaces developed by SURF, ibridges may need to be installed by the user.
2424

2525
- If you are working on a Desktop workspace, you can check by logging in to your workspace, clicking 'applications' in the top left corner, and clicking 'Development'.
2626
- On non-desktop workspaces check if iBridges is installed by typing: `ibridges` in the terminal. If iBridges is installed, this should show you the program's help menu. On Jupyter or Rstudio workspaces, you need to open a terminal first: In Jupyter, click the + button in the file browser and select the terminal in the new Launcher tab ([find a short video here](https://jupyterlab.readthedocs.io/en/stable/user/terminal.html)).
2727
In Rstudio, In the bottom left panel, click the ‘terminal’ tab.
2828

29+
30+
#### What shall I do if iBridges is not on my workspace?
31+
32+
If your workspace has python installed, you can install the packages using the command line (or aka terminal):
33+
34+
::: {.panel-tabset}
35+
36+
## GUI
37+
```bash
38+
pip install ibridgesgui
39+
```
40+
41+
To start the GUI application you would then need to open a terminal and type in `ibridges gui` (Note: this will only work on Desktop workspaces).
42+
43+
If you run into problems with running ibridgesgui, try running `sudo apt install libxcb-cursor0` in the terminal and then running `ibridges gui` again. Please contact us if you need help with installing iBridges or if you want to have iBridges preinstalled on your workspace.
44+
45+
## CLI/API
46+
```bash
47+
uv tool install ibridges
48+
# OR
49+
pipx install ibridges
50+
```
51+
ibridges will then be installed. However it is possible that you may need to run `pipx ensurepath` to make it available on your path and restart your terminal before you can use the `ibridges` command.
52+
53+
Please contact us if you need help with installing iBridges or if you want to have iBridges preinstalled on your workspace.
54+
:::
55+
56+
2957
### Configure iBridges
3058

3159
To connect to a Yoda or any other iRODS instance you will need a so-called `irods_environment.json` which needs to be stored in a specific location. You can create this file by hand or use the iBridges GUI or CLI to create it for you.
@@ -40,37 +68,25 @@ To connect to a Yoda or any other iRODS instance you will need a so-called `irod
4068

4169
If there are no templates in the drop-down menu of the configuration window, you will need to install the plugin. To this end you will need python and git and execute in power shell, git bash or Anaconda terminal:
4270

43-
```
71+
``` bash
4472
pip install ibridges-servers-uu
4573
```
4674

4775
#### Configuration through the CLI
4876

4977
If you want to connect to a server hosted by Utrecht University please check whether this information is already present for ibridges.
5078

51-
```
79+
``` bash
5280
ibridges setup --list
5381
```
5482

5583
From that list simply choose the server you want to connect to with `ibridges setup uu-<server>` and follow the instructions.
5684
In case you get the message `No server information was found.` please install:
5785

58-
```
86+
``` bash
5987
pip install bridges-servers-uu
6088
```
6189

62-
#### What shall I do if iBridges is not on my workspace?
63-
64-
If your workspace has python installed, you can install the packages using the command line (or aka terminal):
65-
66-
- GUI: `pip install ibridgesgui`
67-
- CLI/API: `pip install ibridges`
68-
69-
To start the GUI application you would then need to open a terminal and type in `ibridges-gui`. (Note: this will only work on Desktop workspaces).
70-
71-
Please contact us if you need help with installing iBridges or if you want to have iBridges preinstalled on your workspace.
72-
73-
7490
#### How to connect to a Yoda or iRODS instance not provided by Utrecht University?
7591

7692
In this case you will have to create your irods_environment.json by hand:
@@ -97,7 +113,7 @@ More elaborate instructions can be found in the [iBridges GUI documentation](htt
97113

98114
Simply try `ibridges --help` or `ibridges <command> --help` to get examples and help for the usage.
99115

100-
For instructions on how to use the `ibridges` command please see the [iBridges documentation](https://ibridges.readthedocs.io/en/stable/cli.html).
116+
For instructions on how to use the `ibridges` command please see the [iBridges CLI documentation](https://ibridges.readthedocs.io/en/stable/cli.html).
101117

102118
#### Using the python package
103119

docs/manuals/icommands.qmd

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,25 @@
22
title: "Installing and using iCommands on Surf Research Cloud"
33
---
44

5-
The instructions below describe the process of data transfer between Surf Research Cloud and Yoda using iCommands. This manual is developed for workspaces based on a Linux operating system (including the Jupyter Notebook and RStudio workspaces). Transfer is also possible via the Webdav protocol using [Rclone](https://rclone.org). For large files iCommands is faster due to parallel transfer options.
5+
The instructions below describe the process of data transfer between Surf Research Cloud and [Yoda](https://www.uu.nl/en/research/yoda) using iCommands. This manual is developed for workspaces based on a Linux operating system (such as Ubuntu and Python Workspaces as well as Jupyter Notebook and RStudio workspaces). Transfer is also possible via the Webdav protocol using [Rclone](https://rclone.org). However, for large files iCommands is faster due to parallel transfer options.
66

77
### Prerequisites
88
You will need some basic linux skills to be able to use the iCommands successfully.
99
If you don't have these skills, take some time to practice using sections 1, 2, 3 and 7 of this short [online course](https://swcarpentry.github.io/shell-novice) before proceeding. You can practice in the terminal (see step 1).
1010

11+
You will also need access to [Yoda](https://www.uu.nl/en/research/yoda).
12+
1113
### Step 1: Make sure you are in a terminal
12-
- In Jupyterhub (Jupyter Workspace in Research cloud):
14+
- In Jupyterhub ([Jupyter Notebook](../workspaces/programming/jupyter.qmd) or [VRE Lab](../workspaces/programming/vre-lab.qmd) in Research cloud):
1315
To open a new terminal, click the + button in the file browser and select the terminal in the new Launcher tab
1416
([find a short video here](https://jupyterlab.readthedocs.io/en/stable/user/terminal.html)).
15-
- In Rstudio (Rstudio workspace in Research cloud):
17+
- In Rstudio ([Rstudio workspace](../workspaces/programming/rstudio.qmd) in Research cloud):
1618
In the bottom left panel, click the 'terminal' tab.
19+
- In [Python](../workspaces/programming/python-workbench.qmd) and [Ubuntu](../workspaces/plain/ubuntu.qmd) Desktop workspaces:
20+
Open a terminal. You can find it under Applications.
21+
- In [Python](../workspaces/programming/python-workbench.qmd) and [Ubuntu](../workspaces/plain/ubuntu.qmd) CLI workspaces:
22+
You are already in a terminal after logging in to the workspace.
23+
1724

1825
### Step 2: Install iRODS iCommands
1926
Some workspaces have iRODS iCommands preconfigured. This should be visible in step 1 when creating a workspace.
@@ -35,21 +42,27 @@ sudo apt-get install -y irods-icommands
3542

3643
### Step 4: Initialize connection to iRODS
3744

38-
Initialize the connection to the Yoda system as follows:
45+
Navigate out of the `.irods` folder and to a folder where you want to move files from Yoda. Run `cd ~` to navigate to your home directory and then run the following command to initialize the connection to iRODS:
3946

4047
```
4148
iinit
4249
```
4350

44-
Type your Yoda password when requested.
51+
If you see `command not found: iinit` after running the command, it means that iCommands is not properly installed. Run `sudo apt-get install -y irods-icommands` to install iCommands and try again.
52+
53+
After running `iinit`, you will be prompted to enter your Yoda password. You can find this as a [data access password](https://www.uu.nl/en/research/yoda/using-data-access-passwords). This is not the same as your Research Cloud or solis-id password.
4554

4655
Type `ils` to see whether the connection has been established. The output of the `ils` command will list the current Yoda folder and the folders that are located within the current folder.
4756

4857
`iinit` and `ils` are commands which can only be used when icommands is active. Some of these commands are very similar to standard unix commands. When icommands is active you can still operate the HPC system using normal unix commands. A complete list of all 'icommands' can be found [here](https://docs.irods.org/4.2.9/icommands/user/).
4958

5059
### Step 5: Transferring data
5160

52-
Standard transfer of a single file can be done using `iget` and `iput`
61+
Standard transfer of a single file can be done using `iget` (to retrieve from Yoda to your workspace) and `iput` (to upload to Yoda from your workspace).
62+
```
63+
iget /uu/home/<Yoda-folder-name>/<file-name>
64+
iput <workspace-file-name> /uu/home/<Yoda-folder-name>/
65+
```
5366

5467
The `irsync` command can be used to synchronize entire directories between Yoda and Cartesius. The command is used as follows.
5568

0 commit comments

Comments
 (0)