Skip to content

Commit 6e9749e

Browse files
committed
tools & software rework
1 parent 5b4eba2 commit 6e9749e

2 files changed

Lines changed: 20 additions & 31 deletions

File tree

docs/hpc/06_tools_and_software/04_python_packages_with_virtual_environments.mdx

Lines changed: 15 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -30,58 +30,42 @@ Thus you can consider the following options:
3030
:::
3131
::::
3232

33-
Thus you can consider the following options
34-
35-
- Reinstall your packages if some of the files get deleted
36-
- You can do this manually
37-
- You can also do this automatically. For example, within a workflow of pipeline software like [Nextflow](https://www.nextflow.io/)
38-
- Pay for "Research Project Space" - see details on the [Research Project Space](../03_storage/05_research_project_space.mdx) page.
39-
- Use Singularity and install packages within a corresponding overlay file - See instructions on our [Singularity with Conda](../07_containers/03_singularity_with_conda.md) page.
40-
4133
## Create virtual environment
42-
It is advisable to create private environment inside the project directory. This boosts reproducibility and does not use space in `/home/$USER`
43-
44-
### virtualenv
45-
[virtualenv](https://virtualenv.pypa.io/en/latest/) is a tool to create isolated Python environments
46-
47-
Since Python 3.3, a subset of it has been integrated into the standard library under the venv module.
48-
49-
You can create new virtual environment in two ways:
50-
51-
- empty
52-
- inherit all packages from those installed on HPC already (and available in PATH after you load python module)
53-
34+
It is advisable to create private environment inside the project directory. This boosts reproducibility and does not use space in `/home/$USER`:
5435
```sh
5536
## created directory for your project and cd there
5637
mkdir /scratch/$USER/my_project
5738
cd /scratch/$USER/my_project
39+
```
5840

41+
### `virtualenv`
42+
[`virtualenv`](https://virtualenv.pypa.io/en/latest/) is a tool to create isolated Python environments. Since Python 3.3, a subset of it has been integrated into the standard library under the venv module. You can create new virtual environment in two ways:
43+
- empty
44+
```sh
5945
## Create an EMPTY virtual environment
6046
virtualenv venv
61-
47+
```
48+
- inherit all packages from those installed on HPC already (and available in PATH after you load python module)
49+
```sh
6250
## Create an virtual environment that inherits system packages
6351
virtualenv venv --system-site-packages
6452
```
6553

66-
### venv
67-
[venv](https://docs.python.org/3/library/venv.html) is package shipped with Python. It provides subset of options available in virtualenv tool ([link](https://virtualenv.pypa.io/en/latest/)).
54+
### `venv`
55+
[`venv`](https://docs.python.org/3/library/venv.html) is package shipped with Python. It provides subset of options available in virtualenv tool ([link](https://virtualenv.pypa.io/en/latest/)).
6856
```sh
6957
python -m venv venv
7058
```
7159

72-
Create new virtual environment in current directory
73-
60+
Create new virtual environment in current directory:
7461
- empty
75-
- inherit all packages from those installed on HPC already (and available in PATH after you load python module)
7662
```sh
77-
## created directory for your project and cd there
78-
mkdir /scratch/$USER/my_project
79-
cd /scratch/$USER/my_project
80-
##EMPTY
8163
## (use venv command to create environment called "venv")
8264

8365
python -m venv venv
84-
66+
```
67+
- inherit all packages from those installed on HPC already (and available in PATH after you load python module)
68+
```sh
8569
## Inhering all packages
8670
python -m venv venv --system-site-packages
8771
```

docs/hpc/06_tools_and_software/06_conda_environments.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Conda Environments (Python, R)
22

3+
:::tip
4+
This page describes how you can create conda environments without containers in `$SCRATCH`/`/home/$USER`. We strongly encourage you to create conda enviroments within Overlay files attached to Apptainer containers [as decribed in this section](../07_containers/03_singularity_with_conda.md). Please note that creating both kinds of conda environments is strongly discouraged as it leads to packages from one environment being accidentally used in another.
5+
:::
6+
7+
38
## What is Conda?
49
Package, dependency and environment management for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN, and more. Please find more information at [the official documentation page](https://docs.conda.io/en/latest/)
510

0 commit comments

Comments
 (0)