Skip to content

Commit 67f1368

Browse files
committed
Add shell tips section with review of nohup and nice
1 parent d519bbf commit 67f1368

3 files changed

Lines changed: 38 additions & 38 deletions

File tree

_quarto.yml

Lines changed: 5 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -2,45 +2,25 @@ project:
22
type: book
33

44
book:
5-
title: "Scalable and Computationally Reproducible Approaches to Arctic Research"
6-
doi: 10.18739/A2NK3670P
7-
author: "Matthew B. Jones, S. Jeanette Clark, Jim Regetz, Rushiraj Nenuji, Nicole Greco, Angie Garcia, and Justin Kadi"
8-
date: "April 7, 2025"
9-
issued: 2025
5+
title: "Quick review: remote computing"
6+
author: "Matthew B. Jones"
7+
date: "May 7, 2026"
8+
issued: 2026
109
reader-mode: false
1110
sidebar:
1211
title: "Course Materials"
1312
style: "floating"
1413
search: false
1514
tools:
1615
- icon: house-door-fill
17-
href: https://arcticdata.io
18-
- icon: twitter
19-
href: https://twitter.com/arcticdatactr
16+
href: https://nceas.ucsb.edu
2017
- icon: github
2118
href: https://github.com/NCEAS/scalable-computing-course
22-
2319
chapters:
2420
- index.qmd
25-
- sections/adc-intro.qmd
2621
- sections/remote-computing.qmd
2722
- sections/python-intro.qmd
2823
- sections/parallel-programming.qmd
29-
- sections/data-structures-netcdf.qmd
30-
- sections/parallel-with-dask.qmd
31-
- sections/group-project-1.qmd
32-
- sections/data-ethics.qmd
33-
- sections/geopandas.qmd
34-
- sections/parquet-arrow.qmd
35-
- sections/group-project-2.qmd
36-
- sections/software-design-1.qmd
37-
- sections/adc-data-publishing.qmd
38-
- sections/zarr.qmd
39-
- sections/docker-containers.qmd
40-
- sections/software-design-2.qmd
41-
- sections/cloud-scale-data.qmd
42-
- sections/reproducibility-containers.qmd
43-
- sections/docker-hpc-cloud.qmd
4424

4525
bibliography: references.bib
4626

index.qmd

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,10 @@
1+
![](images/nceas-banner.png)
2+
13
# Preface {.unnumbered}
24

35
## About {-}
46

5-
This 5-day in-person workshop will provide researchers with an introduction to advanced topics in computationally reproducible research in python, including software and techniques for working with very large datasets. This includes working in cloud computing environments, docker containers, and parallel processing using tools like parsl and dask. The workshop will also cover concrete methods for documenting and uploading data to the Arctic Data Center, advanced approaches to tracking data provenance, responsible research and data management practices including data sovereignty and the CARE principles, and ethical concerns with data-intensive modeling and analysis.
6-
7-
8-
![](images/arctic-data-center.png)
9-
10-
<br>
11-
12-
### Schedule {-}
13-
14-
:::{.column-page}
15-
![](images/schedule.png)
16-
:::
17-
7+
This one hour overview will provide researchers with a quick introduction to working in remote server computing environments, and some basics of parallel processing in the shell.
188

199
### Code of Conduct {-}
2010

sections/remote-computing.qmd

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -457,6 +457,36 @@ echo "DONE!"
457457
```
458458
:::
459459

460+
## Shell tips and tricks
461+
462+
When you're working
463+
464+
### Being nice {{< fa heart >}}
465+
466+
When working on a shared server, it's important to be considerate of other users who may also be using the server's resources. Here are some tips for being a good citizen when working on a shared server:
467+
468+
- Check the server's resource usage before running a computationally intensive task. You can use commands like `htop` or `glances`to see how much CPU and memory is being used by other processes on the server
469+
- Use `nice` to run your processes with a lower priority, which can help prevent your processes from hogging resources and slowing down the server for other users. For example, you can run a command with `nice` like this: `nice -n 10 my_command`, where `-n 10` sets the priority level to 10 (the default is 0, and higher numbers indicate lower priority)
470+
471+
### Don't hangup {{< fa phone >}}
472+
473+
When working on a remote server, it's important to be aware that your connection may be interrupted for various reasons (e.g. network issues, server maintenance, etc.), or maybe you just exit your terminal. When you do, the processes you were running will also be terminated. Here are some tips for being persistent when working on a remote server.
474+
475+
- If you're running a long-running process, consider using `nohup` to run it in the background, which allows you to disconnect from the server without interrupting the process. For example:
476+
- `nohup my_command &`
477+
478+
What that does is to run the program `my_command` in the background (the `&` operator indicates to put it in the background) and to ignore the hangup signal (the `nohup` command), which allows the process to continue running even if your connection to the server is interrupted. The output of the command will be saved to a file called `nohup.out` in the current directory.
479+
480+
Alternatively, you can redirect the output to a specific file like this:
481+
- `nohup my_command > output.log 2>&1 &`
482+
483+
That redirects the standard output (stdout) to `output.log` and the standard error (stderr) to the same file (`2>&1` means "redirect stderr to the same location as stdout"). This way, you can keep track of both the normal output and any error messages generated by `my_command` in a single log file. You might instead choose to redirect stdout and stderr to separate files for easier debugging, like this:
484+
- `nohup my_command > output.log 2> error.log &`
485+
486+
### Persistent sessions with tmux {{< fa laptop >}}
487+
488+
`tmux` Tutorial: https://hamvocke.com/blog/a-quick-and-easy-guide-to-tmux/
489+
460490

461491
## {{< fa cloud >}} What is cloud computing anyways?
462492

0 commit comments

Comments
 (0)