You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: index.qmd
+3-13Lines changed: 3 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -1,20 +1,10 @@
1
+

2
+
1
3
# Preface {.unnumbered}
2
4
3
5
## About {-}
4
6
5
-
This 5-day in-person workshop will provide researchers with an introduction to advanced topics in computationally reproducible research in python, including software and techniques for working with very large datasets. This includes working in cloud computing environments, docker containers, and parallel processing using tools like parsl and dask. The workshop will also cover concrete methods for documenting and uploading data to the Arctic Data Center, advanced approaches to tracking data provenance, responsible research and data management practices including data sovereignty and the CARE principles, and ethical concerns with data-intensive modeling and analysis.
6
-
7
-
8
-

9
-
10
-
<br>
11
-
12
-
### Schedule {-}
13
-
14
-
:::{.column-page}
15
-

16
-
:::
17
-
7
+
This one hour overview will provide researchers with a quick introduction to working in remote server computing environments, and some basics of parallel processing in the shell.
Copy file name to clipboardExpand all lines: sections/remote-computing.qmd
+30Lines changed: 30 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -457,6 +457,36 @@ echo "DONE!"
457
457
```
458
458
:::
459
459
460
+
## Shell tips and tricks
461
+
462
+
When you're working
463
+
464
+
### Being nice {{< fa heart >}}
465
+
466
+
When working on a shared server, it's important to be considerate of other users who may also be using the server's resources. Here are some tips for being a good citizen when working on a shared server:
467
+
468
+
- Check the server's resource usage before running a computationally intensive task. You can use commands like `htop` or `glances`to see how much CPU and memory is being used by other processes on the server
469
+
- Use `nice` to run your processes with a lower priority, which can help prevent your processes from hogging resources and slowing down the server for other users. For example, you can run a command with `nice` like this: `nice -n 10 my_command`, where `-n 10` sets the priority level to 10 (the default is 0, and higher numbers indicate lower priority)
470
+
471
+
### Don't hangup {{< fa phone >}}
472
+
473
+
When working on a remote server, it's important to be aware that your connection may be interrupted for various reasons (e.g. network issues, server maintenance, etc.), or maybe you just exit your terminal. When you do, the processes you were running will also be terminated. Here are some tips for being persistent when working on a remote server.
474
+
475
+
- If you're running a long-running process, consider using `nohup` to run it in the background, which allows you to disconnect from the server without interrupting the process. For example:
476
+
-`nohup my_command &`
477
+
478
+
What that does is to run the program `my_command` in the background (the `&` operator indicates to put it in the background) and to ignore the hangup signal (the `nohup` command), which allows the process to continue running even if your connection to the server is interrupted. The output of the command will be saved to a file called `nohup.out` in the current directory.
479
+
480
+
Alternatively, you can redirect the output to a specific file like this:
481
+
-`nohup my_command > output.log 2>&1 &`
482
+
483
+
That redirects the standard output (stdout) to `output.log` and the standard error (stderr) to the same file (`2>&1` means "redirect stderr to the same location as stdout"). This way, you can keep track of both the normal output and any error messages generated by `my_command` in a single log file. You might instead choose to redirect stdout and stderr to separate files for easier debugging, like this:
484
+
-`nohup my_command > output.log 2> error.log &`
485
+
486
+
### Persistent sessions with tmux {{< fa laptop >}}
0 commit comments