Skip to content
Merged
79 changes: 75 additions & 4 deletions docs/hpc/13_tutorial_intro_shell_hpc/02_connecting_to_hpc.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,17 @@ Objectives:
To access the Greene HPC cluster, you must be connected to the NYU network. If you are physically on campus and connected via a wired connection in your office or through NYU’s WiFi, you can directly SSH into the clusters without any additional steps. However, if you are off-campus or working remotely, connecting through the NYU VPN or using the gateway servers is required to establish a secure connection to the HPC systems.
:::

## Secure Connections
The first step in using a cluster is to establish a connection from your computer to the cluster. When we are sitting at a computer (or standing, or holding it in our hands or on our wrists), we have come to expect a visual display with icons, widgets, and perhaps some windows or applications: a *graphical user interface*, or GUI. Since computer clusters are remote resources that we connect to over slow or intermittent interfaces (WiFi and VPNs especially), it is more practical to use a *command-line interface*, or CLI, to send commands as plain-text. If a command returns output, it is printed as plain text as well. The commands we run today will not open a window to show graphical results.

If you have ever opened the Windows Command Prompt or macOS Terminal, you have seen a CLI. This is the CLI on your *local machine*. The only leap to be made here is to open a CLI on a *remote machine*, while taking some precautions so that other folks on the network can’t see (or change) the commands you’re running or the results the remote machine sends back. We will use the Secure SHell protocol (or SSH) to open an encrypted network connection between two machines, allowing you to send & receive text and data without having to worry about prying eyes.

![connect-to-remote](./static/connect-to-remote.svg)

SSH clients are usually command-line tools, where you provide the remote machine address as the only required argument. If your username on the remote system differs from what you use locally, you must provide that as well. If your SSH client has a graphical front-end, such as PuTTY or MobaXterm, you will set these arguments before clicking “connect.” From the terminal, you’ll write something like ssh userName@hostname, where the argument is just like an email address: the “@” symbol is used to separate the personal ID from the address of the remote machine.

When logging in to a laptop, tablet, or other personal device, a username, password, or pattern are normally required to prevent unauthorized access. In these situations, the likelihood of somebody else intercepting your password is low, since logging your keystrokes requires a malicious exploit or physical access. For systems like log-1 running an SSH server, anybody on the network can log in, or try to. Since usernames are often public or easy to guess, your password is often the weakest link in the security chain. Many clusters therefore forbid password-based login, requiring instead that you generate and configure a public-private key pair with a much stronger password. Even though Greene does not require the use SSH keys to login, please consider using the instructions below to use them. It will make for quicker and more secure connections with Greene.

## Remote Connections with the NYU VPN & HPC Gateway Server
If you are connecting from a remote location that is not on the NYU network (your home for example), you have two options:

Expand Down Expand Up @@ -150,13 +161,13 @@ The link above will automatically search for the Open OnDemand site data and coo

Once done, try navigating again to [https://ood.hpc.nyu.edu](https://ood.hpc.nyu.edu) and the site should load. For other issues please email [hpc@nyu.edu](mailto:hpc@nyu.edu).

## Creating an SSH key (optional)
## SSH keys (optional)
SSH keys are an alternative method for authentication to obtain access to remote computing systems. They can also be used for authentication when transferring files or for accessing version control systems. In this section you will create a pair of SSH keys, a private key which you keep on your own computer and a public key which is placed on the remote HPC system that you will log into.

### Windows
### Creating SSH keys on Windows
We mentioned methods for creating SSH keys using some of the [Windows SSH options above](#windows).

### Linux, Mac and Windows Subsystem for Linux
### Creating SSH keys on Linux, Mac and Windows Subsystem for Linux
Once you have opened a terminal check for existing SSH keys and filenames since existing SSH keys could be overwritten by the following command if the filename is the same. If you already have a key with the name given after the `-f` option you will need to change the filename to keep from losing your existing file.

```bash
Expand Down Expand Up @@ -188,6 +199,55 @@ Protect this key as if it unlocks your front door. In many ways, it does.
For more information on SSH security and some of the flags set here, an excellent resource is [Secure Secure Shell](https://blog.stribik.technology/2015/01/04/secure-secure-shell.html).
:::

### SSH Agent for Easier Key Handling
An SSH key is only as strong as the password used to unlock it, but on the other hand, typing out a complex password every time you connect to a machine is tedious and gets old very fast. This is where the SSH Agent comes in.

Using an SSH Agent, you can type your password for the private key once, then have the Agent remember it for some number of hours or until you log off. Unless some nefarious actor has physical access to your machine, this keeps the password safe, and removes the tedium of entering the password multiple times.

Just remember your password, because once it expires in the Agent, you have to type it in again.

#### SSH Agents on Linux, macOS, and Windows
Open your terminal application and check if an agent is running:
```bash
[user@laptop ~]$ ssh-add -l
```
If you get an error like this one,
:::danger[Error]
Error connecting to agent: No such file or directory
:::
… then you need to launch the agent as follows:
```bash
[user@laptop ~]$ eval $(ssh-agent)
```
:::note[What’s in a $(...)?]
The syntax of this SSH Agent command is unusual, based on what we’ve seen in the UNIX Shell tutorial. This is because the ssh-agent command creates opens a connection that only you have access to, and prints a series of shell commands that can be used to reach it – but *does not execute them*!
```bash
[user@laptop ~]$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-Zvvga2Y8kQZN/agent.131521;
export SSH_AUTH_SOCK;
SSH_AGENT_PID=131522;
export SSH_AGENT_PID;
echo Agent pid 131522;
```
The `eval` command interprets this text output as commands and allows you to access the SSH Agent connection you just created.

You could run each line of the `ssh-agent` output yourself, and achieve the same result. Using `eval` just makes this easier.
:::
- Otherwise, your agent is already running: don’t mess with it.

Add your key to the agent, with session expiration after 8 hours:
```bash
[user@laptop ~]$ ssh-add -t 8h ~/.ssh/id_ed25519
Enter passphrase for .ssh/id_ed25519:
Identity added: .ssh/id_ed25519
Lifetime set to 86400 seconds
```
For the duration (8 hours), whenever you use that key, the SSH Agent will provide the key on your behalf without you having to type a single keystroke.

#### SSH Agent on PuTTY
If you are using PuTTY on Windows, download and use `pageant` as the SSH agent. See the [PuTTY documentation](./02_connecting_to_hpc.mdx#opening-a-terminal).


## Modifying your .ssh/config file
Please add the following lines to your `~/.ssh/config` file:
```
Expand Down Expand Up @@ -229,9 +289,11 @@ Let’s attempt to connect to the HPC system now:

If you'd like to connect without typing your password you'll need to copy your public key file to greene first:
```bash
scp ~/.ssh/id_Greene_rsa.pub <NetID>@greene.hpc.nyu.edu:/home/<NetID>/.ssh/authorized_keys
scp ~/.ssh/id_Greene_rsa.pub <NetID>@greene.hpc.nyu.edu:/home/<NetID>
```

You'll need to log in with your password at least once even if you plan to use SSH keys in the future because we'll need to set up your keys.

If you are on NYU WiFi or VPN you can connect directly with:
```bash
ssh <NetID>@greene.hpc.nyu.edu
Expand Down Expand Up @@ -266,6 +328,15 @@ By looking at the information after *Hostname:* and in the prompt you'll notice

If you logged in using PuTTY this will not apply because it does not offer a local terminal.

### Setting up your SSH keys (optional)

If you copied your SSH keys to your home directory in an earlier step, we'll guide you through setting them up for use now.
1. If you do not have a directory name `.ssh` in your home directory create one now with the command: `mkdir ~/.ssh`
- Remember you can list the hidden (dot) files by running the command: `ls -a`
2. add your key to the list of `authorized_keys` with the command: `cat ~/id_Greene_rsa.pub >> ~/.ssh/authorized_keys`

That’s all! Disconnect, then try to log back into the remote: if your key and agent have been configured correctly, you should not be prompted for the password.

### Telling the Difference between the Local Terminal and the Remote Terminal

You can see that the prompt has changed after you log into a remote system. Let's take a closer look at the prompt after login: `[NetID@log-1 ~]$` (in this example) tells us that we are logged into the login node `log-1` with the identity `NetID`.
Expand Down
Loading