Skip to content

Commit dbbfbde

Browse files
mdweisnermdweisner
authored
Merge pull request #171 from NYU-RTS/mdweisner-hpc-connection-patch-1
Update 01_connecting_to_hpc.mdx Incorporated suggestions on phrasing/formatting
2 parents 6073f2b + 9136c8b commit dbbfbde

2 files changed

Lines changed: 103 additions & 52 deletions

File tree

‎docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx‎

Lines changed: 103 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,84 @@ If you'd prefer to access the HPC cluster via a web gateway instead, please refe
1111

1212
The following sections will outline basic ways to connect to the Greene cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines.
1313

14-
:::info Connecting to Torch from outside NYU Network
14+
## Quick Start HPC Connection Guide
15+
### Configuring Your SSH Client
16+
To connect to HPC systems, it's important to configure your machine's SSH client. For Linux and Mac machines, the configuration file is the ~/.ssh/config file on your computer. These are the basic lines that should be added to your ~/.ssh/config file:
17+
```sh
18+
Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu
19+
StrictHostKeyChecking no
20+
ServerAliveInterval 60
21+
ForwardAgent yes
22+
UserKnownHostsFile /dev/null
23+
LogLevel ERROR
24+
25+
Host hpcgwtunnel
26+
HostName gw.hpc.nyu.edu
27+
ForwardX11 no
28+
StrictHostKeyChecking no
29+
LocalForward 8027 greene.hpc.nyu.edu:22
30+
UserKnownHostsFile /dev/null
31+
User <Your NetID>
32+
33+
Host greene
34+
HostName localhost
35+
Port 8027
36+
ForwardX11 yes
37+
StrictHostKeyChecking no
38+
UserKnownHostsFile /dev/null
39+
LogLevel ERROR
40+
User <Your NetID>
41+
```
42+
### Setting Up an SSH Config File on Windows
43+
For Windows, you can now follow a similar configuration setup. Using Powershell you can use the following to create and open a Windows SSH config file:
44+
```sh
45+
# Create the config file with Powershell
46+
New-Item -Path $HOME\.ssh\config -ItemType File
47+
48+
# Open config File with Notepad
49+
C:\WINDOWS\System32\notepad.exe $HOME\.ssh\config
50+
```
51+
Configuring the file above will allow you to use the cmd terminal to connect to Greene.
52+
### Avoiding Man in the Middle Warning.
53+
54+
If you see this warning:
55+
56+
:::warning
57+
58+
```sh
59+
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
60+
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
61+
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
62+
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
63+
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
64+
It is also possible that a host key has just been changed.
65+
```
66+
67+
:::
68+
69+
Do not be alarmed - this is an issue that occurs because the cluster has multiple login nodes (`log-1`, `log-2`, and `log-3`) that `greene.hpc.nyu.edu` resolves to.
1570

71+
To avoid this warning, you can add these lines to your SSH configuration file. Open `~/.ssh/config` and place the following lines in it:
1672

73+
:::tip
74+
This segment, referenced in the recommended config above will nullify the error:
75+
76+
```sh
77+
Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu
78+
StrictHostKeyChecking no
79+
ServerAliveInterval 60
80+
ForwardAgent yes
81+
StrictHostKeyChecking no
82+
UserKnownHostsFile /dev/null
83+
LogLevel ERROR
84+
```
85+
The above will also fix SSH timeout errors by extending the `ServerAliveInterval` argument.
86+
:::
87+
88+
### Connecting to the NYU Network
89+
To access the HPC systems, you must either be on the NYU secure network (such as the NYU WiFi or a wired connection from an NYU office) or use a method of accessing it, such as the NYU VPN or the HPC Gateway servers.
90+
91+
:::info Connecting to the HPC clusters from outside NYU Network
1792
<Tabs>
1893
<TabItem value="VPN" label="VPN (Preferred)">
1994
[Set up your computer to use the NYU VPN][nyu vpn link]. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net
@@ -24,28 +99,40 @@ The following sections will outline basic ways to connect to the Greene cluster.
2499
</Tabs>
25100
:::
26101

27-
You do not need to use the NYU VPN or gateways if you are connected to the NYU network (wired connection in your office or WiFi) or if you have VPN connection initiated. In this case you can ssh directly to the clusters.
28-
29102
## Command Line Interface (with a Terminal)
30103

31104
### Mac & Linux Access
32105

33-
To connect to the gateway servers, simply open a [terminal application][apple terminal link] and enter the following command:
106+
#### VPN Connection
107+
If you're on the VPN or the main NYU network on campus you can SSH directly to greene like so:
34108

35109
```sh
36-
ssh <NetID>@gw.hpc.nyu.edu
110+
ssh <Your NetID>@greene.hpc.nyu.edu
37111
```
112+
113+
#### Gateway / SSH Tunneling Option
114+
115+
To connect to the gateway servers, after you've configured your ~/.ssh/config file, simply open a [terminal application][apple terminal link] and follow these steps:
38116

39-
After typing in your password you will be logged in to the cluster. Once this connection is established, you can make one more hop and connect to one of the HPC clusters:
117+
1. Open a terminal
118+
2. Connect to the gateway via the ssh command:
119+
```sh
120+
ssh hpcgwtunnel
121+
```
122+
3. Authenticate with your password and Duo MFA
40123

124+
4. Open a second terminal
125+
5. Connect to the Greene login nodes via the ssh command:
41126
```sh
42-
# this will connect you to Greene HPC cluster
43-
ssh <NetID>@greene.hpc.nyu.edu
127+
ssh greene
44128
```
129+
6. Authenticate
130+
131+
After typing in your password you will be logged in to the cluster.
45132

46133
### Windows CMD
47134

48-
Windows 11 users have several options. First, the CMD program should contain an ssh client, allowing you to log into Greene or Hudson the same way as with a Linux terminal.
135+
Windows 11 users have several options. First, the CMD program should contain an ssh client, allowing you to log into the HPC systems the same way as with a Linux terminal.
49136

50137
### Windows WSL2
51138

@@ -58,7 +145,9 @@ Instructions on WSL installation can be found here: [https://docs.microsoft.com/
58145
- If you are using WSL 2 (Windows subsystem for Linux), you may not be able to access internet when Cisco AnyConnect VPN, installed from exe file, is activated. A potential solution: uninstall Cisco AnyConnect and install AnyConnect using Microsoft Store, and then setup new VPN connection using settings described on [IT webpage][install vpn on windows link].
59146
:::
60147

61-
## Setting up SSH Keys
148+
## Additional Tips
149+
Below are some additional options and tips for connecting to the HPC resources.
150+
### Setting up SSH Keys
62151

63152
Instead of typing your password every time you need to log in, you can also specify an ssh key.
64153

@@ -73,9 +162,9 @@ Instead of typing your password every time you need to log in, you can also spec
73162

74163
- Next time you will log into cluster no password will be required
75164

76-
For additional recommendations on how to configure your SSH sessions, see the \[ssh configuring and x11 forwarding page].
165+
For additional recommendations on how to configure your SSH sessions, see the [ssh configuring and x11 forwarding page](./02_ssh_tunneling_and_x11_forwarding.md).
77166

78-
## PuTTY (Only for Windows)
167+
### PuTTY (Only for Windows)
79168

80169
There are many SSH clients for Windows OS, but we recommend using [PuTTY SSH][putty link] if you have not already. Once it is installed, launch PuTTY and configure new session "Session" category as in the screenshot below:
81170

@@ -86,8 +175,8 @@ Here we are instructing PuTTY to connect to host gw.hpc.nyu.edu on port 22 using
86175
Once you click "Open", a terminal window with prompt for password will pop up. Enter your NetID password and you should be authorized on the gateway server. Gateways are designed to support only a very minimal set of commands and their only purpose it to let users access HPC systems. Once you are there type in an ssh command that will let you connect to Greene cluster :
87176

88177
```sh
89-
# Greene Login
90-
ssh greene.hpc.nyu.edu
178+
# Gateway Login
179+
ssh gw.hpc.nyu.edu
91180
```
92181

93182
A new command line interface window will open up that prompts you for your password on the gateway server, from there you can connect to Greene by entering the following:

‎docs/hpc/02_connecting_to_hpc/02_ssh_tunneling_and_x11_forwarding.md‎

Lines changed: 0 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -2,44 +2,6 @@
22

33
[xquartz]: https://www.xquartz.org/
44

5-
## Avoiding Man in the Middle Warning.
6-
7-
If you see this warning:
8-
9-
:::warning
10-
11-
```sh
12-
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
13-
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
14-
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
15-
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
16-
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
17-
It is also possible that a host key has just been changed.
18-
```
19-
20-
:::
21-
22-
Do not be alarmed - this is an issue that occurs because the cluster has multiple login nodes (`log-1`, `log-2`, and `log-3`) that `greene.hpc.nyu.edu` resolves to.
23-
24-
To avoid this warning, you can add these lines to your SSH configuration file. Open `~/.ssh/config` and place the following lines in it:
25-
26-
27-
:::tip
28-
29-
```sh
30-
Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu
31-
StrictHostKeyChecking no
32-
ServerAliveInterval 60
33-
ForwardAgent yes
34-
StrictHostKeyChecking no
35-
UserKnownHostsFile /dev/null
36-
LogLevel ERROR
37-
```
38-
39-
:::
40-
41-
The above will also fix SSH timeout errors by extending the `ServerAliveInterval` argument.
42-
435
## SSH Tunneling (Mac, Linux)
446

457
Setting up your workstation for SSH tunneling will make logging in and transferring files significantly easier, and installing and running an X server will allow you to use graphical software on the HPC clusters. X server is a software package that draws on your local screen windows created on a remote computer such as on the remote HPC.

0 commit comments

Comments
 (0)