You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx
+8-65Lines changed: 8 additions & 65 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,9 @@
1
-
importTabsfrom'@theme/Tabs';
2
-
importTabItemfrom'@theme/TabItem';
3
-
4
1
# Connecting to the HPC Cluster
5
2
3
+
:::tip
4
+
This page gives an overview of connecting to the HPC cluster, for a tutorial on this topic, head to the [this section](../12_tutorial_intro_shell_hpc/02_connecting_to_hpc.mdx)!
5
+
:::
6
+
6
7
There are several ways to interact with the Greene HPC cluster. Similar to other Linux clusters, the most common method of connection is via a Command Line Interface (CLI). A CLI is a program that allows you to create and delete files, run programs, and navigate through directories and files via a typed prompt. On Mac, the built-in CLI application is called Terminal. While Windows 11 machines support a Linux Subsystem, which allows for similar functionality, a popular tool used to connect to a Linux server is a free application called [PuTTY][putty link].
7
8
8
9
:::tip
@@ -15,21 +16,13 @@ The following sections will outline basic ways to connect to the Greene cluster.
15
16
### Configuring Your SSH Client
16
17
To connect to HPC systems, it's important to configure your machine's SSH client. For Linux and Mac machines, the configuration file is the ~/.ssh/config file on your computer. These are the basic lines that should be added to your ~/.ssh/config file:
@@ -86,17 +79,10 @@ The above will also fix SSH timeout errors by extending the `ServerAliveInterval
86
79
:::
87
80
88
81
### Connecting to the NYU Network
89
-
To access the HPC systems, you must either be on the NYU secure network (such as the NYU WiFi or a wired connection from an NYU office) or use a method of accessing it, such as the NYU VPN or the HPC Gateway servers.
82
+
To access the HPC systems, you must either be on the NYU secure network (such as the NYU WiFi or a wired connection from an NYU office) or use the NYU VPN.
90
83
91
84
:::info Connecting to the HPC clusters from outside NYU Network
92
-
<Tabs>
93
-
<TabItemvalue="VPN"label="VPN (Preferred)">
94
-
[Set up your computer to use the NYU VPN][nyu vpn link]. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net
95
-
</TabItem>
96
-
<TabItemvalue="Gateway"label="Gateway">
97
-
Go through our gateway servers (example below). Gateways are designed to support only a very minimal set of commands and their only purpose is to let users connect HPC systems without needing to first connect to the VPN
98
-
</TabItem>
99
-
</Tabs>
85
+
[Set up your computer to use the NYU VPN][nyu vpn link]. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net
100
86
:::
101
87
102
88
## Command Line Interface (with a Terminal)
@@ -110,26 +96,6 @@ If you're on the VPN or the main NYU network on campus you can SSH directly to g
110
96
ssh <Your NetID>@greene.hpc.nyu.edu
111
97
```
112
98
113
-
#### Gateway / SSH Tunneling Option
114
-
115
-
To connect to the gateway servers, after you've configured your ~/.ssh/config file, simply open a [terminal application][apple terminal link] and follow these steps:
116
-
117
-
1. Open a terminal
118
-
2. Connect to the gateway via the ssh command:
119
-
```sh
120
-
ssh hpcgwtunnel
121
-
```
122
-
3. Authenticate with your password and Duo MFA
123
-
124
-
4. Open a second terminal
125
-
5. Connect to the Greene login nodes via the ssh command:
126
-
```sh
127
-
ssh greene
128
-
```
129
-
6. Authenticate
130
-
131
-
After typing in your password you will be logged in to the cluster.
132
-
133
99
### Windows CMD
134
100
135
101
Windows 11 users have several options. First, the CMD program should contain an ssh client, allowing you to log into the HPC systems the same way as with a Linux terminal.
@@ -164,30 +130,7 @@ Instead of typing your password every time you need to log in, you can also spec
164
130
165
131
For additional recommendations on how to configure your SSH sessions, see the [ssh configuring and x11 forwarding page](./02_ssh_tunneling_and_x11_forwarding.md).
166
132
167
-
### PuTTY (Only for Windows)
168
-
169
-
There are many SSH clients for Windows OS, but we recommend using [PuTTY SSH][putty link] if you have not already. Once it is installed, launch PuTTY and configure new session "Session" category as in the screenshot below:
Here we are instructing PuTTY to connect to host gw.hpc.nyu.edu on port 22 using SSH protocol (note, that this interface allows you to save this connection configuration for future). Just like for Linux and Mac users, if you are connecting from the outside of NYU network, you need to go through the gateway servers.
174
-
175
-
Once you click "Open", a terminal window with prompt for password will pop up. Enter your NetID password and you should be authorized on the gateway server. Gateways are designed to support only a very minimal set of commands and their only purpose it to let users access HPC systems. Once you are there type in an ssh command that will let you connect to Greene cluster :
176
-
177
-
```sh
178
-
# Gateway Login
179
-
ssh gw.hpc.nyu.edu
180
-
```
181
-
182
-
A new command line interface window will open up that prompts you for your password on the gateway server, from there you can connect to Greene by entering the following:
Copy file name to clipboardExpand all lines: docs/hpc/02_connecting_to_hpc/02_ssh_tunneling_and_x11_forwarding.md
-88Lines changed: 0 additions & 88 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,20 +13,6 @@ Linux users have X set up already. Mac users can download and install [XQuartz][
13
13
To avoid repeatedly setting up a tunnel, you can write the details of the tunnel into your SSH configuration file. Using your favorite editor, open the file `~/.ssh/config` and place the following lines in it:
14
14
15
15
```sh
16
-
# first we create the tunnel, with instructions to pass incoming
17
-
# packets on ports 8027 and 8028 through it and to specific locations
18
-
19
-
Host hpcgwtunnel
20
-
HostName gw.hpc.nyu.edu
21
-
ForwardX11 no
22
-
StrictHostKeyChecking no
23
-
LocalForward 8027 greene.hpc.nyu.edu:22
24
-
UserKnownHostsFile /dev/null
25
-
User <Your NetID>
26
-
27
-
# next we create an alias for incoming packets on the port
28
-
# The alias corresponds to where the tunnel forwards these packets
29
-
30
16
Host greene
31
17
HostName localhost
32
18
Port 8027
@@ -52,80 +38,6 @@ chmod 700 $HOME/.ssh
52
38
chmod 600 ~/.ssh/*
53
39
```
54
40
55
-
### Start the tunnel
56
-
57
-
To create the tunnel, ssh to it with the following command:
58
-
```sh
59
-
ssh hpcgwtunnel
60
-
```
61
-
62
-
:::tip
63
-
64
-
You must leave this window open for the tunnel to remain open. It is best to start a new terminal window for subsequent logins.
65
-
66
-
:::
67
-
68
-
### Log in via the tunnel
69
-
70
-
Open a new terminal window and use ssh to log in to the cluster, as shown below.
71
-
72
-
```sh
73
-
ssh greene
74
-
```
75
-
76
-
Note that you must use the short name defined above in your .ssh/config file, not the fully qualified domain name:
77
-
78
-
**Creating a once-off tunnel.**
79
-
80
-
Alternatively, you can set up a once-off tunnel without editing .ssh/config by running the following command:
81
-
82
-
```sh
83
-
ssh -L 8027:greene:22 NetID@gw.hpc.nyu.edu # to set up a tunnel
84
-
ssh -Y -p 8027 NetID@localhost
85
-
```
86
-
87
-
This is the equivalent to running "ssh hpcgwtunnel" in the reusable tunnel instructions, but the port forwarding is specified on the command line.
88
-
89
-
90
-
## Tunneling (Windows)
91
-
92
-
### Creating the tunnel
93
-
94
-
1. First open Putty and prepare to log in to gw.hpc.nyu.edu. If you saved your session during that process, you can load it by selecting from the "Saved Sessions" box and hitting "Load". Don't hit "Open" yet!
95
-
96
-
2. Under "Connection" -> "SSH", just below "X11", select "Tunnels
97
-
98
-
3. Write "8026" (the port number) in the "Source port" box, and "greene.hpc.nyu.edu:22" (the machine you wish to tunnel to - 22 is the port that ssh listens on) in the "Destination" box
99
-
100
-
4. Click "Add". You can repeat step 3 with a different port number and a different destination. If you do this you will create multiple tunnels, one to each destination
101
-
102
-
5. Before hitting "Open", go back to the "Sessions" page, give the session a name ("hpcgw_tunnel") and hit "Save". Then next time you need not do all this again, just load the saved session
103
-
104
-
6. Hit "Open" to login in to gw.hpc.nyu.edu and create the tunnel. A terminal window will appear, asking for your login name (NYU NetID) and password (NYU password). Windows may also ask you to allow certain connections through its firewall - this is so you can ssh to port 8026 on your workstation - the entrance to the tunnel
105
-
106
-
107
-
:::note
108
-
You can add other NYU hosts to the tunnel by adding a new source port and destination and clicking "Add". For example, you could add "Source port = 8025" and "Destination = EXAMPLE.hpc.nyu.edu:22", then press "Add". You would then perform Step 2 (below) twice - once for greene on port 8026 and once for an example server on port 8025.
109
-
:::
110
-
111
-
Using your SSH tunnel: To log in via the tunnel, first the tunnel must be open. If you've just completed Step 1, it will be open and you can jump down to "Step 2: Logging in via your SSH tunnel". If you completed Step 1 yesterday, and now want to re-use the tunnel you created, first start the tunnel:
112
-
113
-
Starting the tunnel: During a session, you need only do this once - as long as the tunnel is open, new connections will go over it.
114
-
115
-
1. Start Putty.exe (again, if necessary), and load the session you saved in settings during procedure above
116
-
117
-
2. Hit "Open", and log in to the bastion host with your NYU NetID and password. This will create the tunnel.
118
-
119
-
### Logging in via your SSH tunnel
120
-
121
-
1. Start the second Putty.exe. In the "Host Name" box, write "localhost" and in the "Port" box, write "8026" (or whichever port number you specified when you set up the tunnel in the procedure above). We use "localhost" because the entrance of the tunnel is actually on this workstation, at port 8026
122
-
123
-
2. Go to "Connections" -> "SSH" -> "X11" and check "Enable X11 forwarding"
124
-
125
-
3. Optionally, give this session a name (in "Saved Sessions") and hit "Save" to save it. Then next time instead of steps 1 and 2 you can simply load this saved session
126
-
127
-
4. Hit "Open". You will again get a terminal window asking for your login (NYU NetID) and password (NYU password). You are now logged in to the HPC cluster!
128
-
129
41
## X11 Forwarding
130
42
131
43
In rare cases when you need to interact with GUI applications on HPC clusters, you need to enable X11 forwarding for your SSH connection. Mac and Linux users will need to run the ssh commands described above with an additional flag:
Copy file name to clipboardExpand all lines: docs/hpc/12_tutorial_intro_shell_hpc/02_connecting_to_hpc.mdx
+7-38Lines changed: 7 additions & 38 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ Objectives:
11
11
:::
12
12
13
13
:::warning[Prerequisites]
14
-
To access the Greene HPC cluster, you must be connected to the NYU network. If you are physically on campus and connected via a wired connection in your office or through NYU’s WiFi, you can directly SSH into the clusters without any additional steps. However, if you are off-campus or working remotely, connecting through the NYU VPN or using the gateway servers is required to establish a secure connection to the HPC systems.
14
+
To access the Greene HPC cluster, you must be connected to the NYU network. If you are physically on campus and connected via a wired connection in your office or through NYU’s WiFi, you can directly SSH into the clusters without any additional steps.
15
15
:::
16
16
17
17
## Secure Connections
@@ -25,38 +25,22 @@ SSH clients are usually command-line tools, where you provide the remote machine
25
25
26
26
When logging in to a laptop, tablet, or other personal device, a username, password, or pattern are normally required to prevent unauthorized access. In these situations, the likelihood of somebody else intercepting your password is low, since logging your keystrokes requires a malicious exploit or physical access. For systems like log-1 running an SSH server, anybody on the network can log in, or try to. Since usernames are often public or easy to guess, your password is often the weakest link in the security chain. Many clusters therefore forbid password-based login, requiring instead that you generate and configure a public-private key pair with a much stronger password. Even though Greene does not require the use SSH keys to login, please consider using the instructions below to use them. It will make for quicker and more secure connections with Greene.
27
27
28
-
## Remote Connections with the NYU VPN & HPC Gateway Server
29
-
If you are connecting from a remote location that is not on the NYU network (your home for example), you have two options:
30
-
31
-
-**VPN Option**: set up your computer to use the NYU VPN. Once you’ve created a VPN connection, you can proceed as if you were connected to the NYU net.
32
-
-**Gateway Option**: go through our gateway servers (example below). Gateways are designed to support only a very minimal set of commands and their only purpose is to let users connect to HPC systems without needing to first connect to the VPN.
28
+
## Remote Connections with the NYU VPN
29
+
If you are connecting from a remote location that is not on the NYU network (your home for example), you have have to set up your computer to use the NYU VPN. Once you’ve created a VPN connection, you can proceed as if you were connected to the NYU net.
33
30
34
31
## Connect to the NYU VPN
35
32
To connect to VPN, please see the [NYU instructions](https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html).
36
33
37
34
## Log into the Greene Cluster
38
35
39
36
### Inside the NYU network (non-Windows)
40
-
From within the NYU network, that is, from an on-campus location, or after you VPN inside NYU’s network, you can log in to the HPC clusters directly. You do not need to log in to the gateway host.
37
+
From within the NYU network, that is, from an on-campus location, or after you VPN inside NYU’s network, you can log in to the HPC clusters directly.
41
38
42
39
To log in to the HPC cluster (Greene), simply use:
43
40
```bash
44
41
ssh <NetID>@greene.hpc.nyu.edu
45
42
```
46
43
47
-
### Outside the NYU network (non-Windows)
48
-
From an off-campus location (outside NYU-NET), logging in to the HPC clusters is a two-step process:
49
-
50
-
1. log in to the gateway host, `gw.hpc.nyu.edu`. From a Mac or Linux workstation, this is a simple terminal command (replace `<NetID>` with your NetID). Your password is the same password you use for NYU Home:
51
-
```bash
52
-
ssh <NetID>@gw.hpc.nyu.edu
53
-
```
54
-
55
-
1. log in to the cluster. For Greene, this is done with:
56
-
```bash
57
-
ssh <NetID>@greene.hpc.nyu.edu
58
-
```
59
-
60
44
### From Windows
61
45
Windows users will need to use PuTTY, please see the [NYU instructions](https://www.nyu.edu/employees/resources-and-services/media-and-communications/digital-communications/web-guide/website-development/tutorials/making-your-initial-connection/connecting-with-putty.html).
62
46
@@ -71,11 +55,7 @@ Here is the process for opening a terminal in each operating system.
71
55
### Linux
72
56
There are many different versions (aka “flavours”) of Linux and how to open a terminal window can change between flavours. Fortunately most Linux users already know how to open a terminal window since it is a common part of the workflow for Linux users. If this is something that you do not know how to do then a quick search on the Internet for “how to open a terminal window in” with your particular Linux flavour appended to the end should quickly give you the directions you need.
73
57
74
-
To connect to the gateway servers, simply open a terminal application and enter the following command:
75
-
```bash
76
-
ssh <NetID>@gw.hpc.nyu.edu
77
-
```
78
-
After typing in your password you will be logged in to the cluster. Once this connection is established, you can make one more hop and connect to one of the HPC clusters:
58
+
Open a terminal application and enter the following to connect to one of the HPC clusters:
79
59
```bash
80
60
# this will connect you to Greene HPC cluster
81
61
ssh <NetID>@greene.hpc.nyu.edu
@@ -84,11 +64,7 @@ ssh <NetID>@greene.hpc.nyu.edu
84
64
### Mac
85
65
Macs have had a terminal built in since the first version of OS X, since it is built on a UNIX-like operating system, leveraging many parts from BSD (Berkeley Software Distribution). The terminal can be quickly opened through the use of the Searchlight tool. Hold down the command key and press the spacebar. In the search bar that shows up type “terminal”, choose the terminal app from the list of results (it will look like a tiny, black computer screen) and you will be presented with a terminal window. Alternatively, you can find Terminal under “Utilities” in the Applications menu in the Finder.
86
66
87
-
To connect to the gateway servers, simply open a terminal application and enter the following command:
88
-
```bash
89
-
ssh <NetID>@gw.hpc.nyu.edu
90
-
```
91
-
After typing in your password you will be logged in to the cluster. Once this connection is established, you can make one more hop and connect to Greene:
67
+
Open a terminal application and enter the following to connect to Greene:
0 commit comments