From 8221e8606a63050a3e3d3b9b2f8a64f829f0ba5d Mon Sep 17 00:00:00 2001 From: Michael Weisner Date: Tue, 9 Sep 2025 12:33:10 -0400 Subject: [PATCH 1/6] Update 01_connecting_to_hpc.mdx I've rewritten the connection page to restructure, promote the VPN first, and include our SSH config guide which is useful for new users. --- .../01_connecting_to_hpc.mdx | 79 ++++++++++++++++--- 1 file changed, 69 insertions(+), 10 deletions(-) diff --git a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx index a02c098db7..a4539bdedb 100644 --- a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx +++ b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx @@ -11,9 +11,48 @@ If you'd prefer to access the HPC cluster via a web gateway instead, please refe The following sections will outline basic ways to connect to the Greene cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines. -:::info Connecting to Torch from outside NYU Network +## Quick Start HPC Connection Guide +To connect to HPC systems, it's important to configure your machine's SSH client. For Linux and Mac machines, the configuration file is the ~/.ssh/config file on your computer. These are the basic lines that should be added to your ~/.ssh/config file: +```sh +Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu + StrictHostKeyChecking no + ServerAliveInterval 60 + ForwardAgent yes + UserKnownHostsFile /dev/null + LogLevel ERROR + +Host hpcgwtunnel + HostName gw.hpc.nyu.edu + ForwardX11 no + StrictHostKeyChecking no + LocalForward 8027 greene.hpc.nyu.edu:22 + UserKnownHostsFile /dev/null + User + +Host greene + HostName localhost + Port 8027 + ForwardX11 yes + StrictHostKeyChecking no + UserKnownHostsFile /dev/null + LogLevel ERROR + User +``` +### Setting Up an SSH Config File on Windows +For Windows, you can now follow a similar configuration setup. Using Powershell you can use the following to create and open a Windows SSH config file: +```sh +# Create the config file with Powershell +New-Item -Path $HOME\.ssh\config -ItemType File + +# Open config File with Notepad +C:\WINDOWS\System32\notepad.exe $HOME\.ssh\config +``` +Configuring the file above will allow you to use the cmd terminal to connect to Greene. +### Connecting to the NYU Network +To access the HPC systems, you must either be on the NYU secure network (such as the NYU WiFi or a wired connection from an NYU office) or use a method of accessing it, such as the NYU VPN or the HPC Gateway servers. +:::info Connecting to the HPC clusters from outside NYU Network [Set up your computer to use the NYU VPN][nyu vpn link]. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net @@ -24,17 +63,35 @@ The following sections will outline basic ways to connect to the Greene cluster. ::: -You do not need to use the NYU VPN or gateways if you are connected to the NYU network (wired connection in your office or WiFi) or if you have VPN connection initiated. In this case you can ssh directly to the clusters. - ## Command Line Interface (with a Terminal) ### Mac & Linux Access -To connect to the gateway servers, simply open a [terminal application][apple terminal link] and enter the following command: +#### VPN Connection +If you're on the VPN or the main NYU network on campus you can SSH directly to greene like so: ```sh -ssh @gw.hpc.nyu.edu +ssh @greene.hpc.nyu.edu ``` + +#### Gateway / SSH Tunneling Option + +To connect to the gateway servers, after you've configured your ~/.ssh/config file, simply open a [terminal application][apple terminal link] and follow these steps: + +1. Open a terminal +2. Connect to the gateway via the ssh command: +```sh +ssh hpcgwtunnel +``` +3. Authenticate with your password and Duo MFA + +4. Open a second terminal +5. Connect to the Greene login nodes via the ssh command: +```sh +ssh greene +``` +6. Authenticate + After typing in your password you will be logged in to the cluster. Once this connection is established, you can make one more hop and connect to one of the HPC clusters: @@ -45,7 +102,7 @@ ssh @greene.hpc.nyu.edu ### Windows CMD -Windows 11 users have several options. First, the CMD program should contain an ssh client, allowing you to log into Greene or Hudson the same way as with a Linux terminal. +Windows 11 users have several options. First, the CMD program should contain an ssh client, allowing you to log into the HPC systems the same way as with a Linux terminal. ### Windows WSL2 @@ -58,7 +115,9 @@ Instructions on WSL installation can be found here: [https://docs.microsoft.com/ - If you are using WSL 2 (Windows subsystem for Linux), you may not be able to access internet when Cisco AnyConnect VPN, installed from exe file, is activated. A potential solution: uninstall Cisco AnyConnect and install AnyConnect using Microsoft Store, and then setup new VPN connection using settings described on [IT webpage][install vpn on windows link]. ::: -## Setting up SSH Keys +## Additional Tips +Below are some additional options and tips for connecting to the HPC resources. +### Setting up SSH Keys Instead of typing your password every time you need to log in, you can also specify an ssh key. @@ -75,7 +134,7 @@ Instead of typing your password every time you need to log in, you can also spec For additional recommendations on how to configure your SSH sessions, see the \[ssh configuring and x11 forwarding page]. -## PuTTY (Only for Windows) +### PuTTY (Only for Windows) There are many SSH clients for Windows OS, but we recommend using [PuTTY SSH][putty link] if you have not already. Once it is installed, launch PuTTY and configure new session "Session" category as in the screenshot below: @@ -86,8 +145,8 @@ Here we are instructing PuTTY to connect to host gw.hpc.nyu.edu on port 22 using Once you click "Open", a terminal window with prompt for password will pop up. Enter your NetID password and you should be authorized on the gateway server. Gateways are designed to support only a very minimal set of commands and their only purpose it to let users access HPC systems. Once you are there type in an ssh command that will let you connect to Greene cluster : ```sh -# Greene Login -ssh greene.hpc.nyu.edu +# Gateway Login +ssh gw.hpc.nyu.edu ``` A new command line interface window will open up that prompts you for your password on the gateway server, from there you can connect to Greene by entering the following: From c6f07ae2f6df537edd63d8c38fe9fad70712a97d Mon Sep 17 00:00:00 2001 From: Michael Weisner Date: Tue, 9 Sep 2025 12:36:47 -0400 Subject: [PATCH 2/6] Update 01_connecting_to_hpc.mdx Removed extra step from gateway option --- docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx index a4539bdedb..c2a1a14a29 100644 --- a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx +++ b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx @@ -92,13 +92,7 @@ ssh greene ``` 6. Authenticate - -After typing in your password you will be logged in to the cluster. Once this connection is established, you can make one more hop and connect to one of the HPC clusters: - -```sh -# this will connect you to Greene HPC cluster -ssh @greene.hpc.nyu.edu -``` +After typing in your password you will be logged in to the cluster. ### Windows CMD From 33a2705e7b68ccde69b9a2e032be4f192547163a Mon Sep 17 00:00:00 2001 From: Michael Weisner Date: Tue, 9 Sep 2025 12:37:52 -0400 Subject: [PATCH 3/6] Update 01_connecting_to_hpc.mdx Added header about ssh config --- docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx index c2a1a14a29..2d06618cb6 100644 --- a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx +++ b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx @@ -12,6 +12,7 @@ If you'd prefer to access the HPC cluster via a web gateway instead, please refe The following sections will outline basic ways to connect to the Greene cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines. ## Quick Start HPC Connection Guide +### Configuring Your SSH Client To connect to HPC systems, it's important to configure your machine's SSH client. For Linux and Mac machines, the configuration file is the ~/.ssh/config file on your computer. These are the basic lines that should be added to your ~/.ssh/config file: ```sh Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu From c58f2daf97035ce7edde7aa09cc20ce9fed9c424 Mon Sep 17 00:00:00 2001 From: Michael Weisner Date: Tue, 9 Sep 2025 12:47:10 -0400 Subject: [PATCH 4/6] Update 01_connecting_to_hpc.mdx added the Man in the Middle error handling here. --- .../01_connecting_to_hpc.mdx | 35 +++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx index 2d06618cb6..348c436404 100644 --- a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx +++ b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx @@ -49,6 +49,41 @@ New-Item -Path $HOME\.ssh\config -ItemType File C:\WINDOWS\System32\notepad.exe $HOME\.ssh\config ``` Configuring the file above will allow you to use the cmd terminal to connect to Greene. +### Avoiding Man in the Middle Warning. + +If you see this warning: + +:::warning + +```sh +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! +Someone could be eavesdropping on you right now (man-in-the-middle attack)! +It is also possible that a host key has just been changed. +``` + +::: + +Do not be alarmed - this is an issue that occurs because the cluster has multiple login nodes (`log-1`, `log-2`, and `log-3`) that `greene.hpc.nyu.edu` resolves to. + +To avoid this warning, you can add these lines to your SSH configuration file. Open `~/.ssh/config` and place the following lines in it: + +:::tip +This segment, referenced in the recommended config above will nullify the error: + +```sh +Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu + StrictHostKeyChecking no + ServerAliveInterval 60 + ForwardAgent yes + StrictHostKeyChecking no + UserKnownHostsFile /dev/null + LogLevel ERROR +``` +The above will also fix SSH timeout errors by extending the `ServerAliveInterval` argument. +::: ### Connecting to the NYU Network To access the HPC systems, you must either be on the NYU secure network (such as the NYU WiFi or a wired connection from an NYU office) or use a method of accessing it, such as the NYU VPN or the HPC Gateway servers. From 9a3587ee41e6af5cdd1297601fecb811bd70e63a Mon Sep 17 00:00:00 2001 From: Michael Weisner Date: Tue, 9 Sep 2025 12:47:52 -0400 Subject: [PATCH 5/6] Update 02_ssh_tunneling_and_x11_forwarding.md Removed the man in the middle warning here and put it in the quick start since it's connected to ssh config and very common. --- .../02_ssh_tunneling_and_x11_forwarding.md | 38 ------------------- 1 file changed, 38 deletions(-) diff --git a/docs/hpc/02_connecting_to_hpc/02_ssh_tunneling_and_x11_forwarding.md b/docs/hpc/02_connecting_to_hpc/02_ssh_tunneling_and_x11_forwarding.md index 8616116bf1..89bc9a0070 100644 --- a/docs/hpc/02_connecting_to_hpc/02_ssh_tunneling_and_x11_forwarding.md +++ b/docs/hpc/02_connecting_to_hpc/02_ssh_tunneling_and_x11_forwarding.md @@ -2,44 +2,6 @@ [xquartz]: https://www.xquartz.org/ -## Avoiding Man in the Middle Warning. - -If you see this warning: - -:::warning - -```sh -@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ -@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ -@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ -IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! -Someone could be eavesdropping on you right now (man-in-the-middle attack)! -It is also possible that a host key has just been changed. -``` - -::: - -Do not be alarmed - this is an issue that occurs because the cluster has multiple login nodes (`log-1`, `log-2`, and `log-3`) that `greene.hpc.nyu.edu` resolves to. - -To avoid this warning, you can add these lines to your SSH configuration file. Open `~/.ssh/config` and place the following lines in it: - - -:::tip - -```sh -Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu - StrictHostKeyChecking no - ServerAliveInterval 60 - ForwardAgent yes - StrictHostKeyChecking no - UserKnownHostsFile /dev/null - LogLevel ERROR -``` - -::: - -The above will also fix SSH timeout errors by extending the `ServerAliveInterval` argument. - ## SSH Tunneling (Mac, Linux) Setting up your workstation for SSH tunneling will make logging in and transferring files significantly easier, and installing and running an X server will allow you to use graphical software on the HPC clusters. X server is a software package that draws on your local screen windows created on a remote computer such as on the remote HPC. From 9136c8b93c3c975cbb8dc5b75ddacaf86d74f770 Mon Sep 17 00:00:00 2001 From: Michael Weisner Date: Wed, 10 Sep 2025 13:46:11 -0400 Subject: [PATCH 6/6] Update docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx good additional linking Co-authored-by: Robert Young --- docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx index 348c436404..91d03606cc 100644 --- a/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx +++ b/docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx @@ -162,7 +162,7 @@ Instead of typing your password every time you need to log in, you can also spec - Next time you will log into cluster no password will be required -For additional recommendations on how to configure your SSH sessions, see the \[ssh configuring and x11 forwarding page]. +For additional recommendations on how to configure your SSH sessions, see the [ssh configuring and x11 forwarding page](./02_ssh_tunneling_and_x11_forwarding.md). ### PuTTY (Only for Windows)