Merge pull request #223 from NYU-RTS/long-live-torch

torch is now open to all

Author: s-sajid-ali

docs/cloud/03_hpc_bursting_to_cloud/01_intro.md

@@ -16,7 +16,7 @@ To request access to the HPC Bursting capabilities, [please complete this form][
 This is not public, only per request of eligible classes or researchers
 :::
 ```sh
-ssh <NetID>@greene.hpc.nyu.edu
+ssh <NetID>@login.torch.hpc.nyu.edu
 ```
 ssh to the class on GCP (burst login node) - anyone can login but you can only submit jobs if you have approval
 ```sh
@@ -50,9 +50,9 @@ When you run a bursting job the compute nodes will not see those file mounts. Th
 
 The file systems are independent, so you must copy data to the GCP location.
 
-To copy data, you must first start an interactive job. Once started, you can copy your data using scp from the HPC Data Transfer Nodes (greene-dtn). Below is the basic setup to copy files from Torch to your home directory while you are in an interactive bursting job:
+To copy data, you must first start an interactive job. Once started, you can copy your data using scp from the HPC Data Transfer Nodes (torch-dtn). Below is the basic setup to copy files from Torch to your home directory while you are in an interactive bursting job:
 ```sh
-scp <NetID>@greene-dtn.hpc.nyu.edu:/path/to/files /home/<NetID>/
+scp <NetID>@dtn.torch.hpc.nyu.edu:/path/to/files /home/<NetID>/
 ```
 
 

docs/cloud/03_hpc_bursting_to_cloud/02_visualization.md

@@ -10,7 +10,7 @@ Add the following to your SSH config file (~/.ssh/config) on your local workstat
 Host burst
   HostName burst.hpc.nyu.edu
   User <NetID>
-  ProxyJump <NetID>@greene.hpc.nyu.edu
+  ProxyJump <NetID>@login.torch.hpc.nyu.edu
   ProxyJump <NetID>@burst.hpc.nyu.edu
   StrictHostKeyChecking no
   UserKnownHostsFile /dev/null

docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.md

@@ -4,19 +4,19 @@
 This page gives an overview of connecting to the HPC cluster, for a tutorial on this topic, head to the [this section](../12_tutorial_intro_shell_hpc/02_connecting_to_hpc.mdx)! 
 :::
 
-There are several ways to interact with the Greene HPC cluster. Similar to other Linux clusters, the most common method of connection is via a Command Line Interface (CLI). A CLI is a program that allows you to create and delete files, run programs, and navigate through directories and files via a typed prompt. On Mac, the built-in CLI application is called Terminal. While Windows 11 machines support a Linux Subsystem, which allows for similar functionality, a popular tool used to connect to a Linux server is a free application called [MoabXterm](https://mobaxterm.mobatek.net/documentation.html) 
+There are several ways to interact with the Torch HPC cluster. Similar to other Linux clusters, the most common method of connection is via a Command Line Interface (CLI). A CLI is a program that allows you to create and delete files, run programs, and navigate through directories and files via a typed prompt. On Mac, the built-in CLI application is called Terminal. While Windows 11 machines support a Linux Subsystem, which allows for similar functionality, a popular tool used to connect to a Linux server is a free application called [MoabXterm](https://mobaxterm.mobatek.net/documentation.html) 
 
 :::tip
 If you'd prefer to access the HPC cluster via a web gateway instead, please refer to [the section describing Open OnDemand (OOD)](../09_ood/01_ood_intro.md).
 :::
 
-The following sections will outline basic ways to connect to the Greene cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines.
+The following sections will outline basic ways to connect to the Torch cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines.
 
 ## Configuring Your SSH Client
 
 To connect to HPC systems, you have to configure the SSH client on your machine. We recommend the following configuration:
 ```sh
-Host greene.hpc.nyu.edu dtn.hpc.nyu.edu
+Host dtn.torch.hpc.nyu.edu
   User <NetID>
   StrictHostKeyChecking no
   ServerAliveInterval 60
@@ -46,7 +46,7 @@ New-Item -Path $HOME\.ssh\config -ItemType File
 # Open config File with Notepad
 C:\WINDOWS\System32\notepad.exe $HOME\.ssh\config
 ```
-Configuring the file above will allow you to use the cmd terminal to connect to Greene.
+Configuring the file above will allow you to use the cmd terminal to connect to Torch.
 
 ### Common issues
 
@@ -60,7 +60,7 @@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
 Someone could be eavesdropping on you right now (man-in-the-middle attack)!
 It is also possible that a host key has just been changed.
 ```
-Do not be alarmed - this is an issue that occurs because the Greene and Torch clusters have multiple login nodes. For instance, `greene.hpc.nyu.edu` resolves to one of three possible login nodes (`log-1`, `log-2`, and `log-3`). The configuration directive `StrictHostKeyChecking no` referenced in the recommended config above will prevent this error. Ensure that you have not missed adding it.
+Do not be alarmed - this is an issue that occurs because the Torch and Torch clusters have multiple login nodes. For instance, `login.torch.hpc.nyu.edu` resolves to one of three possible login nodes (`log-1`, `log-2`, and `log-3`). The configuration directive `StrictHostKeyChecking no` referenced in the recommended config above will prevent this error. Ensure that you have not missed adding it.
 :::
 
 :::info SSH Timeout errors
@@ -78,10 +78,10 @@ To access the HPC systems, you must either be on the NYU secure network (such as
 
 ### Mac & Linux Access
 
-Once you're on the VPN or the main NYU network on campus you can SSH directly to greene like so:
+Once you're on the VPN or the main NYU network on campus you can SSH directly to torch like so:
 
 ```sh
-ssh <Your NetID>@greene.hpc.nyu.edu
+ssh <Your NetID>@login.torch.hpc.nyu.edu
 ```
 
 ### Windows
@@ -119,7 +119,7 @@ There is an extra layer of security when accessing Torch which is described belo
 -   Return to terminal and hit Enter
 ![torch-ssh-2fa-9](static/torch-ssh-2fa-9.png)
 
-## SSH Keys for Greene
+## SSH Keys for Torch
 :::warning
 SSH keys are not supported on Torch!
 :::

docs/hpc/02_connecting_to_hpc/02_x11_forwarding.md

@@ -5,7 +5,7 @@
 In rare cases when you need to interact with GUI applications on HPC clusters, you need to enable X11 forwarding for your SSH connection. Mac and Linux users will need to run the ssh commands described above with an additional flag:
 
 ```sh
-ssh -Y <NYU_NetID>@greene.hpc.nyu.edu
+ssh -Y <NYU_NetID>@login.torch.hpc.nyu.edu
 ```
 
 However, Mac users need to install [XQuartz][xquartz], since X-server is no longer shipped with the macOS.

docs/hpc/03_storage/01_intro_and_data_management.mdx

@@ -39,7 +39,7 @@ Every individual user has a home directory (under **`/home/$USER`**, environment
 
 User home directories are backed up daily and old files under **`$HOME`** are not purged.
 
-The User home directories are available on all HPC clusters (Greene) and on every cluster node (login nodes, compute nodes) as well as and Data Transfer Node (gDTN). 
+The User home directories are available on all HPC clusters (Torch) and on every cluster node (login nodes, compute nodes) as well as and Data Transfer Node (gDTN). 
 
 :::warning
 Avoid changing file and directory permissions in your home directory to allow other users to access files.
@@ -62,7 +62,7 @@ Files stored in the HPC scratch file system are subject to the <ins>**HPC Scratc
 
 Every user has a dedicated scratch directory (**/scratch/$USER**) with **5 TB** disk quota and **1,000,000 inodes** (files) limit per user. 
 
-The scratch file system is available on all nodes (compute, login, etc.) on Greene as well as Data Transfer Node (gDTN).
+The scratch file system is available on all nodes (compute, login, etc.) on Torch as well as Data Transfer Node (gDTN).
 
 :::warning
 There are **No Back ups of the scratch file system.** ***Files that were deleted accidentally or removed due to storage system failures CAN NOT be recovered.***
@@ -83,7 +83,7 @@ Files stored in the HPC vast file system are subject to the <ins>***HPC Vast old
 
 Every user has a dedicated vast directory (**`/vast/$USER`**) with **2 TB** disk quota and **5,000,000 inodes** (files) limit per user. 
 
-The vast file system is available on all nodes (compute, login, etc.) on Greene as well as Data Transfer Node (gDTN).
+The vast file system is available on all nodes (compute, login, etc.) on Torch as well as Data Transfer Node (gDTN).
 
 :::warning
 There are **No Back ups** of the vastsc file system. ***Files that were deleted accidentally or removed due to storage system failures CAN NOT be recovered.***
@@ -108,7 +108,7 @@ For some of the datasets users must provide a signed usage agreement before acce
 Public datasets available on the HPC clusters can be viewed on the [Datasets page](../04_datasets/01_intro.md).
 
 #### HPC Archive
-Once the Analysis stage of the research data lifecycle has completed, <ins>_HPC users should **tar** their data and code into a single tar.gz file and then copy the  file to their archive directory (**`/archive/$USER`**_).</ins> The HPC Archive file system is not accessible by running jobs; it is suitable for long-term data storage. Each user has access to a default disk quota of **2TB** and ***20,000 inode (files) limit***. The rather low limit on the number of inodes per user is intentional.  The archive file system is available only ***on login nodes*** of Greene. The archive file system is backed up daily.
+Once the Analysis stage of the research data lifecycle has completed, <ins>_HPC users should **tar** their data and code into a single tar.gz file and then copy the  file to their archive directory (**`/archive/$USER`**_).</ins> The HPC Archive file system is not accessible by running jobs; it is suitable for long-term data storage. Each user has access to a default disk quota of **2TB** and ***20,000 inode (files) limit***. The rather low limit on the number of inodes per user is intentional.  The archive file system is available only ***on login nodes*** of Torch. The archive file system is backed up daily.
 
 -   Here is an example ***tar*** command that combines the data in a directory named ***my_run_dir*** under ***`$SCRATCH`*** and outputs the tar file in the user's ***`$ARCHIVE`***: 
 ```sh

docs/hpc/03_storage/02_available_storage_systems.md

@@ -20,7 +20,7 @@ The NYU HPC cluster storage runs on Lenovo Distributed Storage Solution DSS-G ha
 -   I/O Performance: up to 650k input/output operations per second (IOPS)
 
 ## Flash Tier Storage (VAST)
-An all flash file system, using [VAST Flash storage](https://www.vastdata.com/), is now available on Greene. Flash storage is optimal for computational workloads with high I/O rates. For example, If you have jobs to run with huge amount of tiny files, VAST may be a good candidate.  If you and your lab members are interested, please reach out to [hpc@nyu.edu](mailto:hpc@nyu.edu) for more information.
+An all flash file system, using [VAST Flash storage](https://www.vastdata.com/), is now available on Torch. Flash storage is optimal for computational workloads with high I/O rates. For example, If you have jobs to run with huge amount of tiny files, VAST may be a good candidate.  If you and your lab members are interested, please reach out to [hpc@nyu.edu](mailto:hpc@nyu.edu) for more information.
 -   NVMe interface
 -   Total size: 778 TB
 :::note
@@ -29,7 +29,7 @@ An all flash file system, using [VAST Flash storage](https://www.vastdata.com/),
 
 ## Research Project Space (RPS)
 [Research Project Space (RPS)](./05_research_project_space.mdx) volumes provide working spaces for sharing data and code amongst project or lab members.
--   RPS directories are available on the Greene HPC cluster.
+-   RPS directories are available on the Torch HPC cluster.
 -   There is no old-file purging policy on RPS.
 -   RPS is backed up. 
 -   There is a cost per TB per year and inodes per year for RPS volumes.

docs/hpc/03_storage/03_data_transfers.md

@@ -5,14 +5,14 @@ Globus is the recommended tool to use for large-volume data transfers due to the
 :::
 
 ## Data-Transfer nodes
-Attached to the NYU HPC cluster Greene, the Greene Data Transfer Node (gDTN) are  nodes optimized for transferring data between cluster file systems (e.g. scratch)  and other endpoints outside the NYU HPC clusters, including user laptops and desktops. The gDTNs have 100-Gb/s Ethernet connections to the High Speed Research Network (HSRN) and are connected to the HDR Infiniband fabric of the HPC clusters. More information on the hardware characteristics is available at [Greene spec sheet](../10_spec_sheet.md).
+Attached to the NYU HPC cluster Torch, the Torch Data Transfer Node (gDTN) are  nodes optimized for transferring data between cluster file systems (e.g. scratch)  and other endpoints outside the NYU HPC clusters, including user laptops and desktops. The gDTNs have 100-Gb/s Ethernet connections to the High Speed Research Network (HSRN) and are connected to the HDR Infiniband fabric of the HPC clusters. More information on the hardware characteristics is available at [Torch spec sheet](../10_spec_sheet.md).
 
 ### Data Transfer Node Access
 The HPC cluster filesystems include `/home`, `/scratch`, `/archive` and the [HPC Research Project Space](./05_research_project_space.mdx) are available on the gDTN. The Data-Transfer Node (DTN) can be accessed in a variety of ways
--   From NYU-net and the High Speed Research Network: use SSH to the DTN hostname `gdtn.hpc.nyu.edu`
--   From the Greene cluster (e.g., the login nodes): the hostname can be shortened to `gdtn`
+-   From NYU-net and the High Speed Research Network: use SSH to the DTN hostname `gdtn.torch.hpc.nyu.edu`
+-   From the Torch cluster (e.g., the login nodes): the hostname can be shortened to `gdtn`
 :::info
-For example, to log in to a DTN from the Greene cluster, to carry out some copy operation, and to log back out, you can use a command sequence like:
+For example, to log in to a DTN from the Torch cluster, to carry out some copy operation, and to log back out, you can use a command sequence like:
 ```sh
 ssh gdtn
 rsync ...
@@ -30,10 +30,10 @@ logout
 Please use Data Transfer Nodes (DTNs) with these tools. While one can transfer data while on login nodes, it is  considered a bad practice because it can degrade the node's performance.
 :::
 
-Sometimes these two tools are convenient for transferring small files. Using the DTNs does not require to set up an SSH tunnel; use the hostname `dtn.hpc.nyu.edu` for one-step copying. See below for examples of commands invoked on the command line on a laptop running a Unix-like operating system:
+Sometimes these two tools are convenient for transferring small files. Using the DTNs does not require to set up an SSH tunnel; use the hostname `dtn.torch.hpc.nyu.edu` for one-step copying. See below for examples of commands invoked on the command line on a laptop running a Unix-like operating system:
 ```sh
-scp HMLHWBGX7_n01_HK16.fastq.gz jdoe55@dtn.hpc.nyu.edu:/scratch/jdoe55/
-rsync -av HMLHWBGX7_n01_HK16.fastq.gz jdoe55@dtn.hpc.nyu.edu:/scratch/jdoe55/ 
+scp HMLHWBGX7_n01_HK16.fastq.gz jdoe55@dtn.torch.hpc.nyu.edu:/scratch/jdoe55/
+rsync -av HMLHWBGX7_n01_HK16.fastq.gz jdoe55@dtn.torch.hpc.nyu.edu:/scratch/jdoe55/ 
 ```
 In particular, rsync can also be used on the DTNs to copy directories recursively between filesystems, e.g. (assuming that you are logged in to a DTN),
 ```sh
@@ -48,7 +48,7 @@ Windows 10 machines may have the Linux Subsystem installed, which will allow for
 ### Globus
 Globus is the recommended tool to use for large-volume data transfers. It features automatic performance tuning and automatic retries in cases of file-transfer failures. Data-transfer tasks can be submitted via a web portal. The Globus service will take care of the rest, to make sure files are copied efficiently, reliably, and securely. Globus is also a tool for you to share data with collaborators, for whom you only need to provide the email addresses.
 
-The Globus endpoint for Greene is available at `nyu#greene`. The endpoint `nyu#prince` has been retired. Detailed instructions available at [Globus](./04_globus.md)
+The Globus endpoint for Torch is available at `nyu#torch`. Detailed instructions available at [Globus](./04_globus.md)
 
 ### rclone
 rclone - rsync for cloud storage, is a command line program to sync files and directories to and from cloud storage systems such as Google Drive, Amazon Drive, S3, B2 etc. rclone is available on DTNs. [Please see the documentation for how to use it.](https://rclone.org/)