You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cloud/03_hpc_bursting_to_cloud/01_intro.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,17 +38,17 @@ However, approval is required to submit jobs to the partitions. Partitions are s
38
38
39
39
### Storage
40
40
41
-
Greene's `/home` and `/scratch` are mounted (available) at login node of bursting setup.
41
+
Torch's `/home` and `/scratch` are mounted (available) at login node of bursting setup.
42
42
43
-
Compute node however, do have independent `/home` and `/scratch`. These `/home` and `/scratch` mounts are persistent, are available from any compute node and independent from `/home` and `/scratch` at Greene.
43
+
Compute node however, do have independent `/home` and `/scratch`. These `/home` and `/scratch` mounts are persistent, are available from any compute node and independent from `/home` and `/scratch` at Torch.
44
44
45
-
User may need to copy data from Greene's `/home` or `/scratch` to GCP mounted `/home` or `/scratch`
45
+
User may need to copy data from Torch's `/home` or `/scratch` to GCP mounted `/home` or `/scratch`
46
46
47
47
When you run a bursting job the compute nodes will not see those file mounts. This means that you need to copy data to the burst instance.
48
48
49
49
The file systems are independent, so you must copy data to the GCP location.
50
50
51
-
To copy data, you must first start an interactive job. Once started, you can copy your data using scp from the HPC Data Transfer Nodes (greene-dtn). Below is the basic setup to copy files from Greene to your home directory while you are in an interactive bursting job:
51
+
To copy data, you must first start an interactive job. Once started, you can copy your data using scp from the HPC Data Transfer Nodes (greene-dtn). Below is the basic setup to copy files from Torch to your home directory while you are in an interactive bursting job:
Copy file name to clipboardExpand all lines: docs/cloud/04_dataproc/02_data_management.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -54,8 +54,8 @@ You can find the full path to your file/folder in the ingest storage by clicking
54
54
Data uploaded into the ingest website will be visible to all members of the cluster temporarily. If you are uploading files that cannot be shared with all cluster users (e.g., code) please use the alternate method described below.
55
55
:::
56
56
57
-
## Uploading Data to HDFS from Greene
58
-
First download gcloud on a desktop computer with a browser by following the instructions [here](https://cloud.google.com/sdk/docs/install). Ensure that you run the install command described as optional in step 4. Log into Greene and activate the Google Cloud command line interface module:
57
+
## Uploading Data to HDFS from Torch
58
+
First download gcloud on a desktop computer with a browser by following the instructions [here](https://cloud.google.com/sdk/docs/install). Ensure that you run the install command described as optional in step 4. Log into Torch and activate the Google Cloud command line interface module:
59
59
```sh
60
60
ml load google-cloud-sdk/379.0.0
61
61
```
@@ -65,7 +65,7 @@ gcloud auth login
65
65
```
66
66
Copy and paste the command that you are given into a terminal application on your desktop and run it. When prompted, type y to proceed. If you are signed into multiple Google accounts, you will then be presented with a browser window where you can choose your account. Select your NYU account. Google will then present a message saying that “Google Cloud SDK wants to access your Google Account”. Click Allow.
67
67
68
-
Copy the URL that you are given in the terminal window, and paste it into your Greene session on the line where gcloud asks for it.
68
+
Copy the URL that you are given in the terminal window, and paste it into your Torch session on the line where gcloud asks for it.
69
69
70
70
Type gcloud auth list to verify that you are logged in:
71
71
```sh
@@ -81,12 +81,12 @@ Now that you are logged in, use the instructions under the Small Transfers or La
81
81
82
82
### Small Transfers
83
83
84
-
Run the following commands on Greene to ensure that gcloud knows that you are using it with Dataproc (rather than a different Google Cloud application):
84
+
Run the following commands on Torch to ensure that gcloud knows that you are using it with Dataproc (rather than a different Google Cloud application):
85
85
```sh
86
86
gcloud config set project hpc-dataproc-19b8
87
87
gcloud config set compute/zone us-central1-f
88
88
```
89
-
Run the following command on Greene to upload your file to your filesystem home directory on Dataproc:
89
+
Run the following command on Torch to upload your file to your filesystem home directory on Dataproc:
0 commit comments