You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/science/af3-slurm/examples/simple_ipynb_launcher/README.md
+15Lines changed: 15 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,3 +10,18 @@ Please note that the launcher needs 2 specific setup steps:
10
10
11
11
## Usage Guide
12
12
For usage of the Ipynb Launcher consult the [Step-by-Step Instructions](./Ipynb.md)
13
+
14
+
## Known Limitations
15
+
You may encounter the following problems while using the notebook.
16
+
17
+
### Warning during dependency installation
18
+
You may encounter the following warning during dependency installation:
19
+
20
+
```text
21
+
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
22
+
google-cloud-bigtable 1.7.3 requires grpc-google-iam-v1<0.13dev,>=0.12.3, but you have grpc-google-iam-v1 0.14.2 which is incompatible
23
+
```
24
+
25
+
**Resolution**: This warning can be **safely ignored**.
26
+
27
+
The version mismatch does not impact the functionality required by this project. The `google-cloud-bigtable` package is not used in any critical code path, and no issues have been observed during execution.
This step uploads the notebook (`slurm-rest-api-notebook.ipynb`) along with its required scripts and libraries to the bucket defined in the `af3ipynb_bucket` variable in `af3-slurm-deployment.yaml` file.
30
30
31
31
### 2. Grant Secret Access to the Notebook's Service Account
32
-
**Where to run**: **On your local machine** (where the gcloud CLI is authenticated with access to your GCP project).
32
+
**Where to run**: **On the system where you executed `gcluster`** (where the `gcloud` CLI is authenticated with access to your GCP project).
33
33
34
34
This setting ensures that the notebook server can successfully retrieve the specified secret by name. For this you need to make sure that the service account running the Jupyter Notebook instance (typically the Compute Engine default service account) has permission to access the Secret Manager secret that stores your SLURM REST token.
35
35
@@ -47,7 +47,7 @@ You can verify this configuration in the Secret Manager section of the Google Cl
**Where to run**: **On your local machine**, inside your cluster-toolkit directory.
50
+
**Where to run**: **On the system where you executed `gcluster`**, under your `cluster-toolkit` directory.
51
51
52
52
Deploy the Jupyter Notebook environment using the following command:
53
53
@@ -65,10 +65,15 @@ In the Google Cloud Console:
65
65
66
66
2. Open the JupyterLab interface for the newly deployed instance
67
67
68
-
3. Locate and open the `slurm-rest-api-notebook.ipynb` file. If you haven't modified the default value of `af3ipynb_bucket_local_mount` in the `af3-slurm-deployment.yaml`, the notebook should be available at `/home/jupyter/alphafold` folder.
68
+
3. Locate and open the `slurm-rest-api-notebook.ipynb` file. If you haven't modified the default value of `af3ipynb_bucket_local_mount` in the `af3-slurm-deployment.yaml`, the notebook files will be mounted to `/home/jupyter/alphafold` on the Jupyter notebook system.
69
+
70
+
When you first connect to Jupyter Notebook, it only shows the alphafold folder — this can appear as if it's located at `/alphafold`. In reality, Jupyter starts in the `/home/jupyter` directory, and the `alphafold` folder is located inside it.
71
+
72
+
So while it looks like `/alphafold` in the interface, the actual path is `/home/jupyter/alphafold`.
69
73
70
74
### 5. Verify REST Token Access
71
-
**Where to run**: **Inside the JupyterLab terminal** of your deployed Vertex AI Workbench instance.
75
+
**Where to run**: **On the system where you executed `gcluster`** (where the `gcloud` CLI is authenticated with access to your GCP project, similar to step [Grant Secret Access to the Notebook's Service Account](#2-grant-secret-access-to-the-notebooks-service-account)
76
+
).
72
77
73
78
To verify that Secret Manager access is properly configured, open a terminal within JupyterLab and run the following command:
74
79
@@ -80,22 +85,22 @@ If the command returns the secret value successfully, it confirms that the noteb
To remove the Jupyter Notebook deployment when it is no longer needed, run the following command:
85
-
86
-
```bash
87
-
./gcluster destroy af3-slurm-ipynb --auto-approve
88
-
```
89
-
90
-
> [!WARNING]
91
-
> If you do not destroy the Jupyter Notebook deployment, it may continue to incur costs.
92
-
> Additionally, any Cloud Storage buckets you created (via the CLI or console) will not be automatically deleted. You are responsible for cleaning them up manually to avoid unnecessary charges.
93
-
> For deleting the buckets consult [Delete buckets](https://cloud.google.com/storage/docs/deleting-buckets).
88
+
## Using the environment
89
+
Go to [Ipynb.md](./Ipynb.md) for documentation on how to use the IPython environment.
94
90
95
91
## Customization
96
92
You can adjust the notebook setup behavior using blueprint variables in the deployment YAML.
97
93
All configurations should be validated before running jobs.
98
94
If further modifications to SLURM REST/API Server behavior are required, you must destroy and redeploy the cluster with the updated settings.
99
95
100
-
## Using the environment
101
-
Go to [Ipynb.md](./Ipynb.md) for documentation on how to use the IPython environment.
96
+
## Teardown
97
+
To remove the Jupyter Notebook deployment when it is no longer needed, run the following command:
98
+
99
+
```bash
100
+
./gcluster destroy af3-slurm-ipynb --auto-approve
101
+
```
102
+
103
+
> [!WARNING]
104
+
> If you do not destroy the Jupyter Notebook deployment, it may continue to incur costs.
105
+
> Additionally, any Cloud Storage buckets you created (via the CLI or console) will not be automatically deleted. You are responsible for cleaning them up manually to avoid unnecessary charges.
106
+
> For deleting the buckets consult [Delete buckets](https://cloud.google.com/storage/docs/deleting-buckets).
The IPython notebook will need a secure connection with the Slurm REST API servers. We are using the Google Cloud Secret Manager to manage the necessary credentials in a secure way. Please follow the guideline [here](https://cloud.google.com/secret-manager/docs/create-secret-quickstart) to create a Secret Manager.
27
+
The IPython notebook will need a secure connection with the Slurm REST API servers. We are using the Google Cloud Secret Manager to manage the necessary credentials in a secure way. Please follow the guideline [here](https://cloud.google.com/secret-manager/docs/create-secret-quickstart) to create a secret using Secret Manager.
28
28
29
29
You do not need to manually add any data to the secret; the SLURM token will be automatically populated by the system.
0 commit comments