docs: Update readme.md

mesakhcienet · mesakhcienet · commit 72984a50330a · 2025-06-20T16:21:59.000+08:00
diff --git a/examples/science/af3-slurm/examples/simple_ipynb_launcher/Setup-post-cluster-deployment.md b/examples/science/af3-slurm/examples/simple_ipynb_launcher/Setup-post-cluster-deployment.md
@@ -1,9 +1,7 @@
-# Setup: Post-Cluster Deployment
-
-This guide explains how to deploy and access the Jupyter Notebook environment for AlphaFold **after the af3-slurm.yaml cluster has already been deployed** with SLURM REST support enabled.
+# Setup: Setup AF3 with Simple Ipynb Launcher - PART 2: IPython notebook setup
+This guide explains how to deploy and access the Jupyter Notebook environment for AlphaFold **after the af3-slurm.yaml cluster has been deployed** with **SLURM REST support enabled**. If you have not followed the steps outlined in [Setup-pre-cluster-deployment.md](./Setup-pre-cluster-deployment.md) before, please do so before continuing.
 
 ## Prerequisites
-
 Before proceeding, ensure the following configuration values were set correctly in your `af3-slurm-deployment.yaml` file **prior to deploying the SLURM cluster**:
 
 ```yaml
@@ -12,18 +10,10 @@ slurm_rest_token_secret_name: "<your-secret-name>"            # Name of your Sec
 af3ipynb_bucket: "<your-pre-existing-bucket-name>"           # Existing Cloud Storage bucket name
 ```
 
-> [!WARNING]
-> This guide assumes that the SLURM cluster is already deployed with REST API support enabled—i.e., the values above were correctly set before deployment.
-> If any of the values above were omitted or misconfigured, you must destroy the cluster, update the deployment configuration, and redeploy. These settings are essential for enabling SLURM REST-based notebook functionality.
-> This document focuses exclusively on deploying and accessing the Jupyter Notebook environment **after** the cluster is running and properly configured. It does not cover modifying or redeploying the cluster.
-
-If you have **not yet deployed** the cluster, or if you need help setting the appropriate variables prior to deployment, refer to [Setup-pre-cluster-deployment.md](./Setup-pre-cluster-deployment.md) guide.
-
 ## Deploying the Jupyter Notebook Blueprint
 
 ### 1. Upload the required Notebook to the Cloud Storage Bucket
-
-First, access the controller node of your deployed SLURM cluster. Replace placeholders `<controller-node-name>` and `<your-zone>` with your actual node and zone.
+Access the controller node of your deployed cluster (replace placeholders `<controller-node-name>` and `<your-zone>` with your actual node and zone):
 
 ```bash
 gcloud compute ssh <controller-node-name> --zone=<your-zone>
@@ -37,12 +27,11 @@ ansible-playbook ipynb-upload-config.yml
 ```
 
 This step uploads the notebook (`slurm-rest-api-notebook.ipynb`) along with its required scripts and libraries to the bucket defined in the `af3ipynb_bucket` variable in `af3-slurm-deployment.yaml` file.
-Make sure that this bucket was created and specified during the SLURM cluster deployment process.
 
 ### 2. Grant Secret Access to the Notebook's Service Account
+**Where to run**: **On your local machine** (where the gcloud CLI is authenticated with access to your GCP project).
 
-This setting ensures that the notebook server can successfully retrieve the specified secret by name.
-Make sure the service account running the Jupyter Notebook instance (typically the Compute Engine default service account) has permission to access the Secret Manager secret that stores your SLURM REST token.
+This setting ensures that the notebook server can successfully retrieve the specified secret by name. For this you need to make sure that the service account running the Jupyter Notebook instance (typically the Compute Engine default service account) has permission to access the Secret Manager secret that stores your SLURM REST token.
 
 The following command grants the Compute Engine default service account the `roles/secretmanager.secretAccessor` role, allowing it to access the specified secret in Secret Manager. The attached condition always evaluates to true, ensuring access is consistently granted.
 
@@ -57,7 +46,8 @@ You can verify this configuration in the Secret Manager section of the Google Cl
 
 <img src="adm/secret-manager.png" alt="secret-manager" width="1000">
 
-### 4. Launch the Notebook Environment
+### 3. Deploy the Notebook Environment
+**Where to run**: **On your local machine**, inside your cluster-toolkit directory.
 
 Deploy the Jupyter Notebook environment using the following command:
 
@@ -68,10 +58,7 @@ cd cluster-toolkit
 examples/science/af3-slurm/examples/simple_ipynb_launcher/af3-slurm-ipynb.yaml --auto-approve
 ```
 
-This brings up the Jupyter Notebook deployment.
-
-### 5. Access the Notebook via Vertex AI Workbench
-
+### 4. Access the Notebook via Vertex AI Workbench
 In the Google Cloud Console:
 
 1. Navigate to `Vertex AI` → `Workbench` → `Instances`
@@ -80,7 +67,8 @@ In the Google Cloud Console:
 
 3. Locate and open the `slurm-rest-api-notebook.ipynb` file. If you haven't modified the default value of `af3ipynb_bucket_local_mount` in the `af3-slurm-deployment.yaml`, the notebook should be available at `/home/jupyter/alphafold` folder.
 
-### 6. Verify REST Token Access
+### 5. Verify REST Token Access
+**Where to run**: **Inside the JupyterLab terminal** of your deployed Vertex AI Workbench instance.
 
 To verify that Secret Manager access is properly configured, open a terminal within JupyterLab and run the following command:
 
@@ -93,7 +81,6 @@ If the command returns the secret value successfully, it confirms that the noteb
 <img src="adm/rest_api.png" alt="slrum rest api" width="1000">
 
 ## Teardown
-
 To remove the Jupyter Notebook deployment when it is no longer needed, run the following command:
 
 ```bash
@@ -105,7 +92,10 @@ To remove the Jupyter Notebook deployment when it is no longer needed, run the f
 > Additionally, any Cloud Storage buckets you created (via the CLI or console) will not be automatically deleted. You are responsible for cleaning them up manually to avoid unnecessary charges.
 > For deleting the buckets consult [Delete buckets](https://cloud.google.com/storage/docs/deleting-buckets).
 
-## Notes and Customization
+## Customization
 You can adjust the notebook setup behavior using blueprint variables in the deployment YAML.
 All configurations should be validated before running jobs.
 If further modifications to SLURM REST/API Server behavior are required, you must destroy and redeploy the cluster with the updated settings.
+
+## Using the environment
+Go to [Ipynb.md](./Ipynb.md) for documentation on how to use the IPython environment.
diff --git a/examples/science/af3-slurm/examples/simple_ipynb_launcher/Setup-pre-cluster-deployment.md b/examples/science/af3-slurm/examples/simple_ipynb_launcher/Setup-pre-cluster-deployment.md
@@ -1,20 +1,19 @@
-# Setup: Pre-Cluster Deployment
-This guide explains how to properly set up and launch the Simple IPython Launcher environment **before deploying the `af3-slurm.yaml` cluster**.
+# Setup AF3 with Simple Ipynb Launcher - PART 1: specific cluster settings
+This guide explains the additional steps needed to deploy the AF3 solution with the Slurm REST-API required by the [Simple Ipynb launcher](./README.md).
 
-> **Important:** This setup documentation is intended to be followed **before** deploying the SLURM cluster (as referenced [here](../../README.md#deploy-slurm-cluster)).  
-> If you have **already deployed** the cluster, please refer to [`post-cluster-deployment.md`](./Setup-post-cluster-deployment.md) instead.
+> [!NOTE]
+> **Important:** The steps described here are intended to be followed **before** deploying the SLURM cluster (as referenced [here](../../README.md#deploy-slurm-cluster)).  
+> If you have **already deployed** the cluster, you should tear down the cluster and redeploy after the steps outlined here. Once your cluster is deployed, you will have to follow the steps outlined in [Setup AF3 with Simple Ipynb Launcher - PART 2: IPython notebook setup](./Setup-post-cluster-deployment.md)
 
 ## Prerequisites
 
-### Set up Jobs Bucket
-
-If you want to use the simple service launcher, you need to create an additional bucket, that should
-be located in the region where you stand up your cluster:
+### Set up Bucket for IPython Notebook
+If you want to use the Simple ipynb launcher, you need to create an additional bucket for the IPython Notebook that is provided by Google Vertex AI Workbench. It should be located in the region where you stand up your cluster:
 
 ```bash
 #!/bin/bash
 
-UNIQUE_JOB_BUCKET=<your-bucket>
+UNIQUE_AF3IPYNB_BUCKET=<your-bucket>
 PROJECT_ID=<your-gcp-project>
 REGION=<your-preferred-region>
 
@@ -24,118 +23,29 @@ gcloud storage buckets create gs://${UNIQUE_JOB_BUCKET} \
     --uniform-bucket-level-access
 ```
 
-## Deploying a Jupyter Notebook for AlphaFold
-
-You can deploy a Jupyter Notebook environment to run AlphaFold step-by-step. This is useful for interactive exploration and making custom modifications to the AlphaFold pipeline.
-
-> **Important:** To enable the Jupyter Notebook deployment, you **must provide a cloud storage bucket**. If no bucket is specified, the notebook environment will not be created.
+### Configure Secret Manager
+The IPython notebook will need a secure connection with the Slurm REST API servers. We are using the Google Cloud Secret Manager to manage the necessary credentials in a secure way. Please follow the guideline [here](https://cloud.google.com/secret-manager/docs/create-secret-quickstart) to create a Secret Manager.
 
-### Enable the SLURM REST API Server
+You do not need to manually add any data to the secret; the SLURM token will be automatically populated by the system.
 
-Set the `slurm_rest_server_activate` value to `true` in the `af3-slurm-deployment.yaml` file to enable the SLURM REST API server on the controller node of the `af3-slurm.yaml` cluster. This allows the Jupyter Notebook environment to send requests—such as job submissions—to the SLURM scheduler via the REST API.
+## Modify the af3-slurm-deployment.yaml
+Set the `slurm_rest_server_activate` value to `true` in the `af3-slurm-deployment.yaml` file to enable the SLURM REST API server on the controller node of the `af3-slurm.yaml` cluster. This is required for the Jupyter Notebook environment to send requests —such as job submissions— to the SLURM scheduler via the REST API.
 
 ```yaml
 slurm_rest_server_activate: true
 ```
 
-### Configuring the SLURM REST API Token Secret Manager
-
-Set `slurm_rest_token_secret_name` value in the `af3-slurm-deployment.yaml` with the name of an existing secret in Secret Manager. Please follow the guideline [here](https://cloud.google.com/secret-manager/docs/create-secret-quickstart) to create a Secret Manager.
-
-> This setting allows you to specify the name of a Google Cloud Secret Manager secret that holds your SLURM authentication token. Using Secret Manager is a secure way to manage sensitive credentials.
+Set the `slurm_rest_token_secret_name` value in the `af3-slurm-deployment.yaml` with the name of the secret you created as part of the [Prerequisites](#prerequisites).
 
 ```yaml
 slurm_rest_token_secret_name: "<your-secret-name>"
 ```
 
-### Steps to Deploy
-
-1. Ensure you have provided a valid cloud storage bucket in your `af3-slurm-deployment.yaml`:
-
-    ```yaml
-    af3ipynb_bucket: "<your-pre-existing-bucket-name>"
-    ```
-
-2. Build the Jupyter notebook with `af3-slurm-ipynb.yaml`:
-
-    ```bash
-    # Make sure you are under `cluster-toolkit` root folder
-    ~$ cd cluster-toolkit  
-    cluster-toolkit$ ./gcluster deploy -d examples/science/af3-slurm/af3-slurm-deployment.yaml examples/science/af3-slurm/examples/simple_ipynb_launcher/af3-slurm-ipynb.yaml --auto-approve
-    ```
-
-## Upload Notebook to Bucket
-
-To upload the Jupyter notebook to the cloud storage bucket so it can be accessed via JupyterLab:
-
-1. **SSH into controller node** in the cluster.
-
-2. **Navigate to the setup directory**:
-
-   ```bash
-   cd /home/af3ipynb/ipynb_setup
-   ```
-
-3. Run the Ansible playbook to upload the Jupyter notebook and its required library files to the designated bucket:
-
-    ```bash
-    ansible-playbook ipynb-upload-config.yml
-    ```
-
-This playbook will upload `slurm-rest-api-notebook.ipynb` along with its associated scripts and library files to the target bucket (`af3ipynb_bucket`).
-Once the upload is complete, you can access the notebook from the JupyterLab interface via:
-
-> Cloud Console → Vertex AI → Workbench → Instances
-
-## Granting Access to the Token
+Set the `af3ipynb_bucket` value in the `af3-slurm-deployment.yaml` with the name of the secret you created as part of the [Prerequisites](#prerequisites).
 
-  > Before running this section, please complete the [**Upload Notebook to Bucket**](#upload-notebook-to-bucket) section first.
-
-  After the deploying steps are success, you must ensure that the appropriate service account (e.g., the default service account for the Compute Engine instance) has permission to access the secret. This step is crucial because, without access to the secret, the notebook will not be able to authenticate properly or send valid requests to the SLURM REST API.
-
-  If you have not granted access yet, then run the following command, replacing `<your-secret-name>` and `<your-project-id>` with your values:
-
-  ```bash
-  gcloud secrets add-iam-policy-binding <your-secret-name> \
-    --member="serviceAccount:$(gcloud projects describe <your-project-id> --format='value(projectNumber)')-compute@developer.gserviceaccount.com" \
-    --role="roles/secretmanager.secretAccessor" \
-    --condition="expression=true,title=AlwaysTrue,description=Allow access to Secret Manager"
-  ```
-
-  This command grants the Compute Engine default service account the `secretAccessor` role with a condition that always evaluates to true, as shown in the image below. You can view this in the `Secret Manager` page in the Google Cloud Console.
-
-  <img src="adm/secret-manager.png" alt="secret-manager" width="1000">
-
-  This setting allows the notebook server to successfully retrieve the value of the specified secret by its name.
-
-### Verify Access
-
-  To confirm that the service account has the necessary permissions, from the jupyter notebook workbench, you can open a terminal and run the following command:
-
-  ```bash
-  gcloud secrets versions access latest --secret=<your-secret-name>
-  ```
-
-  If the command succeeds, this confirms that your notebook can securely retrieve the authentication token from Secret Manager as shown in the image below.
-
-   <img src="adm/rest_api.png" alt="slrum rest api" width="1000">
-
-## Custom configuration
-
-You can customize settings via blueprint variables before deployment. If modifications are needed later, ensure:
-
-- All required resources are available
-- Configuration changes within the notebook are validated before submitting new jobs.
-
-## Teardown Jupyter Notebook
-
-If you would like to tear down the notebook deployment, use the command below.
-
-```bash
-./gcluster destroy af3-slurm-ipynb --auto-approve
+```yaml
+af3ipynb_bucket: "<your-pre-existing-bucket-name>"
 ```
 
-> [!WARNING]
-> If you do not destroy the Jupyter Notebook deployment, it may continue to incur costs.
-> Additionally, any Cloud Storage buckets you created (via the CLI or console) will not be automatically deleted. You are responsible for cleaning them up manually to avoid unnecessary charges.
-> For deleting the buckets consult [Delete buckets](https://cloud.google.com/storage/docs/deleting-buckets).
+## Continue to cluster deployment
+Go back to [Deploy Slurm Cluster](../../README.md#deploy-slurm-cluster) and deploy the cluster.
