From ca1926c7e4c5555337e55b30180a2614eb3cf15b Mon Sep 17 00:00:00 2001 From: Laura Fitzgerald Date: Tue, 21 Apr 2026 09:55:37 +0100 Subject: [PATCH 1/2] Standardize notebook intro text and clean up import cells - Add 'First, we'll need to import...' text to all notebooks missing it - Remove '# Import pieces from codeflare-sdk' and '# Imports' comments - Remove stray leading blank lines from import cells - Update 5_submit_rayjob_cr auth cell from oc login to kube-authkit --- .../additional-demos/hf_interactive.ipynb | 907 +++++++++--------- .../additional-demos/local_interactive.ipynb | 497 +++++----- .../additional-demos/ray_job_client.ipynb | 639 ++++++------ demo-notebooks/guided-demos/0_basic_ray.ipynb | 459 ++++----- .../guided-demos/1_cluster_job_client.ipynb | 537 +++++------ .../guided-demos/2_basic_interactive.ipynb | 735 +++++++------- .../guided-demos/3_widget_example.ipynb | 5 +- .../4_rayjob_existing_cluster.ipynb | 37 +- .../guided-demos/5_submit_rayjob_cr.ipynb | 39 +- .../notebook-ex-outputs/0_basic_ray.ipynb | 461 ++++----- .../1_cluster_job_client.ipynb | 5 +- .../2_basic_interactive.ipynb | 5 +- .../preview_nbs/0_basic_ray.ipynb | 461 ++++----- .../preview_nbs/1_cluster_job_client.ipynb | 515 +++++----- .../preview_nbs/2_basic_interactive.ipynb | 691 ++++++------- 15 files changed, 3025 insertions(+), 2968 deletions(-) diff --git a/demo-notebooks/additional-demos/hf_interactive.ipynb b/demo-notebooks/additional-demos/hf_interactive.ipynb index d598a24ba..0e1d15d06 100644 --- a/demo-notebooks/additional-demos/hf_interactive.ipynb +++ b/demo-notebooks/additional-demos/hf_interactive.ipynb @@ -1,454 +1,455 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "d3260669-c7ec-4d06-a655-590c5e7ab152", - "metadata": {}, - "source": [ - "# Transfer learning with Huggingface using CodeFlare" - ] - }, - { - "cell_type": "markdown", - "id": "d4acfb10-1aa1-445d-947e-396ea5ebed1a", - "metadata": {}, - "source": [ - "In this notebook you will learn how to leverage the **[huggingface](https://huggingface.co/)** support in ray ecosystem to carry out a text classification task using transfer learning. We will be referencing the examples **[here](https://huggingface.co/docs/transformers/tasks/sequence_classification)** and **[here](https://docs.ray.io/en/latest/train/getting-started-transformers.html)**." - ] - }, - { - "cell_type": "markdown", - "id": "70b77929-e96c-434e-ada3-8b14795bfbb1", - "metadata": {}, - "source": [ - "The example carries out a text classification task on **[imdb dataset](https://huggingface.co/datasets/imdb)** and tries to classify the movie reviews as positive or negative. Huggingface library provides an easy way to build a model and the dataset to carry out this classification task. In this case we will be using **distilbert-base-uncased** model which is a **BERT** based model." - ] - }, - { - "cell_type": "markdown", - "id": "02593d04-40b9-4a07-a32e-40b649444ab5", - "metadata": {}, - "source": [ - "### Getting all the requirements in place" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c737a768-6e31-4767-a301-60ae932b4ed9", - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0734734b", - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding Ray Cluster).\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "220b9d85-3a3c-4c0c-aaf2-0d866823dcd8", - "metadata": {}, - "outputs": [], - "source": [ - "# Create our cluster and submit\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster_name= \"hfgputest\"\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name=cluster_name, \n", - " head_memory_requests=6,\n", - " head_memory_limits=8,\n", - " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", - " num_workers=1,\n", - " worker_cpu_requests=8, \n", - " worker_cpu_limits=8, \n", - " worker_memory_requests=16, \n", - " worker_memory_limits=16, \n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "markdown", - "id": "12eef53c", - "metadata": {}, - "source": [ - "Next, we want to bring our cluster up, so we call `cluster.apply()` below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2ae1d861-b743-4c05-903b-5799072b942b", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.apply()" - ] - }, - { - "cell_type": "markdown", - "id": "657ebdfb", - "metadata": {}, - "source": [ - "Now, we want to check on the initial status of our resource cluster, then wait until it is finally ready for use." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "4d0db5f5-22f1-4806-ae7e-a0ee865625c1", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1d26275e", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "d2969a4b", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "markdown", - "id": "477ac246", - "metadata": {}, - "source": [ - "Let's quickly verify that the specs of the cluster are as expected." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "06a54428-f186-4c27-948e-4eaf9c0e34b5", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "8ac46c87-70f1-4c70-9648-881151665355", - "metadata": {}, - "outputs": [], - "source": [ - "ray_cluster_uri = cluster.cluster_uri()" - ] - }, - { - "cell_type": "markdown", - "id": "64d65c3c", - "metadata": {}, - "source": [ - "Now we can connect directly to our Ray cluster via the Ray python client:" - ] - }, - { - "cell_type": "markdown", - "id": "44dba6a0-8275-4726-8911-6b6ec467b6a3", - "metadata": {}, - "source": [ - "**NOTE**: Now we have our resource cluster with the desired GPUs, so we can interact with it to train the HuggingFace model." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "4c458589-5a17-47c6-a8db-625427ae4fe7", - "metadata": {}, - "outputs": [], - "source": [ - "#before proceeding make sure the cluster exists and the uri is not empty\n", - "assert ray_cluster_uri, \"Ray cluster needs to be started and set before proceeding\"\n", - "\n", - "import ray\n", - "\n", - "# reset the ray context in case there's already one. \n", - "ray.shutdown()\n", - "# establish connection to ray cluster\n", - "\n", - "#install additional libraries that will be required for this training\n", - "runtime_env = {\"pip\": [\"transformers==4.41.2\", \"datasets==2.17.0\", \"accelerate==0.31.0\", \"scikit-learn==1.5.0\"]}\n", - "\n", - "# NOTE: This will work for in-cluster notebook servers (RHODS/ODH), but not for local machines\n", - "# To see how to connect from your laptop, go to demo-notebooks/additional-demos/local_interactive.ipynb\n", - "ray.init(address=ray_cluster_uri, runtime_env=runtime_env)\n", - "\n", - "print(\"Ray cluster is up and running: \", ray.is_initialized())" - ] - }, - { - "cell_type": "markdown", - "id": "94a38146-1321-4b7b-9152-9ebca4eb9444", - "metadata": {}, - "source": [ - "**NOTE** : in this case since we are running a task for which we need additional pip packages. we can install those by passing them in the `runtime_env` variable" - ] - }, - { - "cell_type": "markdown", - "id": "76a1945b-d6c8-49b8-9a4c-b82724cffba9", - "metadata": {}, - "source": [ - "### Transfer learning code from huggingface" - ] - }, - { - "cell_type": "markdown", - "id": "8bdbe888-4f38-4e9a-ae43-67ce89ff9d42", - "metadata": {}, - "source": [ - "We are using the code based on the examples **[here](https://huggingface.co/docs/transformers/tasks/sequence_classification)** and **[here](https://docs.ray.io/en/latest/train/getting-started-transformers.html)**. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e69994b4-1a13-43fe-b698-2a5374cb941b", - "metadata": {}, - "outputs": [], - "source": [ - "@ray.remote\n", - "def train_fn():\n", - " import os\n", - " import numpy as np\n", - " from datasets import load_dataset, load_metric\n", - " import transformers\n", - " from transformers import (\n", - " Trainer,\n", - " TrainingArguments,\n", - " AutoTokenizer,\n", - " AutoModelForSequenceClassification,\n", - " )\n", - " import ray.train.huggingface.transformers\n", - " from ray.train import ScalingConfig\n", - " from ray.train.torch import TorchTrainer\n", - "\n", - " # When running in a multi-node cluster you will need persistent storage that is accessible across all worker nodes. \n", - " # See www.github.com/project-codeflare/codeflare-sdk/tree/main/docs/s3-compatible-storage.md for more information.\n", - " \n", - " def train_func():\n", - " # Datasets\n", - " dataset = load_dataset(\"imdb\")\n", - " tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")\n", - "\n", - " def tokenize_function(examples):\n", - " return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n", - "\n", - " small_train_dataset = (\n", - " dataset[\"train\"].select(range(100)).map(tokenize_function, batched=True)\n", - " )\n", - " small_eval_dataset = (\n", - " dataset[\"test\"].select(range(100)).map(tokenize_function, batched=True)\n", - " )\n", - "\n", - " # Model\n", - " model = AutoModelForSequenceClassification.from_pretrained(\n", - " \"distilbert-base-uncased\", num_labels=2\n", - " )\n", - "\n", - " def compute_metrics(eval_pred):\n", - " metric = load_metric(\"accuracy\")\n", - " logits, labels = eval_pred\n", - " predictions = np.argmax(logits, axis=-1)\n", - " return metric.compute(predictions=predictions, references=labels)\n", - "\n", - " # Hugging Face Trainer\n", - " training_args = TrainingArguments(\n", - " output_dir=\"test_trainer\",\n", - " evaluation_strategy=\"epoch\",\n", - " save_strategy=\"epoch\",\n", - " report_to=\"none\",\n", - " )\n", - "\n", - " trainer = Trainer(\n", - " model=model,\n", - " args=training_args,\n", - " train_dataset=small_train_dataset,\n", - " eval_dataset=small_eval_dataset,\n", - " compute_metrics=compute_metrics,\n", - " )\n", - "\n", - "\n", - " callback = ray.train.huggingface.transformers.RayTrainReportCallback()\n", - " trainer.add_callback(callback)\n", - "\n", - " trainer = ray.train.huggingface.transformers.prepare_trainer(trainer)\n", - "\n", - " trainer.train()\n", - "\n", - "\n", - " ray_trainer = TorchTrainer(\n", - " train_func,\n", - " scaling_config=ScalingConfig(num_workers=2, use_gpu=True),\n", - " # Configure persistent storage that is accessible across \n", - " # all worker nodes.\n", - " # Uncomment and update the RunConfig below to include your storage details.\n", - " # run_config=ray.train.RunConfig(storage_path=\"storage path\"),\n", - " )\n", - " result: ray.train.Result = ray_trainer.fit()" - ] - }, - { - "cell_type": "markdown", - "id": "f9593fee-2b2b-415f-8902-bceec014385f", - "metadata": {}, - "source": [ - "**NOTE:** This code will produce a lot of output and will run for **approximately 2 minutes.** As a part of execution it will download the `imdb` dataset, `distilbert-base-uncased` model and then will start transfer learning task for training the model with this dataset. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7f0985e9-5e88-4d36-ab38-c3001c13f97c", - "metadata": {}, - "outputs": [], - "source": [ - "#call the above cell as a remote ray function\n", - "ray.get(train_fn.remote())" - ] - }, - { - "cell_type": "markdown", - "id": "5af8cd32", - "metadata": {}, - "source": [ - "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "ec421113-0e49-4043-a3b5-66efa5021cdd", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "61bf4946", - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - }, - { - "cell_type": "markdown", - "id": "2b7a183b-5e8e-4adb-b9a6-a349e13512a0", - "metadata": {}, - "source": [ - "## Conclusion\n", - "As shown in the above example, you can run your Huggingface transfer learning tasks easily and natively on CodeFlare. You can scale them from 1 to n GPUs without requiring you to make any significant code changes and leveraging the native Huggingface trainer. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2677c868-a052-4893-9493-6f1dacd8fa27", - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.13" - }, - "vscode": { - "interpreter": { - "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Transfer learning with Huggingface using CodeFlare" + ], + "id": "d3260669-c7ec-4d06-a655-590c5e7ab152" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook you will learn how to leverage the **[huggingface](https://huggingface.co/)** support in ray ecosystem to carry out a text classification task using transfer learning. We will be referencing the examples **[here](https://huggingface.co/docs/transformers/tasks/sequence_classification)** and **[here](https://docs.ray.io/en/latest/train/getting-started-transformers.html)**." + ], + "id": "d4acfb10-1aa1-445d-947e-396ea5ebed1a" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The example carries out a text classification task on **[imdb dataset](https://huggingface.co/datasets/imdb)** and tries to classify the movie reviews as positive or negative. Huggingface library provides an easy way to build a model and the dataset to carry out this classification task. In this case we will be using **distilbert-base-uncased** model which is a **BERT** based model." + ], + "id": "70b77929-e96c-434e-ada3-8b14795bfbb1" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Getting all the requirements in place\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ], + "id": "02593d04-40b9-4a07-a32e-40b649444ab5" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [], + "id": "c737a768-6e31-4767-a301-60ae932b4ed9" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [], + "id": "0734734b" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding Ray Cluster).\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create our cluster and submit\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster_name= \"hfgputest\"\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name=cluster_name, \n", + " head_memory_requests=6,\n", + " head_memory_limits=8,\n", + " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", + " num_workers=1,\n", + " worker_cpu_requests=8, \n", + " worker_cpu_limits=8, \n", + " worker_memory_requests=16, \n", + " worker_memory_limits=16, \n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [], + "id": "220b9d85-3a3c-4c0c-aaf2-0d866823dcd8" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we want to bring our cluster up, so we call `cluster.apply()` below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster." + ], + "id": "12eef53c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.apply()" + ], + "execution_count": null, + "outputs": [], + "id": "2ae1d861-b743-4c05-903b-5799072b942b" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we want to check on the initial status of our resource cluster, then wait until it is finally ready for use." + ], + "id": "657ebdfb" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "4d0db5f5-22f1-4806-ae7e-a0ee865625c1" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [], + "id": "1d26275e" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "d2969a4b" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's quickly verify that the specs of the cluster are as expected." + ], + "id": "477ac246" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [], + "id": "06a54428-f186-4c27-948e-4eaf9c0e34b5" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "ray_cluster_uri = cluster.cluster_uri()" + ], + "execution_count": null, + "outputs": [], + "id": "8ac46c87-70f1-4c70-9648-881151665355" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can connect directly to our Ray cluster via the Ray python client:" + ], + "id": "64d65c3c" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**NOTE**: Now we have our resource cluster with the desired GPUs, so we can interact with it to train the HuggingFace model." + ], + "id": "44dba6a0-8275-4726-8911-6b6ec467b6a3" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "#before proceeding make sure the cluster exists and the uri is not empty\n", + "assert ray_cluster_uri, \"Ray cluster needs to be started and set before proceeding\"\n", + "\n", + "import ray\n", + "\n", + "# reset the ray context in case there's already one. \n", + "ray.shutdown()\n", + "# establish connection to ray cluster\n", + "\n", + "#install additional libraries that will be required for this training\n", + "runtime_env = {\"pip\": [\"transformers==4.41.2\", \"datasets==2.17.0\", \"accelerate==0.31.0\", \"scikit-learn==1.5.0\"]}\n", + "\n", + "# NOTE: This will work for in-cluster notebook servers (RHODS/ODH), but not for local machines\n", + "# To see how to connect from your laptop, go to demo-notebooks/additional-demos/local_interactive.ipynb\n", + "ray.init(address=ray_cluster_uri, runtime_env=runtime_env)\n", + "\n", + "print(\"Ray cluster is up and running: \", ray.is_initialized())" + ], + "execution_count": null, + "outputs": [], + "id": "4c458589-5a17-47c6-a8db-625427ae4fe7" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**NOTE** : in this case since we are running a task for which we need additional pip packages. we can install those by passing them in the `runtime_env` variable" + ], + "id": "94a38146-1321-4b7b-9152-9ebca4eb9444" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Transfer learning code from huggingface" + ], + "id": "76a1945b-d6c8-49b8-9a4c-b82724cffba9" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We are using the code based on the examples **[here](https://huggingface.co/docs/transformers/tasks/sequence_classification)** and **[here](https://docs.ray.io/en/latest/train/getting-started-transformers.html)**. " + ], + "id": "8bdbe888-4f38-4e9a-ae43-67ce89ff9d42" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "@ray.remote\n", + "def train_fn():\n", + " import os\n", + " import numpy as np\n", + " from datasets import load_dataset, load_metric\n", + " import transformers\n", + " from transformers import (\n", + " Trainer,\n", + " TrainingArguments,\n", + " AutoTokenizer,\n", + " AutoModelForSequenceClassification,\n", + " )\n", + " import ray.train.huggingface.transformers\n", + " from ray.train import ScalingConfig\n", + " from ray.train.torch import TorchTrainer\n", + "\n", + " # When running in a multi-node cluster you will need persistent storage that is accessible across all worker nodes. \n", + " # See www.github.com/project-codeflare/codeflare-sdk/tree/main/docs/s3-compatible-storage.md for more information.\n", + " \n", + " def train_func():\n", + " # Datasets\n", + " dataset = load_dataset(\"imdb\")\n", + " tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")\n", + "\n", + " def tokenize_function(examples):\n", + " return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n", + "\n", + " small_train_dataset = (\n", + " dataset[\"train\"].select(range(100)).map(tokenize_function, batched=True)\n", + " )\n", + " small_eval_dataset = (\n", + " dataset[\"test\"].select(range(100)).map(tokenize_function, batched=True)\n", + " )\n", + "\n", + " # Model\n", + " model = AutoModelForSequenceClassification.from_pretrained(\n", + " \"distilbert-base-uncased\", num_labels=2\n", + " )\n", + "\n", + " def compute_metrics(eval_pred):\n", + " metric = load_metric(\"accuracy\")\n", + " logits, labels = eval_pred\n", + " predictions = np.argmax(logits, axis=-1)\n", + " return metric.compute(predictions=predictions, references=labels)\n", + "\n", + " # Hugging Face Trainer\n", + " training_args = TrainingArguments(\n", + " output_dir=\"test_trainer\",\n", + " evaluation_strategy=\"epoch\",\n", + " save_strategy=\"epoch\",\n", + " report_to=\"none\",\n", + " )\n", + "\n", + " trainer = Trainer(\n", + " model=model,\n", + " args=training_args,\n", + " train_dataset=small_train_dataset,\n", + " eval_dataset=small_eval_dataset,\n", + " compute_metrics=compute_metrics,\n", + " )\n", + "\n", + "\n", + " callback = ray.train.huggingface.transformers.RayTrainReportCallback()\n", + " trainer.add_callback(callback)\n", + "\n", + " trainer = ray.train.huggingface.transformers.prepare_trainer(trainer)\n", + "\n", + " trainer.train()\n", + "\n", + "\n", + " ray_trainer = TorchTrainer(\n", + " train_func,\n", + " scaling_config=ScalingConfig(num_workers=2, use_gpu=True),\n", + " # Configure persistent storage that is accessible across \n", + " # all worker nodes.\n", + " # Uncomment and update the RunConfig below to include your storage details.\n", + " # run_config=ray.train.RunConfig(storage_path=\"storage path\"),\n", + " )\n", + " result: ray.train.Result = ray_trainer.fit()" + ], + "execution_count": null, + "outputs": [], + "id": "e69994b4-1a13-43fe-b698-2a5374cb941b" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**NOTE:** This code will produce a lot of output and will run for **approximately 2 minutes.** As a part of execution it will download the `imdb` dataset, `distilbert-base-uncased` model and then will start transfer learning task for training the model with this dataset. " + ], + "id": "f9593fee-2b2b-415f-8902-bceec014385f" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "#call the above cell as a remote ray function\n", + "ray.get(train_fn.remote())" + ], + "execution_count": null, + "outputs": [], + "id": "7f0985e9-5e88-4d36-ab38-c3001c13f97c" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." + ], + "id": "5af8cd32" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [], + "id": "ec421113-0e49-4043-a3b5-66efa5021cdd" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [], + "id": "61bf4946" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "As shown in the above example, you can run your Huggingface transfer learning tasks easily and natively on CodeFlare. You can scale them from 1 to n GPUs without requiring you to make any significant code changes and leveraging the native Huggingface trainer. " + ], + "id": "2b7a183b-5e8e-4adb-b9a6-a349e13512a0" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [], + "execution_count": null, + "outputs": [], + "id": "2677c868-a052-4893-9493-6f1dacd8fa27" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.13" + }, + "vscode": { + "interpreter": { + "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/demo-notebooks/additional-demos/local_interactive.ipynb b/demo-notebooks/additional-demos/local_interactive.ipynb index 0b08d824d..b90dc7175 100644 --- a/demo-notebooks/additional-demos/local_interactive.ipynb +++ b/demo-notebooks/additional-demos/local_interactive.ipynb @@ -1,246 +1,255 @@ { - "cells": [ - { - "cell_type": "code", - "execution_count": null, - "id": "9a44568b-61ef-41c7-8ad1-9a3b128f03a7", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook, we will demonstrate how to connect to a Ray cluster interactively from your local machine using the Ray client route.\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ], + "id": "82b0d2d9" + }, + { + "cell_type": "code", + "metadata": { + "tags": [] + }, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [], + "id": "9a44568b-61ef-41c7-8ad1-9a3b128f03a7" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [], + "id": "2cc66278" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "18de2d65" + }, + { + "cell_type": "code", + "metadata": { + "tags": [] + }, + "source": [ + "# Create and submit our cluster\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster_name = \"hfgputest-1\"\n", + "\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name=cluster_name,\n", + " head_memory_requests=6,\n", + " head_memory_limits=8,\n", + " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", + " num_workers=1,\n", + " worker_cpu_requests=1,\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=6,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [], + "id": "4364ac2e-dd10-4d30-ba66-12708daefb3f" + }, + { + "cell_type": "code", + "metadata": { + "tags": [] + }, + "source": [ + "cluster.apply()" + ], + "execution_count": null, + "outputs": [], + "id": "69968140-15e6-482f-9529-82b0cd19524b" + }, + { + "cell_type": "code", + "metadata": { + "tags": [] + }, + "source": [ + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [], + "id": "e20f9982-f671-460b-8c22-3d62e101fed9" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Connect via the rayclient route" + ], + "id": "12eef53c" + }, + { + "cell_type": "code", + "metadata": { + "tags": [] + }, + "source": [ + "import ray\n", + "\n", + "ray.shutdown()\n", + "ray.init(address=cluster.local_client_url(), logging_level=\"DEBUG\")" + ], + "execution_count": null, + "outputs": [], + "id": "9483bb98-33b3-4beb-9b15-163d7e76c1d7" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "import math\n", + "import ray\n", + "\n", + "@ray.remote\n", + "def heavy_calculation_part(num_iterations):\n", + " result = 0.0\n", + " for i in range(num_iterations):\n", + " for j in range(num_iterations):\n", + " for k in range(num_iterations):\n", + " result += math.sin(i) * math.cos(j) * math.tan(k)\n", + " return result\n", + "@ray.remote\n", + "def heavy_calculation(num_iterations):\n", + " results = ray.get([heavy_calculation_part.remote(num_iterations//30) for _ in range(30)])\n", + " return sum(results)\n" + ], + "execution_count": null, + "outputs": [], + "id": "3436eb4a-217c-4109-a3c3-309fda7e2442" + }, + { + "cell_type": "code", + "metadata": { + "tags": [] + }, + "source": [ + "ref = heavy_calculation.remote(3000)" + ], + "execution_count": null, + "outputs": [], + "id": "5cca1874-2be3-4631-ae48-9adfa45e3af3" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "ray.get(ref)" + ], + "execution_count": null, + "outputs": [], + "id": "01172c29-e8bf-41ef-8db5-eccb07906111" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "ray.cancel(ref)\n", + "ray.shutdown()" + ], + "execution_count": null, + "outputs": [], + "id": "9e79b547-a457-4232-b77d-19147067b972" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [], + "id": "2c198f1f-68bf-43ff-a148-02b5cb000ff2" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [], + "execution_count": null, + "outputs": [], + "id": "6879e471-a69f-4c74-9cec-a195cdead47c" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + }, + "vscode": { + "interpreter": { + "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" + } + } }, - { - "cell_type": "code", - "execution_count": null, - "id": "2cc66278", - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "18de2d65", - "metadata": {}, - "source": [ - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "4364ac2e-dd10-4d30-ba66-12708daefb3f", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# Create and submit our cluster\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster_name = \"hfgputest-1\"\n", - "\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name=cluster_name,\n", - " head_memory_requests=6,\n", - " head_memory_limits=8,\n", - " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", - " num_workers=1,\n", - " worker_cpu_requests=1,\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=6,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "69968140-15e6-482f-9529-82b0cd19524b", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "cluster.apply()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "e20f9982-f671-460b-8c22-3d62e101fed9", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "cluster.wait_ready()" - ] - }, - { - "cell_type": "markdown", - "id": "12eef53c", - "metadata": {}, - "source": [ - "### Connect via the rayclient route" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9483bb98-33b3-4beb-9b15-163d7e76c1d7", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "import ray\n", - "\n", - "ray.shutdown()\n", - "ray.init(address=cluster.local_client_url(), logging_level=\"DEBUG\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3436eb4a-217c-4109-a3c3-309fda7e2442", - "metadata": {}, - "outputs": [], - "source": [ - "import math\n", - "import ray\n", - "\n", - "@ray.remote\n", - "def heavy_calculation_part(num_iterations):\n", - " result = 0.0\n", - " for i in range(num_iterations):\n", - " for j in range(num_iterations):\n", - " for k in range(num_iterations):\n", - " result += math.sin(i) * math.cos(j) * math.tan(k)\n", - " return result\n", - "@ray.remote\n", - "def heavy_calculation(num_iterations):\n", - " results = ray.get([heavy_calculation_part.remote(num_iterations//30) for _ in range(30)])\n", - " return sum(results)\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5cca1874-2be3-4631-ae48-9adfa45e3af3", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "ref = heavy_calculation.remote(3000)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "01172c29-e8bf-41ef-8db5-eccb07906111", - "metadata": {}, - "outputs": [], - "source": [ - "ray.get(ref)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "9e79b547-a457-4232-b77d-19147067b972", - "metadata": {}, - "outputs": [], - "source": [ - "ray.cancel(ref)\n", - "ray.shutdown()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2c198f1f-68bf-43ff-a148-02b5cb000ff2", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "6879e471-a69f-4c74-9cec-a195cdead47c", - "metadata": {}, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" - }, - "vscode": { - "interpreter": { - "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/demo-notebooks/additional-demos/ray_job_client.ipynb b/demo-notebooks/additional-demos/ray_job_client.ipynb index e4f637a89..4cdb1502b 100644 --- a/demo-notebooks/additional-demos/ray_job_client.ipynb +++ b/demo-notebooks/additional-demos/ray_job_client.ipynb @@ -1,322 +1,323 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this demo we will go over the basics of the RayJobClient in the SDK" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this demo we will go over the basics of the RayJobClient in the SDK\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, RayJobClient, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "# Note: auth_token is also used below as the bearer token for the RayJobClient\n", + "auth_token = \"sha256~XXXXX\" # oc whoami -t\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=auth_token,\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True,\n", + "# )\n", + "# auth_token = ... # Retrieve access token from your OIDC provider for RayJobClient\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ] + }, + { + "cell_type": "markdown", + "id": "18de2d65", + "metadata": {}, + "source": [ + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name='jobtest',\n", + " head_memory_requests=6,\n", + " head_memory_limits=8,\n", + " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", + " num_workers=2,\n", + " worker_cpu_requests=1,\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=6,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Bring up the cluster\n", + "cluster.apply()\n", + "cluster.wait_ready()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "cluster.details()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Ray Job Submission - Authorized Ray Cluster" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* Submit a job using an authorized Ray dashboard and the Job Submission Client\n", + "* Provide an entrypoint command directed to your job script\n", + "* Set up your runtime environment" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Gather the dashboard URL\n", + "ray_dashboard = cluster.cluster_dashboard_uri()\n", + "\n", + "# Create the header for passing your bearer token\n", + "header = {\n", + " 'Authorization': f'Bearer {auth_token}'\n", + "}\n", + "\n", + "# Initialize the RayJobClient\n", + "client = RayJobClient(address=ray_dashboard, headers=header, verify=True)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit an example mnist job using the RayJobClient\n", + "submission_id = client.submit_job(\n", + " entrypoint=\"python mnist.py\",\n", + " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", + ")\n", + "print(submission_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Get the job's logs\n", + "client.get_job_logs(submission_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Get the job's status\n", + "client.get_job_status(submission_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Get job related info\n", + "client.get_job_info(submission_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# List all existing jobs\n", + "client.list_jobs()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Iterate through the logs of a job \n", + "async for lines in client.tail_job_logs(submission_id):\n", + " print(lines, end=\"\") " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Delete a job\n", + "# Can run client.cancel_job(submission_id) first if job is still running\n", + "client.delete_job(submission_id)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Unauthorized Ray Cluster with the Ray Job Client" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\"\"\"\n", + "Initialise the RayJobClient with the Ray Dashboard\n", + "\"\"\"\n", + "ray_dashboard = cluster.cluster_dashboard_uri()\n", + "client = RayJobClient(address=ray_dashboard, verify=False)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Submit an example mnist job using the RayJobClient\n", + "submission_id = client.submit_job(\n", + " entrypoint=\"python mnist.py\",\n", + " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", + ")\n", + "print(submission_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Stop the job \n", + "client.stop_job(submission_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Delete the job\n", + "client.delete_job(submission_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "cluster.down()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + } }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, RayJobClient, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "# Note: auth_token is also used below as the bearer token for the RayJobClient\n", - "auth_token = \"sha256~XXXXX\" # oc whoami -t\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=auth_token,\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True,\n", - "# )\n", - "# auth_token = ... # Retrieve access token from your OIDC provider for RayJobClient\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "18de2d65", - "metadata": {}, - "source": [ - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name='jobtest',\n", - " head_memory_requests=6,\n", - " head_memory_limits=8,\n", - " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", - " num_workers=2,\n", - " worker_cpu_requests=1,\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=6,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()\n", - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Ray Job Submission - Authorized Ray Cluster" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "* Submit a job using an authorized Ray dashboard and the Job Submission Client\n", - "* Provide an entrypoint command directed to your job script\n", - "* Set up your runtime environment" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Gather the dashboard URL\n", - "ray_dashboard = cluster.cluster_dashboard_uri()\n", - "\n", - "# Create the header for passing your bearer token\n", - "header = {\n", - " 'Authorization': f'Bearer {auth_token}'\n", - "}\n", - "\n", - "# Initialize the RayJobClient\n", - "client = RayJobClient(address=ray_dashboard, headers=header, verify=True)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Submit an example mnist job using the RayJobClient\n", - "submission_id = client.submit_job(\n", - " entrypoint=\"python mnist.py\",\n", - " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", - ")\n", - "print(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get the job's logs\n", - "client.get_job_logs(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get the job's status\n", - "client.get_job_status(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get job related info\n", - "client.get_job_info(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# List all existing jobs\n", - "client.list_jobs()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Iterate through the logs of a job \n", - "async for lines in client.tail_job_logs(submission_id):\n", - " print(lines, end=\"\") " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Delete a job\n", - "# Can run client.cancel_job(submission_id) first if job is still running\n", - "client.delete_job(submission_id)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Unauthorized Ray Cluster with the Ray Job Client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "\"\"\"\n", - "Initialise the RayJobClient with the Ray Dashboard\n", - "\"\"\"\n", - "ray_dashboard = cluster.cluster_dashboard_uri()\n", - "client = RayJobClient(address=ray_dashboard, verify=False)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Submit an example mnist job using the RayJobClient\n", - "submission_id = client.submit_job(\n", - " entrypoint=\"python mnist.py\",\n", - " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", - ")\n", - "print(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Stop the job \n", - "client.stop_job(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Delete the job\n", - "client.delete_job(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" - } - }, - "nbformat": 4, - "nbformat_minor": 2 + "nbformat": 4, + "nbformat_minor": 2 } diff --git a/demo-notebooks/guided-demos/0_basic_ray.ipynb b/demo-notebooks/guided-demos/0_basic_ray.ipynb index 8eb47e948..823506be5 100644 --- a/demo-notebooks/guided-demos/0_basic_ray.ipynb +++ b/demo-notebooks/guided-demos/0_basic_ray.ipynb @@ -1,230 +1,231 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "8d4a42f6", - "metadata": {}, - "source": [ - "In this notebook, we will go through the basics of using the SDK to:\n", - " - Spin up a Ray cluster with our desired resources\n", - " - View the status and specs of our Ray cluster\n", - " - Take down the Ray cluster when finished" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a", - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "614daa0c", - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0f4bc870-091f-4e11-9642-cba145710159", - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name='raytest', \n", - " head_cpu_requests='500m',\n", - " head_cpu_limits='500m',\n", - " head_memory_requests=5,\n", - " head_memory_limits=8,\n", - " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", - " num_workers=2,\n", - " worker_cpu_requests='250m',\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=6,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "markdown", - "id": "12eef53c", - "metadata": {}, - "source": [ - "To create the Ray Cluster, we can click the `Cluster Up` button to submit our Ray Cluster onto the queue, and begin the process of creating a Ray Cluster resource. Alternatively, you can run the code cell below to do the same." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200", - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()" - ] - }, - { - "cell_type": "markdown", - "id": "657ebdfb", - "metadata": {}, - "source": [ - "Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3c1b4311-2e61-44c9-8225-87c2db11363d", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a99d5aff", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "df71c1ed", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "markdown", - "id": "b3a55fe4", - "metadata": {}, - "source": [ - "Let's quickly verify that the specs of the cluster are as expected." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "id": "5af8cd32", - "metadata": {}, - "source": [ - "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0d41b90e", - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.11" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook, we will go through the basics of using the SDK to:\n", + " - Spin up a Ray cluster with our desired resources\n", + " - View the status and specs of our Ray cluster\n", + " - Take down the Ray cluster when finished\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ], + "id": "8d4a42f6" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [], + "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [], + "id": "614daa0c" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name='raytest', \n", + " head_cpu_requests='500m',\n", + " head_cpu_limits='500m',\n", + " head_memory_requests=5,\n", + " head_memory_limits=8,\n", + " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", + " num_workers=2,\n", + " worker_cpu_requests='250m',\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=6,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [], + "id": "0f4bc870-091f-4e11-9642-cba145710159" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To create the Ray Cluster, we can click the `Cluster Up` button to submit our Ray Cluster onto the queue, and begin the process of creating a Ray Cluster resource. Alternatively, you can run the code cell below to do the same." + ], + "id": "12eef53c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Bring up the cluster\n", + "cluster.apply()" + ], + "execution_count": null, + "outputs": [], + "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use." + ], + "id": "657ebdfb" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "3c1b4311-2e61-44c9-8225-87c2db11363d" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [], + "id": "a99d5aff" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "df71c1ed" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's quickly verify that the specs of the cluster are as expected." + ], + "id": "b3a55fe4" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [], + "id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." + ], + "id": "5af8cd32" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [], + "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [], + "id": "0d41b90e" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/demo-notebooks/guided-demos/1_cluster_job_client.ipynb b/demo-notebooks/guided-demos/1_cluster_job_client.ipynb index 9a807f749..812c87eca 100644 --- a/demo-notebooks/guided-demos/1_cluster_job_client.ipynb +++ b/demo-notebooks/guided-demos/1_cluster_job_client.ipynb @@ -1,270 +1,271 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this demo we will go over the basics of the Ray Job Submission Client in the SDK" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this demo we will go over the basics of the Ray Job Submission Client in the SDK\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name='jobtest',\n", + " head_cpu_requests=1,\n", + " head_cpu_limits=1,\n", + " head_memory_requests=6,\n", + " head_memory_limits=8,\n", + " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", + " num_workers=2,\n", + " worker_cpu_requests='250m',\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=6,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To create the Ray Cluster, we can tick the `Wait for cluster?` checkbox and click the `Cluster Up` button to submit our Ray Cluster onto the queue, and begin the process of creating a Ray Cluster resource while waiting for the Ray Dashboard to be available. Alternatively, you can run the code cell below to do the same." + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Bring up the cluster\n", + "cluster.apply()\n", + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Ray Job Submission" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* Initialise the Cluster Job Client \n", + "* Provide an entrypoint command directed to your job script\n", + "* Set up your runtime environment" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Initialize the Job Submission Client\n", + "\"\"\"\n", + "The SDK will automatically gather the dashboard address and authenticate using the Ray Job Submission Client\n", + "\"\"\"\n", + "client = cluster.job_client" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Submit an example mnist job using the Job Submission Client\n", + "submission_id = client.submit_job(\n", + " entrypoint=\"python mnist_fashion.py\",\n", + " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", + ")\n", + "print(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Get the job's logs\n", + "client.get_job_logs(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Get the job's status\n", + "client.get_job_status(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Get job related info\n", + "client.get_job_info(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# List all existing jobs\n", + "client.list_jobs()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Iterate through the logs of a job \n", + "async for lines in client.tail_job_logs(submission_id):\n", + " print(lines, end=\"\") " + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Delete a job\n", + "# Can run client.stop_job(submission_id) first if job is still running\n", + "client.delete_job(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name='jobtest',\n", - " head_cpu_requests=1,\n", - " head_cpu_limits=1,\n", - " head_memory_requests=6,\n", - " head_memory_limits=8,\n", - " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", - " num_workers=2,\n", - " worker_cpu_requests='250m',\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=6,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To create the Ray Cluster, we can tick the `Wait for cluster?` checkbox and click the `Cluster Up` button to submit our Ray Cluster onto the queue, and begin the process of creating a Ray Cluster resource while waiting for the Ray Dashboard to be available. Alternatively, you can run the code cell below to do the same." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()\n", - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Ray Job Submission" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "* Initialise the Cluster Job Client \n", - "* Provide an entrypoint command directed to your job script\n", - "* Set up your runtime environment" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Initialize the Job Submission Client\n", - "\"\"\"\n", - "The SDK will automatically gather the dashboard address and authenticate using the Ray Job Submission Client\n", - "\"\"\"\n", - "client = cluster.job_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Submit an example mnist job using the Job Submission Client\n", - "submission_id = client.submit_job(\n", - " entrypoint=\"python mnist_fashion.py\",\n", - " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", - ")\n", - "print(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get the job's logs\n", - "client.get_job_logs(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get the job's status\n", - "client.get_job_status(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get job related info\n", - "client.get_job_info(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# List all existing jobs\n", - "client.list_jobs()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Iterate through the logs of a job \n", - "async for lines in client.tail_job_logs(submission_id):\n", - " print(lines, end=\"\") " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Delete a job\n", - "# Can run client.stop_job(submission_id) first if job is still running\n", - "client.delete_job(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.12.11" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} + "nbformat": 4, + "nbformat_minor": 2 +} \ No newline at end of file diff --git a/demo-notebooks/guided-demos/2_basic_interactive.ipynb b/demo-notebooks/guided-demos/2_basic_interactive.ipynb index 136a149f9..176f0ad12 100644 --- a/demo-notebooks/guided-demos/2_basic_interactive.ipynb +++ b/demo-notebooks/guided-demos/2_basic_interactive.ipynb @@ -1,369 +1,370 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "bbc21043", - "metadata": {}, - "source": [ - "In this notebook, we will go over how to leverage the SDK to directly work interactively with a Ray Cluster during development." - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook, we will go over how to leverage the SDK to directly work interactively with a Ray Cluster during development.\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ], + "id": "bbc21043" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [], + "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [], + "id": "614daa0c" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once again, let's start by running through the same cluster setup as before:\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster_name = \"interactivetest\"\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name=cluster_name,\n", + " head_cpu_requests=1,\n", + " head_cpu_limits=1,\n", + " head_memory_requests=6,\n", + " head_memory_limits=8,\n", + " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", + " num_workers=2,\n", + " worker_cpu_requests='250m',\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=6,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [], + "id": "0f4bc870-091f-4e11-9642-cba145710159" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To create the Ray Cluster, we can tick the `Wait for cluster?` checkbox and click the `Cluster Up` button to submit our Ray Cluster onto the queue, and begin the process of creating a Ray Cluster resource while waiting for the Ray Dashboard to be available. Alternatively, you can run the code cell below to do the same." + ], + "id": "6973247b" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Bring up the cluster\n", + "cluster.apply()\n", + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [], + "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [], + "id": "df71c1ed" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This time we will demonstrate another potential method of use: working with the Ray cluster interactively.\n", + "\n", + "Using the SDK, we can get both the Ray cluster URI and dashboard URI:" + ], + "id": "33663f47" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "ray_dashboard_uri = cluster.cluster_dashboard_uri()\n", + "ray_cluster_uri = cluster.cluster_uri()\n", + "print(ray_dashboard_uri)\n", + "print(ray_cluster_uri)" + ], + "execution_count": null, + "outputs": [], + "id": "c1719bca" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can connect directly to our Ray cluster via the Ray python client:" + ], + "id": "2a2aca6a" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# before proceeding make sure the cluster exists and the uri is not empty\n", + "assert ray_cluster_uri, \"Ray cluster needs to be started and set before proceeding\"\n", + "\n", + "import ray\n", + "\n", + "# reset the ray context in case there's already one. \n", + "ray.shutdown()\n", + "# establish connection to ray cluster\n", + "\n", + "# install additional libraries that will be required for model training\n", + "runtime_env = {\"pip\": [\"transformers==4.41.2\", \"datasets==2.17.0\", \"accelerate==0.31.0\", \"scikit-learn==1.5.0\"]}\n", + "# NOTE: This will work for in-cluster notebook servers (RHODS/ODH), but not for local machines\n", + "# To see how to connect from your laptop, go to demo-notebooks/additional-demos/local_interactive.ipynb\n", + "ray.init(address=ray_cluster_uri, runtime_env=runtime_env)\n", + "\n", + "print(\"Ray cluster is up and running: \", ray.is_initialized())" + ], + "execution_count": null, + "outputs": [], + "id": "300146dc" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we are connected (and have passed in some package requirements), let's try writing some training code:" + ], + "id": "9711030b" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "@ray.remote\n", + "def train_fn():\n", + " import os\n", + " import numpy as np\n", + " from datasets import load_dataset, load_metric\n", + " import transformers\n", + " from transformers import (\n", + " Trainer,\n", + " TrainingArguments,\n", + " AutoTokenizer,\n", + " AutoModelForSequenceClassification,\n", + " )\n", + " import ray.train.huggingface.transformers\n", + " from ray.train import ScalingConfig\n", + " from ray.train.torch import TorchTrainer\n", + "\n", + " # When running in a multi-node cluster you will need persistent storage that is accessible across all worker nodes. \n", + " # See www.github.com/project-codeflare/codeflare-sdk/tree/main/docs/s3-compatible-storage.md for more information.\n", + " \n", + " def train_func():\n", + " # Datasets\n", + " dataset = load_dataset(\"imdb\")\n", + " tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")\n", + "\n", + " def tokenize_function(examples):\n", + " return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n", + "\n", + " small_train_dataset = (\n", + " dataset[\"train\"].select(range(100)).map(tokenize_function, batched=True)\n", + " )\n", + " small_eval_dataset = (\n", + " dataset[\"test\"].select(range(100)).map(tokenize_function, batched=True)\n", + " )\n", + "\n", + " # Model\n", + " model = AutoModelForSequenceClassification.from_pretrained(\n", + " \"distilbert-base-uncased\", num_labels=2\n", + " )\n", + "\n", + " def compute_metrics(eval_pred):\n", + " metric = load_metric(\"accuracy\")\n", + " logits, labels = eval_pred\n", + " predictions = np.argmax(logits, axis=-1)\n", + " return metric.compute(predictions=predictions, references=labels)\n", + "\n", + " # Hugging Face Trainer\n", + " training_args = TrainingArguments(\n", + " output_dir=\"test_trainer\",\n", + " evaluation_strategy=\"epoch\",\n", + " save_strategy=\"epoch\",\n", + " report_to=\"none\",\n", + " )\n", + "\n", + " trainer = Trainer(\n", + " model=model,\n", + " args=training_args,\n", + " train_dataset=small_train_dataset,\n", + " eval_dataset=small_eval_dataset,\n", + " compute_metrics=compute_metrics,\n", + " )\n", + "\n", + "\n", + " callback = ray.train.huggingface.transformers.RayTrainReportCallback()\n", + " trainer.add_callback(callback)\n", + "\n", + " trainer = ray.train.huggingface.transformers.prepare_trainer(trainer)\n", + "\n", + " trainer.train()\n", + "\n", + "\n", + " ray_trainer = TorchTrainer(\n", + " train_func,\n", + " scaling_config=ScalingConfig(\n", + " # num_workers = number of worker nodes with the ray head node included\n", + " num_workers=3,\n", + " use_gpu=True,\n", + " resources_per_worker={\n", + " \"CPU\": 1,\n", + " },\n", + " trainer_resources={\n", + " \"CPU\": 0,\n", + " }\n", + " )\n", + " # Configure persistent storage that is accessible across \n", + " # all worker nodes.\n", + " # Uncomment and update the RunConfig below to include your storage details.\n", + " # run_config=ray.train.RunConfig(storage_path=\"storage path\"),\n", + " )\n", + " result: ray.train.Result = ray_trainer.fit()" + ], + "execution_count": null, + "outputs": [], + "id": "1b36e0d9" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once we want to test our code out, we can run the training function we defined above remotely on our Ray cluster:" + ], + "id": "d4d8fd65" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "#call the above cell as a remote ray function\n", + "ray.get(train_fn.remote())" + ], + "execution_count": null, + "outputs": [], + "id": "5901d958" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once complete, we can bring our Ray cluster down and clean up:" + ], + "id": "5af8cd32" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [], + "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [], + "id": "0d41b90e" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.13" + }, + "vscode": { + "interpreter": { + "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" + } + } }, - { - "cell_type": "code", - "execution_count": null, - "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a", - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "614daa0c", - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Once again, let's start by running through the same cluster setup as before:\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0f4bc870-091f-4e11-9642-cba145710159", - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster_name = \"interactivetest\"\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name=cluster_name,\n", - " head_cpu_requests=1,\n", - " head_cpu_limits=1,\n", - " head_memory_requests=6,\n", - " head_memory_limits=8,\n", - " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", - " num_workers=2,\n", - " worker_cpu_requests='250m',\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=6,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "markdown", - "id": "6973247b", - "metadata": {}, - "source": [ - "To create the Ray Cluster, we can tick the `Wait for cluster?` checkbox and click the `Cluster Up` button to submit our Ray Cluster onto the queue, and begin the process of creating a Ray Cluster resource while waiting for the Ray Dashboard to be available. Alternatively, you can run the code cell below to do the same." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200", - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()\n", - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "df71c1ed", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "id": "33663f47", - "metadata": {}, - "source": [ - "This time we will demonstrate another potential method of use: working with the Ray cluster interactively.\n", - "\n", - "Using the SDK, we can get both the Ray cluster URI and dashboard URI:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c1719bca", - "metadata": {}, - "outputs": [], - "source": [ - "ray_dashboard_uri = cluster.cluster_dashboard_uri()\n", - "ray_cluster_uri = cluster.cluster_uri()\n", - "print(ray_dashboard_uri)\n", - "print(ray_cluster_uri)" - ] - }, - { - "cell_type": "markdown", - "id": "2a2aca6a", - "metadata": {}, - "source": [ - "Now we can connect directly to our Ray cluster via the Ray python client:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "300146dc", - "metadata": {}, - "outputs": [], - "source": [ - "# before proceeding make sure the cluster exists and the uri is not empty\n", - "assert ray_cluster_uri, \"Ray cluster needs to be started and set before proceeding\"\n", - "\n", - "import ray\n", - "\n", - "# reset the ray context in case there's already one. \n", - "ray.shutdown()\n", - "# establish connection to ray cluster\n", - "\n", - "# install additional libraries that will be required for model training\n", - "runtime_env = {\"pip\": [\"transformers==4.41.2\", \"datasets==2.17.0\", \"accelerate==0.31.0\", \"scikit-learn==1.5.0\"]}\n", - "# NOTE: This will work for in-cluster notebook servers (RHODS/ODH), but not for local machines\n", - "# To see how to connect from your laptop, go to demo-notebooks/additional-demos/local_interactive.ipynb\n", - "ray.init(address=ray_cluster_uri, runtime_env=runtime_env)\n", - "\n", - "print(\"Ray cluster is up and running: \", ray.is_initialized())" - ] - }, - { - "cell_type": "markdown", - "id": "9711030b", - "metadata": {}, - "source": [ - "Now that we are connected (and have passed in some package requirements), let's try writing some training code:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1b36e0d9", - "metadata": {}, - "outputs": [], - "source": [ - "@ray.remote\n", - "def train_fn():\n", - " import os\n", - " import numpy as np\n", - " from datasets import load_dataset, load_metric\n", - " import transformers\n", - " from transformers import (\n", - " Trainer,\n", - " TrainingArguments,\n", - " AutoTokenizer,\n", - " AutoModelForSequenceClassification,\n", - " )\n", - " import ray.train.huggingface.transformers\n", - " from ray.train import ScalingConfig\n", - " from ray.train.torch import TorchTrainer\n", - "\n", - " # When running in a multi-node cluster you will need persistent storage that is accessible across all worker nodes. \n", - " # See www.github.com/project-codeflare/codeflare-sdk/tree/main/docs/s3-compatible-storage.md for more information.\n", - " \n", - " def train_func():\n", - " # Datasets\n", - " dataset = load_dataset(\"imdb\")\n", - " tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")\n", - "\n", - " def tokenize_function(examples):\n", - " return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n", - "\n", - " small_train_dataset = (\n", - " dataset[\"train\"].select(range(100)).map(tokenize_function, batched=True)\n", - " )\n", - " small_eval_dataset = (\n", - " dataset[\"test\"].select(range(100)).map(tokenize_function, batched=True)\n", - " )\n", - "\n", - " # Model\n", - " model = AutoModelForSequenceClassification.from_pretrained(\n", - " \"distilbert-base-uncased\", num_labels=2\n", - " )\n", - "\n", - " def compute_metrics(eval_pred):\n", - " metric = load_metric(\"accuracy\")\n", - " logits, labels = eval_pred\n", - " predictions = np.argmax(logits, axis=-1)\n", - " return metric.compute(predictions=predictions, references=labels)\n", - "\n", - " # Hugging Face Trainer\n", - " training_args = TrainingArguments(\n", - " output_dir=\"test_trainer\",\n", - " evaluation_strategy=\"epoch\",\n", - " save_strategy=\"epoch\",\n", - " report_to=\"none\",\n", - " )\n", - "\n", - " trainer = Trainer(\n", - " model=model,\n", - " args=training_args,\n", - " train_dataset=small_train_dataset,\n", - " eval_dataset=small_eval_dataset,\n", - " compute_metrics=compute_metrics,\n", - " )\n", - "\n", - "\n", - " callback = ray.train.huggingface.transformers.RayTrainReportCallback()\n", - " trainer.add_callback(callback)\n", - "\n", - " trainer = ray.train.huggingface.transformers.prepare_trainer(trainer)\n", - "\n", - " trainer.train()\n", - "\n", - "\n", - " ray_trainer = TorchTrainer(\n", - " train_func,\n", - " scaling_config=ScalingConfig(\n", - " # num_workers = number of worker nodes with the ray head node included\n", - " num_workers=3,\n", - " use_gpu=True,\n", - " resources_per_worker={\n", - " \"CPU\": 1,\n", - " },\n", - " trainer_resources={\n", - " \"CPU\": 0,\n", - " }\n", - " )\n", - " # Configure persistent storage that is accessible across \n", - " # all worker nodes.\n", - " # Uncomment and update the RunConfig below to include your storage details.\n", - " # run_config=ray.train.RunConfig(storage_path=\"storage path\"),\n", - " )\n", - " result: ray.train.Result = ray_trainer.fit()" - ] - }, - { - "cell_type": "markdown", - "id": "d4d8fd65", - "metadata": {}, - "source": [ - "Once we want to test our code out, we can run the training function we defined above remotely on our Ray cluster:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5901d958", - "metadata": {}, - "outputs": [], - "source": [ - "#call the above cell as a remote ray function\n", - "ray.get(train_fn.remote())" - ] - }, - { - "cell_type": "markdown", - "id": "5af8cd32", - "metadata": {}, - "source": [ - "Once complete, we can bring our Ray cluster down and clean up:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0d41b90e", - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.13" - }, - "vscode": { - "interpreter": { - "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/demo-notebooks/guided-demos/3_widget_example.ipynb b/demo-notebooks/guided-demos/3_widget_example.ipynb index 7e900063e..8add41555 100644 --- a/demo-notebooks/guided-demos/3_widget_example.ipynb +++ b/demo-notebooks/guided-demos/3_widget_example.ipynb @@ -8,7 +8,9 @@ "In this notebook, we will go through the basics of using the SDK to:\n", " - Spin up a Ray cluster with our desired resources\n", " - View the status and specs of our Ray cluster\n", - " - Take down the Ray cluster when finished" + " - Take down the Ray cluster when finished\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." ] }, { @@ -18,7 +20,6 @@ "metadata": {}, "outputs": [], "source": [ - "# Import pieces from codeflare-sdk\n", "from codeflare_sdk import Cluster, ClusterConfiguration, view_clusters, set_api_client\n", "from kube_authkit import AuthConfig, get_k8s_client" ] diff --git a/demo-notebooks/guided-demos/4_rayjob_existing_cluster.ipynb b/demo-notebooks/guided-demos/4_rayjob_existing_cluster.ipynb index eabd6e936..1b29bd36b 100644 --- a/demo-notebooks/guided-demos/4_rayjob_existing_cluster.ipynb +++ b/demo-notebooks/guided-demos/4_rayjob_existing_cluster.ipynb @@ -37,15 +37,8 @@ "metadata": {}, "outputs": [], "source": [ - "from codeflare_sdk import Cluster, ClusterConfiguration, RayJob" - ] - }, - { - "cell_type": "markdown", - "id": "649c5911", - "metadata": {}, - "source": [ - "Run the below `oc login` command using your Token and Server URL. Ensure the command is prepended by `!` and not `%`. This will work when running both locally and within RHOAI." + "from codeflare_sdk import Cluster, ClusterConfiguration, RayJob, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" ] }, { @@ -55,7 +48,31 @@ "metadata": {}, "outputs": [], "source": [ - "!oc login --token= --server=" + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" ] }, { diff --git a/demo-notebooks/guided-demos/5_submit_rayjob_cr.ipynb b/demo-notebooks/guided-demos/5_submit_rayjob_cr.ipynb index cb370c30d..4da297180 100644 --- a/demo-notebooks/guided-demos/5_submit_rayjob_cr.ipynb +++ b/demo-notebooks/guided-demos/5_submit_rayjob_cr.ipynb @@ -29,15 +29,8 @@ "metadata": {}, "outputs": [], "source": [ - "from codeflare_sdk import RayJob, ManagedClusterConfig" - ] - }, - { - "cell_type": "markdown", - "id": "649c5911", - "metadata": {}, - "source": [ - "Run the below `oc login` command using your Token and Server URL. Ensure the command is prepended by `!` and not `%`. This will work when running both locally and within RHOAI." + "from codeflare_sdk import RayJob, ManagedClusterConfig, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" ] }, { @@ -47,7 +40,33 @@ "metadata": {}, "outputs": [], "source": [ - "!oc login --token= --server=" + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" ] }, { diff --git a/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb b/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb index db2d47d01..a87e93224 100644 --- a/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb +++ b/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb @@ -1,231 +1,232 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "8d4a42f6", - "metadata": {}, - "source": [ - "In this notebook, we will go through the basics of using the SDK to:\n", - " - Spin up a Ray cluster with our desired resources\n", - " - View the status and specs of our Ray cluster\n", - " - Take down the Ray cluster when finished" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a", - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "614daa0c", - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0f4bc870-091f-4e11-9642-cba145710159", - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name='raytest',\n", - " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", - " num_workers=2,\n", - " worker_cpu_requests=1,\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=4,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "markdown", - "id": "12eef53c", - "metadata": {}, - "source": [ - "Next, we want to bring our cluster up, so we call `cluster.apply()` below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200", - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()" - ] - }, - { - "cell_type": "markdown", - "id": "657ebdfb", - "metadata": {}, - "source": [ - "Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3c1b4311-2e61-44c9-8225-87c2db11363d", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a99d5aff", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "df71c1ed", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "markdown", - "id": "b3a55fe4", - "metadata": {}, - "source": [ - "Let's quickly verify that the specs of the cluster are as expected." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "id": "5af8cd32", - "metadata": {}, - "source": [ - "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0d41b90e", - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" - }, - "vscode": { - "interpreter": { - "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook, we will go through the basics of using the SDK to:\n", + " - Spin up a Ray cluster with our desired resources\n", + " - View the status and specs of our Ray cluster\n", + " - Take down the Ray cluster when finished\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ], + "id": "8d4a42f6" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [], + "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [], + "id": "614daa0c" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name='raytest',\n", + " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", + " num_workers=2,\n", + " worker_cpu_requests=1,\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=4,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [], + "id": "0f4bc870-091f-4e11-9642-cba145710159" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we want to bring our cluster up, so we call `cluster.apply()` below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster." + ], + "id": "12eef53c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Bring up the cluster\n", + "cluster.apply()" + ], + "execution_count": null, + "outputs": [], + "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use." + ], + "id": "657ebdfb" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "3c1b4311-2e61-44c9-8225-87c2db11363d" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [], + "id": "a99d5aff" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "df71c1ed" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's quickly verify that the specs of the cluster are as expected." + ], + "id": "b3a55fe4" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [], + "id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." + ], + "id": "5af8cd32" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [], + "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [], + "id": "0d41b90e" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + }, + "vscode": { + "interpreter": { + "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/demo-notebooks/guided-demos/notebook-ex-outputs/1_cluster_job_client.ipynb b/demo-notebooks/guided-demos/notebook-ex-outputs/1_cluster_job_client.ipynb index 776502292..2d136d1a3 100644 --- a/demo-notebooks/guided-demos/notebook-ex-outputs/1_cluster_job_client.ipynb +++ b/demo-notebooks/guided-demos/notebook-ex-outputs/1_cluster_job_client.ipynb @@ -4,7 +4,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In this demo we will go over the basics of the Ray Job Submission Client in the SDK" + "In this demo we will go over the basics of the Ray Job Submission Client in the SDK\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." ] }, { @@ -13,7 +15,6 @@ "metadata": {}, "outputs": [], "source": [ - "# Import pieces from codeflare-sdk\n", "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", "from kube_authkit import AuthConfig, get_k8s_client" ] diff --git a/demo-notebooks/guided-demos/notebook-ex-outputs/2_basic_interactive.ipynb b/demo-notebooks/guided-demos/notebook-ex-outputs/2_basic_interactive.ipynb index 5976ea17d..8b5fcdeac 100644 --- a/demo-notebooks/guided-demos/notebook-ex-outputs/2_basic_interactive.ipynb +++ b/demo-notebooks/guided-demos/notebook-ex-outputs/2_basic_interactive.ipynb @@ -5,7 +5,9 @@ "id": "bbc21043", "metadata": {}, "source": [ - "In this notebook, we will go over how to leverage the SDK to directly work interactively with a Ray cluster during development." + "In this notebook, we will go over how to leverage the SDK to directly work interactively with a Ray cluster during development.\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." ] }, { @@ -15,7 +17,6 @@ "metadata": {}, "outputs": [], "source": [ - "# Import pieces from codeflare-sdk\n", "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", "from kube_authkit import AuthConfig, get_k8s_client" ] diff --git a/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb b/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb index db2d47d01..a87e93224 100644 --- a/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb +++ b/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb @@ -1,231 +1,232 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "8d4a42f6", - "metadata": {}, - "source": [ - "In this notebook, we will go through the basics of using the SDK to:\n", - " - Spin up a Ray cluster with our desired resources\n", - " - View the status and specs of our Ray cluster\n", - " - Take down the Ray cluster when finished" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a", - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "614daa0c", - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0f4bc870-091f-4e11-9642-cba145710159", - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name='raytest',\n", - " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", - " num_workers=2,\n", - " worker_cpu_requests=1,\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=4,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "markdown", - "id": "12eef53c", - "metadata": {}, - "source": [ - "Next, we want to bring our cluster up, so we call `cluster.apply()` below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200", - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()" - ] - }, - { - "cell_type": "markdown", - "id": "657ebdfb", - "metadata": {}, - "source": [ - "Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "3c1b4311-2e61-44c9-8225-87c2db11363d", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a99d5aff", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "df71c1ed", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.status()" - ] - }, - { - "cell_type": "markdown", - "id": "b3a55fe4", - "metadata": {}, - "source": [ - "Let's quickly verify that the specs of the cluster are as expected." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "id": "5af8cd32", - "metadata": {}, - "source": [ - "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0d41b90e", - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" - }, - "vscode": { - "interpreter": { - "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook, we will go through the basics of using the SDK to:\n", + " - Spin up a Ray cluster with our desired resources\n", + " - View the status and specs of our Ray cluster\n", + " - Take down the Ray cluster when finished\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ], + "id": "8d4a42f6" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [], + "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [], + "id": "614daa0c" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name='raytest',\n", + " head_extended_resource_requests={'nvidia.com/gpu':0}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':0},\n", + " num_workers=2,\n", + " worker_cpu_requests=1,\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=4,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [], + "id": "0f4bc870-091f-4e11-9642-cba145710159" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we want to bring our cluster up, so we call `cluster.apply()` below to submit our Ray Cluster onto the queue, and begin the process of obtaining our resource cluster." + ], + "id": "12eef53c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Bring up the cluster\n", + "cluster.apply()" + ], + "execution_count": null, + "outputs": [], + "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we want to check on the status of our resource cluster, and wait until it is finally ready for use." + ], + "id": "657ebdfb" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "3c1b4311-2e61-44c9-8225-87c2db11363d" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [], + "id": "a99d5aff" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.status()" + ], + "execution_count": null, + "outputs": [], + "id": "df71c1ed" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's quickly verify that the specs of the cluster are as expected." + ], + "id": "b3a55fe4" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [], + "id": "7fd45bc5-03c0-4ae5-9ec5-dd1c30f1a084" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, we bring our resource cluster down and release/terminate the associated resources, bringing everything back to the way it was before our cluster was brought up." + ], + "id": "5af8cd32" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [], + "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [], + "id": "0d41b90e" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + }, + "vscode": { + "interpreter": { + "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb b/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb index 71ee0ee83..b6427972e 100644 --- a/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb +++ b/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb @@ -1,259 +1,260 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this demo we will go over the basics of the Ray Job Submission Client in the SDK" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this demo we will go over the basics of the Ray Job Submission Client in the SDK\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name='jobtest',\n", + " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", + " num_workers=2,\n", + " worker_cpu_requests=1,\n", + " worker_cpu_limits=1,\n", + " worker_memory_requests=4,\n", + " worker_memory_limits=4,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources\n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Bring up the cluster\n", + "cluster.apply()\n", + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Ray Job Submission" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "* Initialise the Cluster Job Client \n", + "* Provide an entrypoint command directed to your job script\n", + "* Set up your runtime environment" + ] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Initialize the Job Submission Client\n", + "\"\"\"\n", + "The SDK will automatically gather the dashboard address and authenticate using the Ray Job Submission Client\n", + "\"\"\"\n", + "client = cluster.job_client" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Submit an example mnist job using the Job Submission Client\n", + "submission_id = client.submit_job(\n", + " entrypoint=\"python mnist_fashion.py\",\n", + " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", + ")\n", + "print(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Get the job's logs\n", + "client.get_job_logs(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Get the job's status\n", + "client.get_job_status(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Get job related info\n", + "client.get_job_info(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# List all existing jobs\n", + "client.list_jobs()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Iterate through the logs of a job \n", + "async for lines in client.tail_job_logs(submission_id):\n", + " print(lines, end=\"\") " + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Delete a job\n", + "# Can run client.cancel_job(submission_id) first if job is still running\n", + "client.delete_job(submission_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + } }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Here, we want to define our cluster by specifying the resources we require for our batch workload. Below, we define our cluster object (which generates a corresponding RayCluster).\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name='jobtest',\n", - " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", - " num_workers=2,\n", - " worker_cpu_requests=1,\n", - " worker_cpu_limits=1,\n", - " worker_memory_requests=4,\n", - " worker_memory_limits=4,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources\n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()\n", - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Ray Job Submission" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "* Initialise the Cluster Job Client \n", - "* Provide an entrypoint command directed to your job script\n", - "* Set up your runtime environment" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Initialize the Job Submission Client\n", - "\"\"\"\n", - "The SDK will automatically gather the dashboard address and authenticate using the Ray Job Submission Client\n", - "\"\"\"\n", - "client = cluster.job_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Submit an example mnist job using the Job Submission Client\n", - "submission_id = client.submit_job(\n", - " entrypoint=\"python mnist_fashion.py\",\n", - " runtime_env={\"working_dir\": \"./\",\"pip\": \"requirements.txt\"},\n", - ")\n", - "print(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get the job's logs\n", - "client.get_job_logs(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get the job's status\n", - "client.get_job_status(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Get job related info\n", - "client.get_job_info(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# List all existing jobs\n", - "client.list_jobs()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Iterate through the logs of a job \n", - "async for lines in client.tail_job_logs(submission_id):\n", - " print(lines, end=\"\") " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Delete a job\n", - "# Can run client.cancel_job(submission_id) first if job is still running\n", - "client.delete_job(submission_id)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} + "nbformat": 4, + "nbformat_minor": 2 +} \ No newline at end of file diff --git a/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb b/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb index 1558bcadf..d2227f973 100644 --- a/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb +++ b/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb @@ -1,347 +1,348 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "bbc21043", - "metadata": {}, - "source": [ - "In this notebook, we will go over how to leverage the SDK to directly work interactively with a Ray Cluster during development." - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this notebook, we will go over how to leverage the SDK to directly work interactively with a Ray Cluster during development.\n", + "\n", + "First, we'll need to import the relevant CodeFlare SDK packages. You can do this by executing the below cell." + ], + "id": "bbc21043" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", + "from kube_authkit import AuthConfig, get_k8s_client" + ], + "execution_count": null, + "outputs": [], + "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", + "\n", + "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", + "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", + "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", + "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", + "# auth_config = AuthConfig(method=\"auto\")\n", + "\n", + "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", + "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", + "auth_config = AuthConfig(\n", + " method=\"openshift\",\n", + " k8s_api_host=\"https://api.example.com:6443\",\n", + " token=\"sha256~XXXXX\", # oc whoami -t\n", + ")\n", + "\n", + "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", + "# auth_config = AuthConfig(\n", + "# method=\"oidc\",\n", + "# k8s_api_host=\"https://api.example.com:6443\",\n", + "# oidc_issuer=\"https://your-oidc-provider.com\",\n", + "# client_id=\"your-client-id\",\n", + "# use_device_flow=True, # Interactive device flow for notebook environments\n", + "# )\n", + "\n", + "api_client = get_k8s_client(config=auth_config)\n", + "set_api_client(api_client)" + ], + "execution_count": null, + "outputs": [], + "id": "614daa0c" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once again, let's start by running through the same cluster setup as before:\n", + "\n", + "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", + "\n", + "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", + "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", + "\n", + "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." + ], + "id": "bc27f84c" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Create and configure our cluster object\n", + "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", + "cluster_name = \"interactivetest\"\n", + "cluster = Cluster(ClusterConfiguration(\n", + " name=cluster_name,\n", + " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", + " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", + " num_workers=2,\n", + " worker_cpu_requests=2,\n", + " worker_cpu_limits=2,\n", + " worker_memory_requests=8,\n", + " worker_memory_limits=8,\n", + " # image=\"\", # Optional Field \n", + " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", + " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", + "))" + ], + "execution_count": null, + "outputs": [], + "id": "0f4bc870-091f-4e11-9642-cba145710159" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# Bring up the cluster\n", + "cluster.apply()\n", + "cluster.wait_ready()" + ], + "execution_count": null, + "outputs": [], + "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.details()" + ], + "execution_count": null, + "outputs": [], + "id": "df71c1ed" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This time we will demonstrate another potential method of use: working with the Ray cluster interactively.\n", + "\n", + "Using the SDK, we can get both the Ray cluster URI and dashboard URI:" + ], + "id": "33663f47" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "ray_dashboard_uri = cluster.cluster_dashboard_uri()\n", + "ray_cluster_uri = cluster.cluster_uri()\n", + "print(ray_dashboard_uri)\n", + "print(ray_cluster_uri)" + ], + "execution_count": null, + "outputs": [], + "id": "c1719bca" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can connect directly to our Ray cluster via the Ray python client:" + ], + "id": "2a2aca6a" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "#before proceeding make sure the cluster exists and the uri is not empty\n", + "assert ray_cluster_uri, \"Ray cluster needs to be started and set before proceeding\"\n", + "\n", + "import ray\n", + "\n", + "# reset the ray context in case there's already one. \n", + "ray.shutdown()\n", + "# establish connection to ray cluster\n", + "\n", + "#install additional libraries that will be required for model training\n", + "runtime_env = {\"pip\": [\"transformers==4.41.2\", \"datasets==2.17.0\", \"accelerate==0.31.0\", \"scikit-learn==1.5.0\"]}\n", + "# NOTE: This will work for in-cluster notebook servers (RHODS/ODH), but not for local machines\n", + "# To see how to connect from your laptop, go to demo-notebooks/additional-demos/local_interactive.ipynb\n", + "ray.init(address=ray_cluster_uri, runtime_env=runtime_env)\n", + "\n", + "print(\"Ray cluster is up and running: \", ray.is_initialized())" + ], + "execution_count": null, + "outputs": [], + "id": "300146dc" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we are connected (and have passed in some package requirements), let's try writing some training code:" + ], + "id": "9711030b" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "@ray.remote\n", + "def train_fn():\n", + " import os\n", + " import numpy as np\n", + " from datasets import load_dataset, load_metric\n", + " import transformers\n", + " from transformers import (\n", + " Trainer,\n", + " TrainingArguments,\n", + " AutoTokenizer,\n", + " AutoModelForSequenceClassification,\n", + " )\n", + " import ray.train.huggingface.transformers\n", + " from ray.train import ScalingConfig\n", + " from ray.train.torch import TorchTrainer\n", + "\n", + " # When running in a multi-node cluster you will need persistent storage that is accessible across all worker nodes. \n", + " # See www.github.com/project-codeflare/codeflare-sdk/tree/main/docs/s3-compatible-storage.md for more information.\n", + " \n", + " def train_func():\n", + " # Datasets\n", + " dataset = load_dataset(\"imdb\")\n", + " tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")\n", + "\n", + " def tokenize_function(examples):\n", + " return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n", + "\n", + " small_train_dataset = (\n", + " dataset[\"train\"].select(range(100)).map(tokenize_function, batched=True)\n", + " )\n", + " small_eval_dataset = (\n", + " dataset[\"test\"].select(range(100)).map(tokenize_function, batched=True)\n", + " )\n", + "\n", + " # Model\n", + " model = AutoModelForSequenceClassification.from_pretrained(\n", + " \"distilbert-base-uncased\", num_labels=2\n", + " )\n", + "\n", + " def compute_metrics(eval_pred):\n", + " metric = load_metric(\"accuracy\")\n", + " logits, labels = eval_pred\n", + " predictions = np.argmax(logits, axis=-1)\n", + " return metric.compute(predictions=predictions, references=labels)\n", + "\n", + " # Hugging Face Trainer\n", + " training_args = TrainingArguments(\n", + " output_dir=\"test_trainer\",\n", + " evaluation_strategy=\"epoch\",\n", + " save_strategy=\"epoch\",\n", + " report_to=\"none\",\n", + " )\n", + "\n", + " trainer = Trainer(\n", + " model=model,\n", + " args=training_args,\n", + " train_dataset=small_train_dataset,\n", + " eval_dataset=small_eval_dataset,\n", + " compute_metrics=compute_metrics,\n", + " )\n", + "\n", + "\n", + " callback = ray.train.huggingface.transformers.RayTrainReportCallback()\n", + " trainer.add_callback(callback)\n", + "\n", + " trainer = ray.train.huggingface.transformers.prepare_trainer(trainer)\n", + "\n", + " trainer.train()\n", + "\n", + "\n", + " ray_trainer = TorchTrainer(\n", + " train_func,\n", + " scaling_config=ScalingConfig(num_workers=3, use_gpu=True),\n", + " # Configure persistent storage that is accessible across \n", + " # all worker nodes.\n", + " # Uncomment and update the RunConfig below to include your storage details.\n", + " # run_config=ray.train.RunConfig(storage_path=\"storage path\"),\n", + " )\n", + " result: ray.train.Result = ray_trainer.fit()" + ], + "execution_count": null, + "outputs": [], + "id": "1b36e0d9" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once we want to test our code out, we can run the training function we defined above remotely on our Ray cluster:" + ], + "id": "d4d8fd65" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "#call the above cell as a remote ray function\n", + "ray.get(train_fn.remote())" + ], + "execution_count": null, + "outputs": [], + "id": "5901d958" + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Once complete, we can bring our Ray cluster down and clean up:" + ], + "id": "5af8cd32" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "cluster.down()" + ], + "execution_count": null, + "outputs": [], + "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57" + }, + { + "cell_type": "code", + "metadata": {}, + "source": [ + "# No explicit logout needed - authentication is managed automatically by kube-authkit" + ], + "execution_count": null, + "outputs": [], + "id": "0d41b90e" + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + }, + "vscode": { + "interpreter": { + "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" + } + } }, - { - "cell_type": "code", - "execution_count": null, - "id": "b55bc3ea-4ce3-49bf-bb1f-e209de8ca47a", - "metadata": {}, - "outputs": [], - "source": [ - "# Import pieces from codeflare-sdk\n", - "from codeflare_sdk import Cluster, ClusterConfiguration, set_api_client\n", - "from kube_authkit import AuthConfig, get_k8s_client" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "614daa0c", - "metadata": {}, - "outputs": [], - "source": [ - "# Authenticate to your Kubernetes/OpenShift cluster using kube-authkit\n", - "\n", - "# Option 1: Auto-detect credentials (kubeconfig or in-cluster service account)\n", - "# NOTE: In RHOAI Workbenches the workbench service account may not have Ray RBAC\n", - "# permissions. Use Option 2 (token) unless your admin has granted SA permissions\n", - "# (see RHOAIENG-46748). Auto-detect works if you have a local kubeconfig.\n", - "# auth_config = AuthConfig(method=\"auto\")\n", - "\n", - "# Option 2 (Recommended for RHOAI Workbenches): Token-based authentication\n", - "# Get your token with: oc whoami -t (or from the OpenShift console → Copy login command)\n", - "auth_config = AuthConfig(\n", - " method=\"openshift\",\n", - " k8s_api_host=\"https://api.example.com:6443\",\n", - " token=\"sha256~XXXXX\", # oc whoami -t\n", - ")\n", - "\n", - "# Option 3: OIDC authentication (for BYOIDC-enabled clusters)\n", - "# auth_config = AuthConfig(\n", - "# method=\"oidc\",\n", - "# k8s_api_host=\"https://api.example.com:6443\",\n", - "# oidc_issuer=\"https://your-oidc-provider.com\",\n", - "# client_id=\"your-client-id\",\n", - "# use_device_flow=True, # Interactive device flow for notebook environments\n", - "# )\n", - "\n", - "api_client = get_k8s_client(config=auth_config)\n", - "set_api_client(api_client)" - ] - }, - { - "cell_type": "markdown", - "id": "bc27f84c", - "metadata": {}, - "source": [ - "Once again, let's start by running through the same cluster setup as before:\n", - "\n", - "NOTE: The default images used by the CodeFlare SDK for creating a RayCluster resource depend on the installed Python version:\n", - "\n", - "- For Python 3.11: 'quay.io/modh/ray:2.52.1-py311-cu121'\n", - "- For Python 3.12: 'quay.io/modh/ray:2.53.0-py312-cu128'\n", - "\n", - "If you prefer to use a custom Ray image that better suits your needs, you can specify it in the image field to override the default." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0f4bc870-091f-4e11-9642-cba145710159", - "metadata": {}, - "outputs": [], - "source": [ - "# Create and configure our cluster object\n", - "# The SDK will try to find the name of your default local queue based on the annotation \"kueue.x-k8s.io/default-queue\": \"true\" unless you specify the local queue manually below\n", - "cluster_name = \"interactivetest\"\n", - "cluster = Cluster(ClusterConfiguration(\n", - " name=cluster_name,\n", - " head_extended_resource_requests={'nvidia.com/gpu':1}, # For GPU enabled workloads set the head_extended_resource_requests and worker_extended_resource_requests\n", - " worker_extended_resource_requests={'nvidia.com/gpu':1},\n", - " num_workers=2,\n", - " worker_cpu_requests=2,\n", - " worker_cpu_limits=2,\n", - " worker_memory_requests=8,\n", - " worker_memory_limits=8,\n", - " # image=\"\", # Optional Field \n", - " write_to_file=False, # When enabled Ray Cluster yaml files are written to /HOME/.codeflare/resources \n", - " # local_queue=\"local-queue-name\" # Specify the local queue manually\n", - "))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f0884bbc-c224-4ca0-98a0-02dfa09c2200", - "metadata": {}, - "outputs": [], - "source": [ - "# Bring up the cluster\n", - "cluster.apply()\n", - "cluster.wait_ready()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "df71c1ed", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.details()" - ] - }, - { - "cell_type": "markdown", - "id": "33663f47", - "metadata": {}, - "source": [ - "This time we will demonstrate another potential method of use: working with the Ray cluster interactively.\n", - "\n", - "Using the SDK, we can get both the Ray cluster URI and dashboard URI:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c1719bca", - "metadata": {}, - "outputs": [], - "source": [ - "ray_dashboard_uri = cluster.cluster_dashboard_uri()\n", - "ray_cluster_uri = cluster.cluster_uri()\n", - "print(ray_dashboard_uri)\n", - "print(ray_cluster_uri)" - ] - }, - { - "cell_type": "markdown", - "id": "2a2aca6a", - "metadata": {}, - "source": [ - "Now we can connect directly to our Ray cluster via the Ray python client:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "300146dc", - "metadata": {}, - "outputs": [], - "source": [ - "#before proceeding make sure the cluster exists and the uri is not empty\n", - "assert ray_cluster_uri, \"Ray cluster needs to be started and set before proceeding\"\n", - "\n", - "import ray\n", - "\n", - "# reset the ray context in case there's already one. \n", - "ray.shutdown()\n", - "# establish connection to ray cluster\n", - "\n", - "#install additional libraries that will be required for model training\n", - "runtime_env = {\"pip\": [\"transformers==4.41.2\", \"datasets==2.17.0\", \"accelerate==0.31.0\", \"scikit-learn==1.5.0\"]}\n", - "# NOTE: This will work for in-cluster notebook servers (RHODS/ODH), but not for local machines\n", - "# To see how to connect from your laptop, go to demo-notebooks/additional-demos/local_interactive.ipynb\n", - "ray.init(address=ray_cluster_uri, runtime_env=runtime_env)\n", - "\n", - "print(\"Ray cluster is up and running: \", ray.is_initialized())" - ] - }, - { - "cell_type": "markdown", - "id": "9711030b", - "metadata": {}, - "source": [ - "Now that we are connected (and have passed in some package requirements), let's try writing some training code:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "1b36e0d9", - "metadata": {}, - "outputs": [], - "source": [ - "@ray.remote\n", - "def train_fn():\n", - " import os\n", - " import numpy as np\n", - " from datasets import load_dataset, load_metric\n", - " import transformers\n", - " from transformers import (\n", - " Trainer,\n", - " TrainingArguments,\n", - " AutoTokenizer,\n", - " AutoModelForSequenceClassification,\n", - " )\n", - " import ray.train.huggingface.transformers\n", - " from ray.train import ScalingConfig\n", - " from ray.train.torch import TorchTrainer\n", - "\n", - " # When running in a multi-node cluster you will need persistent storage that is accessible across all worker nodes. \n", - " # See www.github.com/project-codeflare/codeflare-sdk/tree/main/docs/s3-compatible-storage.md for more information.\n", - " \n", - " def train_func():\n", - " # Datasets\n", - " dataset = load_dataset(\"imdb\")\n", - " tokenizer = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")\n", - "\n", - " def tokenize_function(examples):\n", - " return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n", - "\n", - " small_train_dataset = (\n", - " dataset[\"train\"].select(range(100)).map(tokenize_function, batched=True)\n", - " )\n", - " small_eval_dataset = (\n", - " dataset[\"test\"].select(range(100)).map(tokenize_function, batched=True)\n", - " )\n", - "\n", - " # Model\n", - " model = AutoModelForSequenceClassification.from_pretrained(\n", - " \"distilbert-base-uncased\", num_labels=2\n", - " )\n", - "\n", - " def compute_metrics(eval_pred):\n", - " metric = load_metric(\"accuracy\")\n", - " logits, labels = eval_pred\n", - " predictions = np.argmax(logits, axis=-1)\n", - " return metric.compute(predictions=predictions, references=labels)\n", - "\n", - " # Hugging Face Trainer\n", - " training_args = TrainingArguments(\n", - " output_dir=\"test_trainer\",\n", - " evaluation_strategy=\"epoch\",\n", - " save_strategy=\"epoch\",\n", - " report_to=\"none\",\n", - " )\n", - "\n", - " trainer = Trainer(\n", - " model=model,\n", - " args=training_args,\n", - " train_dataset=small_train_dataset,\n", - " eval_dataset=small_eval_dataset,\n", - " compute_metrics=compute_metrics,\n", - " )\n", - "\n", - "\n", - " callback = ray.train.huggingface.transformers.RayTrainReportCallback()\n", - " trainer.add_callback(callback)\n", - "\n", - " trainer = ray.train.huggingface.transformers.prepare_trainer(trainer)\n", - "\n", - " trainer.train()\n", - "\n", - "\n", - " ray_trainer = TorchTrainer(\n", - " train_func,\n", - " scaling_config=ScalingConfig(num_workers=3, use_gpu=True),\n", - " # Configure persistent storage that is accessible across \n", - " # all worker nodes.\n", - " # Uncomment and update the RunConfig below to include your storage details.\n", - " # run_config=ray.train.RunConfig(storage_path=\"storage path\"),\n", - " )\n", - " result: ray.train.Result = ray_trainer.fit()" - ] - }, - { - "cell_type": "markdown", - "id": "d4d8fd65", - "metadata": {}, - "source": [ - "Once we want to test our code out, we can run the training function we defined above remotely on our Ray cluster:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5901d958", - "metadata": {}, - "outputs": [], - "source": [ - "#call the above cell as a remote ray function\n", - "ray.get(train_fn.remote())" - ] - }, - { - "cell_type": "markdown", - "id": "5af8cd32", - "metadata": {}, - "source": [ - "Once complete, we can bring our Ray cluster down and clean up:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5f36db0f-31f6-4373-9503-dc3c1c4c3f57", - "metadata": {}, - "outputs": [], - "source": [ - "cluster.down()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "0d41b90e", - "metadata": {}, - "outputs": [], - "source": [ - "# No explicit logout needed - authentication is managed automatically by kube-authkit" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" - }, - "vscode": { - "interpreter": { - "hash": "f9f85f796d01129d0dd105a088854619f454435301f6ffec2fea96ecbd9be4ac" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file From 55a1e053e825b11669c4029184a07398d55e6a0e Mon Sep 17 00:00:00 2001 From: Laura Fitzgerald Date: Tue, 21 Apr 2026 13:28:11 +0100 Subject: [PATCH 2/2] fix(notebooks): add missing end-of-file newlines Fixes end-of-file-fixer pre-commit hook failures on 9 notebooks that were missing a trailing newline after the closing brace. --- demo-notebooks/additional-demos/hf_interactive.ipynb | 2 +- demo-notebooks/additional-demos/local_interactive.ipynb | 2 +- demo-notebooks/guided-demos/0_basic_ray.ipynb | 2 +- demo-notebooks/guided-demos/1_cluster_job_client.ipynb | 2 +- demo-notebooks/guided-demos/2_basic_interactive.ipynb | 2 +- .../guided-demos/notebook-ex-outputs/0_basic_ray.ipynb | 2 +- demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb | 2 +- .../guided-demos/preview_nbs/1_cluster_job_client.ipynb | 2 +- .../guided-demos/preview_nbs/2_basic_interactive.ipynb | 2 +- 9 files changed, 9 insertions(+), 9 deletions(-) diff --git a/demo-notebooks/additional-demos/hf_interactive.ipynb b/demo-notebooks/additional-demos/hf_interactive.ipynb index 0e1d15d06..407d08c78 100644 --- a/demo-notebooks/additional-demos/hf_interactive.ipynb +++ b/demo-notebooks/additional-demos/hf_interactive.ipynb @@ -452,4 +452,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} diff --git a/demo-notebooks/additional-demos/local_interactive.ipynb b/demo-notebooks/additional-demos/local_interactive.ipynb index b90dc7175..4383c0770 100644 --- a/demo-notebooks/additional-demos/local_interactive.ipynb +++ b/demo-notebooks/additional-demos/local_interactive.ipynb @@ -252,4 +252,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} diff --git a/demo-notebooks/guided-demos/0_basic_ray.ipynb b/demo-notebooks/guided-demos/0_basic_ray.ipynb index 823506be5..1e38a32ba 100644 --- a/demo-notebooks/guided-demos/0_basic_ray.ipynb +++ b/demo-notebooks/guided-demos/0_basic_ray.ipynb @@ -228,4 +228,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} diff --git a/demo-notebooks/guided-demos/1_cluster_job_client.ipynb b/demo-notebooks/guided-demos/1_cluster_job_client.ipynb index 812c87eca..80c6cca1c 100644 --- a/demo-notebooks/guided-demos/1_cluster_job_client.ipynb +++ b/demo-notebooks/guided-demos/1_cluster_job_client.ipynb @@ -268,4 +268,4 @@ }, "nbformat": 4, "nbformat_minor": 2 -} \ No newline at end of file +} diff --git a/demo-notebooks/guided-demos/2_basic_interactive.ipynb b/demo-notebooks/guided-demos/2_basic_interactive.ipynb index 176f0ad12..375d9e256 100644 --- a/demo-notebooks/guided-demos/2_basic_interactive.ipynb +++ b/demo-notebooks/guided-demos/2_basic_interactive.ipynb @@ -367,4 +367,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} diff --git a/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb b/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb index a87e93224..15c7a578f 100644 --- a/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb +++ b/demo-notebooks/guided-demos/notebook-ex-outputs/0_basic_ray.ipynb @@ -229,4 +229,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} diff --git a/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb b/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb index a87e93224..15c7a578f 100644 --- a/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb +++ b/demo-notebooks/guided-demos/preview_nbs/0_basic_ray.ipynb @@ -229,4 +229,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +} diff --git a/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb b/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb index b6427972e..3c979c501 100644 --- a/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb +++ b/demo-notebooks/guided-demos/preview_nbs/1_cluster_job_client.ipynb @@ -257,4 +257,4 @@ }, "nbformat": 4, "nbformat_minor": 2 -} \ No newline at end of file +} diff --git a/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb b/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb index d2227f973..0a48e247e 100644 --- a/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb +++ b/demo-notebooks/guided-demos/preview_nbs/2_basic_interactive.ipynb @@ -345,4 +345,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} \ No newline at end of file +}