Merge pull request #3112 from pareenaverma/content_review

Tech review of flyte LP

Author: pareenaverma

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/_index.md

@@ -1,5 +1,6 @@
 ---
 title: Build ML Workflow Pipelines with Flyte and gRPC on Google Cloud C4A Axion processors
+description: Learn how to build scalable machine learning workflow pipelines on Google Cloud C4A Axion processors using Flyte for workflow orchestration and gRPC for distributed service communication.
 
 draft: true
 cascade:
@@ -20,7 +21,7 @@ prerequisites:
   - A [Google Cloud Platform (GCP)](https://cloud.google.com/free) account with billing enabled
   - Basic familiarity with Python
   - Basic understanding of machine learning pipelines
-  - Familiarity with Linux command-line operations
+
 
 author: Pareena Verma
 

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/architecture.md

@@ -6,8 +6,6 @@ weight: 8
 layout: learningpathall
 ---
 
-# ML Pipeline Architecture
-
 In this section, you explore the architecture behind the distributed machine learning pipeline built using Flyte and gRPC on Google Axion Arm-based infrastructure.
 
 This architecture demonstrates how modern ML workflows are orchestrated using workflow engines while delegating specific tasks to distributed services.

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/background.md

@@ -32,4 +32,4 @@ In this section, you learned about:
 * gRPC as a communication layer for distributed services
 * How Flyte and gRPC can be used together to build scalable ML training pipelines
 
-Next, You will deploy Flyte tools, create a gRPC-based feature engineering service, and build a distributed ML workflow pipeline that orchestrates data processing and model training tasks on Axion infrastructure.
+Next, you will deploy Flyte tools, create a gRPC-based feature engineering service, and build a distributed ML workflow pipeline that orchestrates data processing and model training tasks on Axion infrastructure.

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/create-grpc-service.md

@@ -6,9 +6,6 @@ weight: 5
 layout: learningpathall
 ---
 
-# Create Feature Engineering gRPC Service
-
-
 In modern machine learning pipelines, feature engineering is often implemented as a separate service so it can scale independently from the training workflow.
 
 In this section, you create a **gRPC-based feature engineering service** that generates features used by the machine learning pipeline.
@@ -46,7 +43,7 @@ vi feature.proto
 
 Add the following code.
 
-```python
+```protobuf
 syntax = "proto3";
 
 service FeatureService {
@@ -65,11 +62,17 @@ message FeatureResponse {
 This file defines the service interface used by the workflow and the feature service.
 
 ## Generate gRPC code
-Compile the protobuf file to generate Python client and server code.
 
+Make sure the `flyte-env` virtual environment is active before running the following commands. If you opened a new terminal, reactivate it:
 
-```python
-python3.11 -m grpc_tools.protoc \
+```bash
+source ~/flyte-env/bin/activate
+```
+
+Compile the protobuf file to generate Python client and server code.
+
+```bash
+python -m grpc_tools.protoc \
 -I. \
 --python_out=. \
 --grpc_python_out=. \
@@ -147,7 +150,7 @@ This service receives a value from the workflow and generates a derived feature
 Start the gRPC service.
 
 ```bash
-python3.11 feature_server.py
+python feature_server.py
 ```
 
 The output is similar to:
@@ -157,10 +160,10 @@ Feature gRPC service running on port 50051
 
 ## What you've learned and what's next
 
-- Created a project directory for the ML workflow
-- Defined a gRPC service using protobuf
-- Generated Python client and server code
-- Implemented a feature engineering microservice
-- Started the gRPC feature service
+- Create a project directory for the ML workflow
+- Define a gRPC service using protobuf
+- Generate Python client and server code
+- Implement a feature engineering microservice
+- Start the gRPC feature service
 
 In the next section, you will create a Flyte ML training workflow that calls this feature engineering service during pipeline execution.

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/create-ml-workflow.md

@@ -53,6 +53,7 @@ This architecture separates workflow orchestration from feature generation, allo
 Create the workflow file.
 
 ```bash
+cd $HOME/flyte-ml-pipeline
 vi workflow.py
 ```
 
@@ -179,4 +180,4 @@ In this section, you learned how to:
 - Integrate Flyte tasks with a gRPC microservice
 - Orchestrate ML pipeline execution
 
-In the next section, you will run the co**mplete ML training pipeline and observe how Flyte interacts with the feature engineering service during workflow execution**.
+In the next section, you will run the complete ML training pipeline and observe how Flyte interacts with the feature engineering service during workflow execution.

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/install-flyte.md

@@ -59,7 +59,7 @@ python3.11 --version
 The output is similar to:
 
 ```output
-Python 3.11.x
+Python 3.11.10
 ```
 
 **Why this matters:**
@@ -68,36 +68,57 @@ Python 3.11.x
 - Modern workflow libraries are optimized for Python 3.11
 - Ensures compatibility with Flyte and gRPC libraries
 
-## Install Flyte SDK
+## Create a virtual environment
 
-Install the Flyte Python SDK used to define and execute workflows.
+`flytekit 1.16.15` requires `setuptools<70`, while `grpcio-tools` requires `setuptools>=77`. These constraints are mutually exclusive at the system level. A virtual environment resolves this by isolating the dependency graph from any pre-installed system packages, allowing pip to resolve all constraints together from scratch.
+
+Create and activate a virtual environment:
 
 ```bash
-python3.11 -m pip install --upgrade pip
-python3.11 -m pip install flytekit
+python3.11 -m venv flyte-env
+source flyte-env/bin/activate
 ```
 
-**Verify installation:**
+Your prompt will change to show `(flyte-env)`, confirming the environment is active. All subsequent `pip install` commands in this Learning Path use this environment.
+
+## Install Flyte SDK and gRPC libraries
+
+With the virtual environment active, upgrade pip and install Flyte together with the gRPC communication libraries in a single step. This lets pip resolve a consistent set of versions, including `setuptools`, across all packages at once.
 
 ```bash
-python3.11 -c "import flytekit; print(flytekit.__version__)"
+pip install --upgrade pip
+pip install flytekit grpcio grpcio-tools protobuf
 ```
 
-This confirms that Flyte is correctly installed.
+**Verify Flyte installation:**
 
-## Install gRPC libraries
+```bash
+python -c "import flytekit; print(flytekit.__version__)"
+```
 
-Install the libraries required for communication between distributed services.
+The output is similar to:
+
+```output
+1.16.15
+```
+
+**Verify gRPC installation:**
 
 ```bash
-python3.11 -m pip install grpcio grpcio-tools protobuf
+python -c "import grpc; import grpc_tools; import google.protobuf; print('gRPC libraries OK')"
+```
+
+The output is similar to:
+
+```output
+gRPC libraries OK
 ```
 
-These libraries enable remote procedure calls between workflow tasks and microservices.
+This confirms that Flyte and the gRPC libraries are correctly installed and compatible within the virtual environment.
 
 ## Install Flyte CLI
 
-Download and install the Flyte command-line tool.
+The Flyte CLI is a standalone binary and doesn't use pip, so you install it to the system path independently of the virtual environment.
 
 ```bash
 curl -L https://github.com/flyteorg/flytectl/releases/latest/download/flytectl_Linux_arm64.tar.gz -o flytectl.tar.gz
@@ -126,10 +147,10 @@ This confirms that the Flyte CLI is correctly installed.
 
 In this section, you learned how to:
 
-- Prepared a SUSE arm64 environment for ML workflow development
-- Installed Python 3.11 and development dependencies
-- Installed Flyte SDK for workflow orchestration
-- Installed gRPC libraries for service communication
-- Configured Flyte CLI tools
+- Prepare a SUSE arm64 environment for ML workflow development
+- Install Python 3.11 and development dependencies
+- Create a virtual environment to resolve the `setuptools` conflict between `flytekit` and `grpcio-tools`
+- Install Flyte SDK and gRPC libraries in a single isolated environment
+- Install Flyte CLI tools
 
 In the next section, you will create a gRPC-based feature engineering service that will be integrated with the Flyte ML workflow pipeline.

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/instance.md

@@ -25,7 +25,7 @@ To create a virtual machine based on the C4A instance type:
   * Set **Series** to `C4A`.
   * Select `c4a-standard-4` for machine type.
 
-![Screenshot of the Google Cloud Console showing the Machine configuration section. The Series dropdown is set to C4A and the machine type c4a-standard-4 is selected alt-txt#center](images/gcp-vm.png "Configuring machine type to C4A in Google Cloud Console")
+![Screenshot of the Google Cloud Console showing the Machine configuration section. The Series dropdown is set to C4A and the machine type c4a-standard-4 is selected#center](images/gcp-vm.png "Configuring machine type to C4A in Google Cloud Console")
 
 
 * Under **OS and storage**, select **Change**, and then choose an arm64 operating system image.
@@ -34,17 +34,16 @@ To create a virtual machine based on the C4A instance type:
   * Increase **Size (GB)** from **10** to **100** to allocate sufficient disk space.
   * Select **Choose** to apply the changes.
 * Under **Networking**, enable **Allow HTTP traffic** and **Allow HTTPS traffic**
-* Also, add the following tag: `allow-flyte-with-grpc` so this virtual machine matches the firewall rule you created in the previous section.
 * For some organizations not using the **'default'** network interface, you may need to select the network appropriate for your organization.
 * Select **Create** to launch the virtual machine.
 
 After the instance starts, select **SSH** next to the VM in the instance list to open a browser-based terminal session.
 
-![Google Cloud Console VM instances page displaying running instance with green checkmark and SSH button in the Connect column alt-txt#center](images/gcp-pubip-ssh.png "Connecting to a running C4A VM using SSH")
+![Google Cloud Console VM instances page displaying running instance with green checkmark and SSH button in the Connect column#center](images/gcp-pubip-ssh.png "Connecting to a running C4A VM using SSH")
 
 A new browser window opens with a terminal connected to your VM.
 
-![Browser-based SSH terminal window with black background showing Linux command prompt and Google Cloud branding at top alt-txt#center](images/gcp-shell.png "Terminal session connected to the VM")
+![Browser-based SSH terminal window with black background showing Linux command prompt and Google Cloud branding at top#center](images/gcp-shell.png "Terminal session connected to the VM")
 
 ## What you've learned and what's next
 

content/learning-paths/servers-and-cloud-computing/flyte-with-grpc/run-ml-pipeline.md

@@ -6,8 +6,6 @@ weight: 7
 layout: learningpathall
 ---
 
-# Run ML Training Pipeline
-
 In this section, you execute the distributed machine learning pipeline built using **Flyte and gRPC**.
 
 The ML workflow will:
@@ -22,10 +20,16 @@ The feature engineering service runs independently and communicates with the wor
 
 ## Start the feature engineering service
 
-First, start the feature engineering service that was created in the previous section.
+Make sure the `flyte-env` virtual environment is active. If you opened a new terminal, reactivate it:
+
+```bash
+source ~/flyte-env/bin/activate
+```
+
+Start the feature engineering service that was created in the previous section.
 
 ```bash
-python3.11 feature_server.py
+python feature_server.py
 ```
 
 The output is similar to:
@@ -46,7 +50,7 @@ cd ~/flyte-ml-pipeline
 **Run the workflow:**
 
 ```bash
-python3.11 workflow.py
+python workflow.py
 ```
 
 ## Example pipeline execution output