diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/_index.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/_index.md new file mode 100644 index 0000000000..22d9404d06 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/_index.md @@ -0,0 +1,80 @@ +--- +title: Build a mixed-placement AI shopping assistant on GKE with Axion + +minutes_to_complete: 120 +description: Deploy an AI shopping assistant for an Online Boutique storefront on GKE and compare N4A and C4A placement for the assistant tier. +who_is_this_for: This is an advanced topic for cloud developers, platform engineers, and site reliability engineers who run applications on Google Kubernetes Engine and want to place application tiers on the Axion-based machine series that fits each workload. + +learning_objectives: + - Explain why a storefront application and an AI assistant can have different workload profiles + - Build and push a `linux/arm64` container image for the shopping assistant service + - Use Kustomize overlays to run the assistant on N4A first, then move it to C4A + - Capture and compare benchmark summaries for the same assistant workload on N4A and C4A + +prerequisites: + - A [Google Cloud account](https://console.cloud.google.com/) with billing enabled + - Access to a [GKE Standard cluster with Arm node pools](https://cloud.google.com/kubernetes-engine/docs/how-to/create-arm-clusters-nodes), including an N4A node pool and a C4A node pool + - Permission to create or use an [Artifact Registry Docker repository](https://cloud.google.com/artifact-registry/docs/docker/store-docker-container-images) in the same project + - Access to Google Cloud Shell, or a local Linux or macOS terminal with `gcloud`, `kubectl`, `docker`, `git`, `jq`, and `python3` + - Basic familiarity with Docker, Kubernetes, Kustomize, and Google Kubernetes Engine + +author: + - Rani Chowdary Mandepudi + +generate_summary_faq: true +rerun_summary: false +rerun_faqs: false + +### Tags +skilllevels: Advanced +subjects: Containers and Virtualization +cloud_service_providers: + - Google Cloud +armips: + - Neoverse +operatingsystems: + - Linux +tools_software_languages: + - Kubernetes + - GKE + - Docker + - Kustomize + - Python + - Ollama + +further_reading: + - resource: + title: Google Kubernetes Engine documentation + link: https://cloud.google.com/kubernetes-engine/docs + type: documentation + - resource: + title: Create standard clusters and node pools with Arm nodes + link: https://cloud.google.com/kubernetes-engine/docs/how-to/create-arm-clusters-nodes + type: documentation + - resource: + title: Google Axion processors + link: https://cloud.google.com/products/axion + type: documentation + - resource: + title: Kustomize documentation + link: https://kubectl.docs.kubernetes.io/references/kustomize/ + type: documentation + - resource: + title: Ollama documentation + link: https://ollama.com/ + type: documentation + - resource: + title: Migrate x86 workloads to Arm on Google Kubernetes Engine with Axion processors + link: /learning-paths/servers-and-cloud-computing/gke-multi-arch-axion/ + type: website + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- + +Deploy an Online Boutique storefront on Google Kubernetes Engine (GKE), add an AI shopping assistant, and compare the same assistant workload on N4A and C4A node pools powered by Google Axion processors. + +You start with the storefront on N4A, build the missing `shoppingassistantservice` image for `linux/arm64`, validate the assistant on N4A, and then move only the assistant tier to C4A. You finish by running the same fixed benchmark in both placements so you can compare request success, latency, and Kubernetes CPU telemetry. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/_next-steps.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/background.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/background.md new file mode 100644 index 0000000000..8cbcd4aacb --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/background.md @@ -0,0 +1,78 @@ +--- +title: Understand mixed placement for a storefront AI assistant +description: Learn why the storefront and assistant tiers use different Axion placements before you build and deploy the assistant. +weight: 2 +layout: "learningpathall" +--- + +## How the application is organized + +Modern AI applications often contain more than one workload shape. A storefront tier is steady, service-oriented, and always on. An AI assistant tier is burstier and more latency-sensitive because it performs reasoning work only when users ask for help. + +This Learning Path uses that split in a live Online Boutique storefront on Google Kubernetes Engine (GKE). The storefront starts on N4A nodes powered by Google Axion processors. You add the missing `shoppingassistantservice`, run it on N4A first, and then move only that assistant tier to C4A. + +The goal isn't to prove that one Axion-based machine series replaces the other. You use N4A for the steady storefront tier and evaluate C4A for the AI reasoning tier so you can decide which machine series fits each workload. + +![Architecture diagram showing Cloud Shell and Kustomize deploying Online Boutique to GKE on Axion. Storefront microservices run on N4A, and the shopping assistant runs on C4A with access to cart, catalog, and a local Ollama Gemma model.#center](images/mixed-placement-agentic-storefront-on-axion.png "Mixed-placement agentic storefront on Axion") + +The diagram shows the final mixed-placement pattern you build toward. The storefront images already run on Arm nodes, and the assistant image you build later is `linux/arm64` because both target node pools are Arm-based. + +## How the storefront already runs on Arm + +The storefront baseline uses container images that can run on Arm nodes. A common way to publish portable container images is to use a multi-architecture image, which is one image reference that contains variants for more than one CPU architecture, such as `linux/amd64` and `linux/arm64`. + +When Kubernetes schedules a pod that uses a multi-architecture image on an Arm node, the container runtime pulls the Arm-compatible variant from the same image reference. That is why the storefront can already run on Axion nodes before you build the assistant image. + +This Learning Path starts with Arm-compatible storefront images already available. To learn the full build-and-publish workflow for multi-architecture images on GKE, see [Migrate x86 workloads to Arm on Google Kubernetes Engine with Axion processors](/learning-paths/servers-and-cloud-computing/gke-multi-arch-axion/). + +Inspect a public multi-architecture image manifest: + +```bash +docker buildx imagetools inspect docker.io/library/python:3.12-slim +``` + +The output is similar to: + +```output +Name: docker.io/library/python:3.12-slim +MediaType: application/vnd.oci.image.index.v1+json + +Manifests: + Platform: linux/amd64 + Platform: linux/arm64/v8 +``` + +The exact digest values can differ. The important signal is that one image reference can include an Arm-compatible variant. You confirm the storefront image reference from the source tree in the setup step. + +## How the assistant works + +The `shoppingassistantservice` is the AI layer for the application. It runs as its own Kubernetes service and receives assistant requests from the storefront frontend. + +The assistant uses live storefront services as tools: + +- `search_catalog` queries `productcatalogservice`. +- `get_product_details` fetches a specific product from the live catalog. +- `get_cart` reads the current cart from `cartservice`. +- `add_to_cart` updates the cart only after user confirmation. + +The assistant sends grounded context to a local Ollama sidecar in the same pod. The sidecar runs the `gemma3:1b-it-qat` model for the reasoning step. + +This design keeps the comparison focused. You don't add a separate vector database, retrieval pipeline, or hosted large language model API. You move the assistant logic and local reasoning runtime together as one AI tier. + +## How requests flow through the storefront + +When a shopper uses the assistant, the request follows this path: + +1. The browser sends the request to `frontend`. +2. `frontend` forwards the request to `shoppingassistantservice`. +3. The assistant calls live services such as `productcatalogservice` and `cartservice`. +4. The assistant sends grounded context to the local Ollama sidecar. +5. The assistant returns the response through `frontend`. + +The assistant is agentic because it does more than generate text. It uses application tools, keeps short-lived session state, and asks for confirmation before it calls `cartservice` to update the cart. + +## What you've learned and what's next + +You've learned why the storefront tier and AI assistant tier use different Axion placements. You also saw how Arm-compatible image variants make the existing storefront portable to Axion nodes. + +Next, you'll set up environment variables, connect to the GKE cluster, and clone the source tree that contains the assistant implementation. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/build-assistant-image.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/build-assistant-image.md new file mode 100644 index 0000000000..a86b093e03 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/build-assistant-image.md @@ -0,0 +1,99 @@ +--- +title: Build and push the assistant image +description: Build the shopping assistant container image for arm64 and push it to Artifact Registry for deployment on Axion. +weight: 6 +layout: "learningpathall" +--- + +## Build one Arm image for both Axion placements + +Both the N4A and C4A node pools in this Learning Path use Arm-based Google Axion processors. You only need one `linux/arm64` assistant image, and that image can run in either placement. + +{{% notice Note %}} +This is an Arm-targeted build for Axion, not a full multi-architecture build. A multi-architecture image is useful when the same tag must support non-Arm environments such as `linux/amd64`. +{{% /notice %}} + +If you opened a new terminal, return to the source tree and restore the required variables: + +```bash +cd "${HOME}/n4a-c4a/microservices-demo" + +export PROJECT_ID="$(gcloud config get-value project)" +export ARTIFACT_REGION="us-central1" +export ARTIFACT_REPO="axion-workshop" +``` + +## Define the image name + +Create one reusable image path: + +```bash +export ASSISTANT_IMAGE_REPO="${ARTIFACT_REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REPO}/shoppingassistantservice" +export ASSISTANT_IMAGE_TAG="lab-v1" +export ASSISTANT_IMAGE="${ASSISTANT_IMAGE_REPO}:${ASSISTANT_IMAGE_TAG}" + +echo "${ASSISTANT_IMAGE}" +``` + +## Verify Docker and Buildx + +Confirm that Docker and Buildx are available: + +```bash +docker version +docker buildx version +``` + +If Docker is not running in your environment, start it before you continue. + +## Configure Artifact Registry authentication + +Configure Docker authentication for Artifact Registry: + +```bash +gcloud auth configure-docker "${ARTIFACT_REGION}-docker.pkg.dev" --quiet +``` + +Cloud Shell might print a warning about existing credential helpers. You can continue if the command finishes successfully. + +## Create a Buildx builder + +Create or reuse a Buildx builder: + +```bash +if docker buildx inspect axion-builder >/dev/null 2>&1; then + docker buildx use axion-builder +else + docker buildx create --name axion-builder --use +fi + +docker buildx inspect --bootstrap +``` + +The output should show a working builder. Confirm that the `Platforms` line includes `linux/arm64` before you continue. + +## Build and push the assistant image + +Build the assistant image for `linux/arm64` and push it to Artifact Registry: + +```bash +docker buildx build --platform linux/arm64 -t "${ASSISTANT_IMAGE}" --push src/shoppingassistantservice +``` + +The command packages the Flask assistant service, protobuf stubs, and Python dependencies. The important success signal is that the build finishes and pushes the image to your Artifact Registry path. + +## Verify the pushed tag + +List the tags for the assistant image: + +```bash +gcloud artifacts docker tags list "${ASSISTANT_IMAGE_REPO}" +``` + +The output should include the `lab-v1` tag. + +## What you've accomplished + +You've built and published an Arm image for the shopping assistant. Kubernetes can now pull this image when you deploy the assistant on N4A. + +Next, you'll create the N4A overlay and add the assistant to the running storefront. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/deploy-assistant-n4a.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/deploy-assistant-n4a.md new file mode 100644 index 0000000000..3b2577e936 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/deploy-assistant-n4a.md @@ -0,0 +1,164 @@ +--- +title: Deploy the assistant on N4A +description: Add the assistant component to the storefront and keep the application on the N4A node pool. +weight: 7 +layout: "learningpathall" +--- + +## Create the N4A overlay + +If you opened a new terminal, return to the source tree and restore the required variables: + +```bash +cd "${HOME}/n4a-c4a/microservices-demo" + +export PROJECT_ID="$(gcloud config get-value project)" +export ARTIFACT_REGION="us-central1" +export ARTIFACT_REPO="axion-workshop" +export ASSISTANT_IMAGE_REPO="${ARTIFACT_REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REPO}/shoppingassistantservice" +export ASSISTANT_IMAGE_TAG="lab-v1" +export N4A_NODE_POOL_NAME="arm64-pool-n4a2" +export FRONTEND_IP="$(kubectl get service frontend-external \ + -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" +export APP_URL="http://${FRONTEND_IP}" +``` + +Create a Kustomize overlay that adds the assistant component and keeps the full storefront, including the assistant, on the N4A node pool: + +```bash +mkdir -p kustomize/overlays/assistant-n4a + +cat < kustomize/overlays/assistant-n4a/kustomization.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: +- ../../base +components: +- ../../components/shopping-assistant +images: +- name: shoppingassistantservice + newName: ${ASSISTANT_IMAGE_REPO} + newTag: ${ASSISTANT_IMAGE_TAG} +patches: +- path: node-selector.yaml + target: + kind: Deployment +EOF + +cat < kustomize/overlays/assistant-n4a/node-selector.yaml +- op: add + path: /spec/template/spec/nodeSelector + value: + cloud.google.com/gke-nodepool: ${N4A_NODE_POOL_NAME} +- op: add + path: /spec/template/spec/tolerations + value: + - key: kubernetes.io/arch + operator: Equal + value: arm64 + effect: NoSchedule +EOF +``` + +## Render the overlay + +Review the overlay files and render the manifest: + +```bash +sed -n '1,120p' kustomize/overlays/assistant-n4a/kustomization.yaml +sed -n '1,120p' kustomize/overlays/assistant-n4a/node-selector.yaml +kubectl kustomize kustomize/overlays/assistant-n4a | sed -n '1,240p' +``` + +Do not apply the overlay if the rendered manifest has an empty image name, image tag, or node-pool value. + +## Apply the overlay + +Apply the overlay and wait for the frontend and assistant deployments: + +```bash +kubectl apply -k kustomize/overlays/assistant-n4a +kubectl rollout status deployment/frontend --timeout=600s +kubectl rollout status deployment/shoppingassistantservice --timeout=1200s +``` + +{{% notice Note %}} +`kubectl apply -k` reapplies the full rendered application state. It is normal to see existing deployments reported as `configured`. The `shopping-assistant` component patches `frontend`, so a frontend rollout is expected. +{{% /notice %}} + +## Confirm pod placement + +Check where the frontend and assistant pods landed: + +```bash +kubectl get pods -o wide | grep -E 'frontend|shoppingassistantservice' + +ASSISTANT_POD="$(kubectl get pod -l app=shoppingassistantservice \ + -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)" + +echo "${ASSISTANT_POD}" +if [ -n "${ASSISTANT_POD}" ]; then + kubectl describe pod "${ASSISTANT_POD}" | \ + grep -E 'Node:|Image:|Pulling|Pulled|Ready:' || true +fi +``` + +The `frontend` and `shoppingassistantservice` pods should both be on the N4A node pool. + +## Check the assistant logs + +Inspect both containers in the assistant pod: + +```bash +kubectl logs deployment/shoppingassistantservice -c server --tail=20 +kubectl logs deployment/shoppingassistantservice -c ollama --tail=20 +``` + +The `server` container serves the assistant API. The `ollama` container prepares the local model runtime. + +{{% notice Tip %}} +The first rollout can take a few minutes because the Ollama sidecar pulls the model when the pod starts. Early `/readyz` checks can return `503` until the model is available. +{{% /notice %}} + +## Validate the assistant through the storefront + +Create a session with cookies and send one assistant request through the storefront: + +```bash +curl --max-time 30 -s -c /tmp/assistant.cookies "${APP_URL}/assistant" >/dev/null + +for i in {1..12}; do + if curl --max-time 120 -s \ + -b /tmp/assistant.cookies \ + -c /tmp/assistant.cookies \ + -H 'Content-Type: application/json' \ + -d '{"message":"Find a mug for a minimalist desk setup."}' \ + "${APP_URL}/bot" | tee /tmp/assistant-response.json | jq .; then + break + fi + sleep 10 +done +``` + +The JSON response should include a non-empty assistant message. The exact product names can vary, but the response should be grounded in the live product catalog. + +## Try the browser flow + +Open the storefront URL in your browser. Refresh the page if it was already open before the assistant-enabled frontend rolled out. + +Try these prompts in the assistant panel: + +```text +Find one desk accessory that looks clean and minimalist for a home office. +Add that item to my cart. +yes +What is in my cart now? +``` + +The assistant should recommend a catalog item, ask for confirmation before changing the cart, and then show the cart contents. + +## What you've accomplished + +You've deployed the assistant on N4A and validated it through the same storefront path that a shopper uses. + +Next, you'll observe the assistant as a live workload and capture the N4A benchmark baseline. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/images/mixed-placement-agentic-storefront-on-axion.png b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/images/mixed-placement-agentic-storefront-on-axion.png new file mode 100644 index 0000000000..9a2fd8a01f Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/images/mixed-placement-agentic-storefront-on-axion.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/images/storefront-running-on-google-axion.jpeg b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/images/storefront-running-on-google-axion.jpeg new file mode 100644 index 0000000000..64a4b205a6 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/images/storefront-running-on-google-axion.jpeg differ diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/move-assistant-c4a.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/move-assistant-c4a.md new file mode 100644 index 0000000000..d9395fab9e --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/move-assistant-c4a.md @@ -0,0 +1,211 @@ +--- +title: Move the assistant to C4A and compare results +description: Move only the assistant deployment to C4A, rerun the fixed benchmark, and compare it with the N4A baseline. +weight: 9 +layout: "learningpathall" +--- + +## Create the mixed-placement overlay + +If you opened a new terminal, return to the source tree and restore the required variables: + +```bash +cd "${HOME}/n4a-c4a/microservices-demo" + +export PROJECT_ID="$(gcloud config get-value project)" +export ARTIFACT_REGION="us-central1" +export ARTIFACT_REPO="axion-workshop" +export ASSISTANT_IMAGE_REPO="${ARTIFACT_REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REPO}/shoppingassistantservice" +export ASSISTANT_IMAGE_TAG="lab-v1" +export N4A_NODE_POOL_NAME="arm64-pool-n4a2" +export C4A_NODE_POOL_NAME="arm64-pool-c4a" +export FRONTEND_IP="$(kubectl get service frontend-external \ + -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" +export APP_URL="http://${FRONTEND_IP}" +``` + +Create a second overlay that keeps the storefront on N4A and moves only `shoppingassistantservice` to C4A: + +```bash +mkdir -p kustomize/overlays/assistant-c4a + +cat < kustomize/overlays/assistant-c4a/kustomization.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: +- ../../base +components: +- ../../components/shopping-assistant +images: +- name: shoppingassistantservice + newName: ${ASSISTANT_IMAGE_REPO} + newTag: ${ASSISTANT_IMAGE_TAG} +patches: +- path: keep-storefront-on-n4a.yaml + target: + kind: Deployment +- path: shoppingassistant-c4a.yaml + target: + kind: Deployment + name: shoppingassistantservice +EOF + +cat < kustomize/overlays/assistant-c4a/keep-storefront-on-n4a.yaml +- op: add + path: /spec/template/spec/nodeSelector + value: + cloud.google.com/gke-nodepool: ${N4A_NODE_POOL_NAME} +- op: add + path: /spec/template/spec/tolerations + value: + - key: kubernetes.io/arch + operator: Equal + value: arm64 + effect: NoSchedule +EOF + +cat < kustomize/overlays/assistant-c4a/shoppingassistant-c4a.yaml +- op: replace + path: /spec/template/spec/nodeSelector + value: + cloud.google.com/gke-nodepool: ${C4A_NODE_POOL_NAME} +- op: replace + path: /spec/template/spec/tolerations + value: + - key: kubernetes.io/arch + operator: Equal + value: arm64 + effect: NoSchedule +EOF +``` + +## Render the mixed-placement overlay + +Review and render the overlay before you apply it: + +```bash +sed -n '1,140p' kustomize/overlays/assistant-c4a/kustomization.yaml +sed -n '1,140p' kustomize/overlays/assistant-c4a/keep-storefront-on-n4a.yaml +sed -n '1,140p' kustomize/overlays/assistant-c4a/shoppingassistant-c4a.yaml +kubectl kustomize kustomize/overlays/assistant-c4a | sed -n '1,240p' +``` + +Do not continue if `newName`, `newTag`, or either node-pool value is blank. + +{{% notice Note %}} +The first patch keeps the storefront on N4A by default. The second patch overrides only `shoppingassistantservice` and moves that deployment to C4A. +{{% /notice %}} + +## Apply the mixed-placement overlay + +Apply the overlay and wait for the rollout: + +```bash +kubectl apply -k kustomize/overlays/assistant-c4a +kubectl rollout status deployment/frontend --timeout=600s +kubectl rollout status deployment/shoppingassistantservice --timeout=1200s +``` + +Confirm the final placement: + +```bash +kubectl get pods -o wide | grep -E 'frontend|shoppingassistantservice' +kubectl get nodes -L cloud.google.com/gke-nodepool,node.kubernetes.io/instance-type,kubernetes.io/arch +``` + +The frontend should remain on N4A, and `shoppingassistantservice` should now run on C4A. + +## Warm the assistant on C4A + +Send one request through the storefront after the move: + +```bash +curl --max-time 30 -s -c /tmp/assistant.cookies "${APP_URL}/assistant" >/dev/null + +for i in {1..12}; do + if curl --max-time 120 -s \ + -b /tmp/assistant.cookies \ + -c /tmp/assistant.cookies \ + -H 'Content-Type: application/json' \ + -d '{"message":"Show me one desk item that looks minimal and practical."}' \ + "${APP_URL}/bot" | tee /tmp/assistant-response.json | jq .; then + break + fi + sleep 10 +done +``` + +The response should contain a non-empty message. + +## Run the fixed C4A benchmark + +Run the same fixed batch with the expected pool set to C4A: + +```bash +python3 workshop/fixed_batch_trial.py \ + --base-url "${APP_URL}" \ + --expected-pool "${C4A_NODE_POOL_NAME}" \ + --concurrency 2 \ + --prompts-per-worker 8 \ + --delay-seconds 0.2 \ + --request-timeout-seconds 120 \ + --telemetry-interval-seconds 0.5 \ + --idle-threshold-cores 1.0 \ + --idle-stable-samples 3 \ + --idle-timeout-seconds 180 \ + --request-output-file workshop/request-fixed-c4a.jsonl \ + --telemetry-output-file workshop/telemetry-fixed-c4a.jsonl | \ + tee workshop/fixed-c4a-summary.log +``` + +Print the C4A summary line: + +```bash +grep 'SUMMARY_JSON' workshop/fixed-c4a-summary.log +``` + +The summary should show successful requests and the assistant running on the C4A node pool. Do not compare the runs if the summary line is missing or reports a different node pool. + +## Compare the two runs + +Compare the N4A and C4A summaries: + +```bash +python3 workshop/compare_summaries.py workshop/fixed-n4a-summary.log workshop/fixed-c4a-summary.log +``` + +Focus on these signals: + +- Successful request count +- Total benchmark duration +- Average request latency +- Maximum request latency +- Average assistant CPU cores during the measured batch + +Your numbers can vary by prompt mix, model warmup state, node pressure, and cluster conditions. Treat the comparison as evidence for this assistant workflow, and repeat the same pattern with your own traffic before choosing a placement. + +{{% notice Note %}} +Assistant CPU values come from sampled Kubernetes telemetry. Short bursts can be underrepresented, especially on faster runs. Treat request success, benchmark duration, and latency as the primary comparison signals. +{{% /notice %}} + +## Explain the result + +You changed one architectural variable: the placement of the assistant. The storefront tier, prompts, model, traffic shape, and benchmark script stayed the same. + +The application now has three states: + +- Storefront only on N4A +- Storefront plus assistant on N4A +- Storefront on N4A with only the assistant on C4A + +That sequence is the core architectural point. A modern application can contain a steady application tier and a bursty AI reasoning tier. N4A supports the steady storefront and core services, while C4A gives you a placement to evaluate for concentrated, latency-sensitive assistant work. + +For this application, the useful pattern is not replacing one machine series with another. It is placing each workload tier where its behavior fits best. + +If your N4A and C4A results are close, or if N4A looks better for a particular run, that is still useful information. The decision should come from the workload behavior you observe, not from assuming one Axion-based machine series is always the right answer. + +## What you've accomplished + +You've built, observed, and compared a mixed-placement storefront application on Axion. You kept the storefront on N4A, moved only the assistant tier to C4A, and used the same benchmark to compare both placements. + +You are now ready to use the same overlay and benchmark pattern to evaluate which Axion-based machine series matches each tier's workload behavior. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/observe-benchmark-n4a.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/observe-benchmark-n4a.md new file mode 100644 index 0000000000..8f6e4563d2 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/observe-benchmark-n4a.md @@ -0,0 +1,187 @@ +--- +title: Observe and benchmark the assistant on N4A +description: Start the telemetry dashboard, generate assistant traffic, and capture a fixed N4A benchmark summary. +weight: 8 +layout: "learningpathall" +--- + +## Review the telemetry scripts + +The source tree includes helper scripts that focus traffic and telemetry on the assistant workflow: + +- `workshop/telemetry_collector.py` samples Kubernetes metrics and records the assistant node pool. +- `workshop/mini_dashboard.py` renders a simple live dashboard. +- `workshop/request_driver.py` sends live assistant traffic. +- `workshop/fixed_batch_trial.py` runs a repeatable benchmark. +- `workshop/compare_summaries.py` compares benchmark summaries. + +Compile the scripts before you run them: + +```bash +kubectl get deployment shoppingassistantservice +kubectl get pods -l app=shoppingassistantservice -o wide +kubectl top pod -l app=shoppingassistantservice --containers + +python3 -m py_compile workshop/telemetry_collector.py +python3 -m py_compile workshop/mini_dashboard.py +python3 -m py_compile workshop/request_driver.py +python3 -m py_compile workshop/fixed_batch_trial.py +python3 -m py_compile workshop/compare_summaries.py +``` + +If `kubectl top` is not available yet, make sure metrics collection is enabled in the cluster before you rely on CPU telemetry. + +## Open three terminals + +Use three terminals so telemetry, the dashboard, and benchmark commands can run at the same time. + +If a fresh terminal does not have the variables from earlier, run: + +```bash +cd "${HOME}/n4a-c4a/microservices-demo" + +export PROJECT_ID="$(gcloud config get-value project)" +export ARTIFACT_REGION="us-central1" +export ARTIFACT_REPO="axion-workshop" +export REPO_ROOT="${HOME}/n4a-c4a" +export REPO="${REPO_ROOT}/microservices-demo" +export FRONTEND_IP="$(kubectl get service frontend-external \ + -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" +export APP_URL="http://${FRONTEND_IP}" +export N4A_NODE_POOL_NAME="arm64-pool-n4a2" +export C4A_NODE_POOL_NAME="arm64-pool-c4a" +export ASSISTANT_IMAGE_REPO="${ARTIFACT_REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REPO}/shoppingassistantservice" +export ASSISTANT_IMAGE_TAG="lab-v1" +export ASSISTANT_IMAGE="${ASSISTANT_IMAGE_REPO}:${ASSISTANT_IMAGE_TAG}" +``` + +Use this terminal map: + +| Terminal | Purpose | +|----------|---------| +| Terminal 1 | Telemetry collector | +| Terminal 2 | Dashboard web server | +| Terminal 3 | Live traffic and fixed benchmarks | + +## Start telemetry collection + +Run this in Terminal 1. The `rm` command clears previous live telemetry samples for the dashboard: + +```bash +cd "${REPO}" || exit 1 +rm -f workshop/telemetry-live.jsonl + +python3 workshop/telemetry_collector.py \ + --interval-seconds 2 \ + --output-file workshop/telemetry-live.jsonl +``` + +Leave this command running. + +## Start the dashboard + +Run this in Terminal 2. The `rm` command clears previous live request data for the dashboard: + +```bash +cd "${REPO}" || exit 1 +rm -f workshop/request-live.jsonl workshop/request-live.log + +python3 workshop/mini_dashboard.py \ + --telemetry-file workshop/telemetry-live.jsonl \ + --request-file workshop/request-live.jsonl \ + --port 5000 +``` + +If port `5000` is already in use, choose another available port, such as `5001`. If you use Cloud Shell, open **Web Preview** and select the port you used. + +## Confirm assistant health + +Run this in Terminal 3: + +```bash +cd "${REPO}" || exit 1 + +export FRONTEND_IP="$(kubectl get service frontend-external \ + -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" +export APP_URL="http://${FRONTEND_IP}" + +kubectl get deployment shoppingassistantservice +kubectl rollout status deployment/shoppingassistantservice --timeout=1200s +kubectl get pods -o wide | grep -E 'frontend|shoppingassistantservice' +``` + +## Generate a short live burst + +Start a short request burst so the dashboard shows activity: + +```bash +rm -f workshop/request-live.pid + +python3 workshop/request_driver.py \ + --base-url "${APP_URL}" \ + --concurrency 4 \ + --delay-seconds 0 \ + --output-file workshop/request-live.jsonl > workshop/request-live.log 2>&1 & + +echo $! > workshop/request-live.pid +``` + +Let it run for about 20 seconds, then inspect the recent log lines: + +```bash +sleep 20 +tail -n 20 workshop/request-live.log +``` + +Stop the live burst before you run the fixed benchmark: + +```bash +REQUEST_DRIVER_PID="$(cat workshop/request-live.pid)" + +if [ -n "${REQUEST_DRIVER_PID}" ] && \ + ps -p "${REQUEST_DRIVER_PID}" -o command= | grep -q 'workshop/request_driver.py'; then + kill "${REQUEST_DRIVER_PID}" + wait "${REQUEST_DRIVER_PID}" 2>/dev/null || true +fi + +rm -f workshop/request-live.pid +``` + +Stopping the live burst keeps the N4A and C4A benchmark runs comparable. + +## Run the fixed N4A benchmark + +Run the fixed batch while the assistant is still on N4A: + +```bash +python3 workshop/fixed_batch_trial.py \ + --base-url "${APP_URL}" \ + --expected-pool "${N4A_NODE_POOL_NAME}" \ + --concurrency 2 \ + --prompts-per-worker 8 \ + --delay-seconds 0.2 \ + --request-timeout-seconds 120 \ + --telemetry-interval-seconds 0.5 \ + --idle-threshold-cores 1.0 \ + --idle-stable-samples 3 \ + --idle-timeout-seconds 180 \ + --request-output-file workshop/request-fixed-n4a.jsonl \ + --telemetry-output-file workshop/telemetry-fixed-n4a.jsonl | \ + tee workshop/fixed-n4a-summary.log +``` + +The command can keep running after the last request lines appear. It is still sampling telemetry and waiting for the assistant to settle back toward idle. + +Print the summary line: + +```bash +grep 'SUMMARY_JSON' workshop/fixed-n4a-summary.log +``` + +The summary should show successful requests and the assistant running on the N4A node pool. Do not continue if the summary line is missing or reports a different node pool. + +## What you've accomplished + +You've observed the assistant as a live workload and captured the N4A benchmark baseline. + +Next, you'll move only the assistant tier to C4A and run the same benchmark again. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/project-setup.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/project-setup.md new file mode 100644 index 0000000000..443f516625 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/project-setup.md @@ -0,0 +1,148 @@ +--- +title: Set up the source tree and cluster access +description: Configure the Google Cloud variables, connect kubectl to the GKE cluster, and clone the source tree for the storefront assistant. +weight: 3 +layout: "learningpathall" +--- + +## Configure your shell + +Use [Cloud Shell](https://cloud.google.com/shell/docs/using-cloud-shell) or a local Linux or macOS terminal that has the required tools installed. + +Confirm that the tools are available: + +```bash +gcloud --version +kubectl version --client +docker buildx version +git --version +python3 --version +jq --version +``` + +## Set environment variables + +Set the project, cluster, Artifact Registry, and node-pool variables. Adjust the values to match your prepared environment: + +```bash +export PROJECT_ID="$(gcloud config get-value project)" +export CLUSTER_NAME="online-boutique" +export ARTIFACT_REGION="us-central1" +export ARTIFACT_REPO="axion-workshop" +export ARTIFACT_REGISTRY="${ARTIFACT_REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REPO}" +export N4A_NODE_POOL_NAME="arm64-pool-n4a2" +export C4A_NODE_POOL_NAME="arm64-pool-c4a" +``` + +If `PROJECT_ID` is empty or prints `(unset)`, set it manually: + +```bash +export PROJECT_ID="YOUR_PROJECT_ID" +gcloud config set project "${PROJECT_ID}" +export ARTIFACT_REGISTRY="${ARTIFACT_REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REPO}" +``` + +## Discover the cluster location + +Query Google Cloud for the actual cluster location. This avoids hard-coding a zone or region: + +```bash +export CLUSTER_LOCATION="$(gcloud container clusters list \ + --project "${PROJECT_ID}" \ + --filter="name=${CLUSTER_NAME}" \ + --format='value(location)' | head -n 1)" + +echo "${CLUSTER_LOCATION}" +``` + +The output should print the cluster location, such as `us-central1` or `us-central1-a`. + +Do not continue if the output is empty. Confirm `CLUSTER_NAME` and `PROJECT_ID`, and then run the command again. + +## Connect kubectl to the cluster + +Configure `gcloud` and get cluster credentials: + +```bash +gcloud config set project "${PROJECT_ID}" +gcloud container clusters get-credentials "${CLUSTER_NAME}" \ + --location "${CLUSTER_LOCATION}" +``` + +Confirm the current Kubernetes context: + +```bash +kubectl config current-context +kubectl get nodes +``` + +## Confirm the Artifact Registry repository + +The assistant image is pushed to Artifact Registry. Create the Docker repository if it doesn't already exist: + +```bash +gcloud artifacts repositories describe "${ARTIFACT_REPO}" \ + --location "${ARTIFACT_REGION}" >/dev/null 2>&1 || \ +gcloud artifacts repositories create "${ARTIFACT_REPO}" \ + --repository-format=docker \ + --location "${ARTIFACT_REGION}" \ + --description="Axion storefront assistant images" +``` + +## Authenticate Docker to Artifact Registry + +Authenticate Docker so it can push your assistant image: + +```bash +gcloud auth configure-docker "${ARTIFACT_REGION}-docker.pkg.dev" --quiet +``` + +## Clone the lab source tree + +Clone the Arm-maintained Online Boutique source tree for this Learning Path. If an existing checkout is present, move it aside before cloning a fresh copy: + +```bash +export REPO_ROOT="${HOME}/n4a-c4a" +export REPO="${REPO_ROOT}/microservices-demo" +mkdir -p "${REPO_ROOT}" + +if [ -d "${REPO}" ]; then + mv "${REPO}" "${REPO}.$(date +%Y%m%d%H%M%S).bak" +fi + +git clone --branch arm-multiarch-baseline --single-branch \ + https://github.com/ArmDeveloperEcosystem/microservices-demo.git "${REPO}" + +cd "${REPO}" +``` + +Confirm the working tree: + +```bash +pwd +git branch --show-current +``` + +The branch should be `arm-multiarch-baseline`. + +## Confirm the Arm foundation + +Check that the cluster has Arm node pools for N4A and C4A: + +```bash +kubectl get nodes -L kubernetes.io/arch,cloud.google.com/gke-nodepool +``` + +Inspect the frontend manifest in the source tree: + +```bash +grep -n "image:" kustomize/base/frontend.yaml +``` + +The cluster uses Arm-based Axion node pools, and the storefront manifest uses image references that can resolve to Arm-compatible image variants. + +## What you've accomplished + +You've configured your Google Cloud project, connected `kubectl` to the prepared GKE cluster, confirmed Artifact Registry access, and cloned the source tree that contains the assistant implementation. + +Next, you'll validate the starting storefront baseline on N4A before you add the assistant. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/review-assistant.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/review-assistant.md new file mode 100644 index 0000000000..eff56a894d --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/review-assistant.md @@ -0,0 +1,87 @@ +--- +title: Review the shopping assistant implementation +description: Inspect the assistant source, tool functions, Dockerfile, and Kubernetes component before you build the image. +weight: 5 +layout: "learningpathall" +--- + +## Inspect the assistant files + +The source tree already contains the missing AI tier. Before you build it, review the files that make up the assistant: + +```bash +find src/shoppingassistantservice -maxdepth 2 -type f | sort +find kustomize/components/shopping-assistant -maxdepth 2 -type f | sort +``` + +You should see the assistant source folder and the `shopping-assistant` Kustomize component. + +{{% notice Note %}} +If you see a `__pycache__` directory, you can ignore it. Python creates that directory during local syntax checks. +{{% /notice %}} + +## Review the runtime dependencies and gRPC stubs + +The assistant uses a small Python runtime and generated gRPC stubs to call the live storefront services: + +```bash +sed -n '1,20p' src/shoppingassistantservice/requirements.txt +sed -n '1,25p' src/shoppingassistantservice/demo_pb2.py +grep -nE 'class (CartServiceStub|ProductCatalogServiceStub)' \ + src/shoppingassistantservice/demo_pb2_grpc.py +``` + +The generated protobuf files are dense because they are machine-generated. You only need to confirm that the cart and product catalog stubs exist. + +## Review the assistant service + +Read the beginning and end of the assistant service: + +```bash +sed -n '1,220p' src/shoppingassistantservice/shoppingassistantservice.py +tail -n 20 src/shoppingassistantservice/shoppingassistantservice.py +``` + +Focus on these parts of the file: + +- Imports and environment variables +- Tool functions that call the live storefront services +- Request handlers that decide how the assistant responds +- The `app.run(...)` entrypoint + +Highlight the tool functions: + +```bash +grep -nE 'def (search_catalog|get_product_details|get_cart|add_to_cart|handle_[a-z_]+)' \ + src/shoppingassistantservice/shoppingassistantservice.py +``` + +These functions define the assistant behavior. The service searches the catalog, fetches product details, reads the cart, and updates the cart after confirmation. These are live application tool calls, not mock responses, which is what makes the assistant part of the storefront workflow. + +## Confirm that the Python source is valid + +Compile the assistant source: + +```bash +python3 -m py_compile src/shoppingassistantservice/shoppingassistantservice.py +``` + +The command should return no output. Do not continue if this command fails. + +## Review the container and Kubernetes component + +Review the Dockerfile and the Kustomize component: + +```bash +sed -n '1,80p' src/shoppingassistantservice/Dockerfile +sed -n '1,120p' kustomize/components/shopping-assistant/kustomization.yaml +sed -n '1,220p' kustomize/components/shopping-assistant/shoppingassistantservice.yaml +``` + +The Dockerfile packages the Python assistant service. The Kubernetes component creates `shoppingassistantservice`, adds the Ollama sidecar, and patches `frontend` so the assistant becomes part of the storefront experience. Because the assistant and Ollama run in the same pod, moving this deployment later moves the AI logic and local reasoning runtime together. + +## What you've accomplished + +You've reviewed the source code, service contracts, container packaging, and Kubernetes component for the assistant tier. + +Next, you'll build the assistant image for `linux/arm64` and push it to Artifact Registry. diff --git a/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/validate-n4a-baseline.md b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/validate-n4a-baseline.md new file mode 100644 index 0000000000..806b5b6c5e --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/storefront-ai-assistant-gke-axion/validate-n4a-baseline.md @@ -0,0 +1,151 @@ +--- +title: Deploy and validate the storefront baseline on N4A +description: Deploy the Online Boutique storefront on N4A and confirm that the assistant service has not been deployed yet. +weight: 4 +layout: "learningpathall" +--- + +## Check the starting architecture + +Before you add the assistant tier, create and confirm the starting state of the application: + +- The cluster contains N4A and C4A node pools. +- The storefront baseline runs on N4A after you apply the baseline overlay. +- `shoppingassistantservice` is not running yet. + +The missing assistant is intentional. It is the service you build and deploy in the next steps. + +If you are rerunning this Learning Path in a cluster that already has the assistant from an earlier run, remove the old assistant resources before you create the baseline: + +```bash +kubectl delete deployment shoppingassistantservice --ignore-not-found +kubectl delete service shoppingassistantservice --ignore-not-found +kubectl delete serviceaccount shoppingassistantservice --ignore-not-found +``` + +## Deploy the storefront baseline + +Create a Kustomize overlay that runs the baseline storefront on the N4A node pool: + +```bash +mkdir -p kustomize/overlays/storefront-n4a + +cat < kustomize/overlays/storefront-n4a/kustomization.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: +- ../../base +patches: +- path: node-selector.yaml + target: + kind: Deployment +EOF + +cat < kustomize/overlays/storefront-n4a/node-selector.yaml +- op: add + path: /spec/template/spec/nodeSelector + value: + cloud.google.com/gke-nodepool: ${N4A_NODE_POOL_NAME} +- op: add + path: /spec/template/spec/tolerations + value: + - key: kubernetes.io/arch + operator: Equal + value: arm64 + effect: NoSchedule +EOF +``` + +Render the overlay and confirm that the N4A node pool value is present: + +```bash +sed -n '1,120p' kustomize/overlays/storefront-n4a/kustomization.yaml +sed -n '1,120p' kustomize/overlays/storefront-n4a/node-selector.yaml +kubectl kustomize kustomize/overlays/storefront-n4a | sed -n '1,240p' +``` + +Do not apply the overlay if the rendered node-pool value is blank. + +Apply the baseline and wait for the storefront deployments: + +```bash +kubectl apply -k kustomize/overlays/storefront-n4a +kubectl rollout status deployment/frontend --timeout=600s +kubectl rollout status deployment/cartservice --timeout=600s +kubectl rollout status deployment/checkoutservice --timeout=600s +kubectl rollout status deployment/productcatalogservice --timeout=600s +kubectl rollout status deployment/recommendationservice --timeout=600s +kubectl rollout status deployment/shippingservice --timeout=600s +kubectl rollout status deployment/paymentservice --timeout=600s +kubectl rollout status deployment/currencyservice --timeout=600s +kubectl rollout status deployment/emailservice --timeout=600s +kubectl rollout status deployment/adservice --timeout=600s +``` + +## List the cluster nodes + +Show the node pool, instance type, and CPU architecture for each node: + +```bash +kubectl get nodes \ + -L cloud.google.com/gke-nodepool,node.kubernetes.io/instance-type,kubernetes.io/arch +``` + +The output should include the N4A node pool, the C4A node pool, and `arm64` nodes in both pools. + +## List the running application pods + +Check the running storefront pods: + +```bash +kubectl get pods -o wide +``` + +Filter the output to the services you care about most: + +```bash +kubectl get pods -o wide | \ + grep -E 'frontend|cartservice|checkoutservice|productcatalogservice|shoppingassistantservice' +``` + +At this point, `frontend`, `cartservice`, `checkoutservice`, and `productcatalogservice` should be running. `shoppingassistantservice` should not appear yet. + +## Resolve the storefront endpoint + +Capture the external storefront URL: + +```bash +export FRONTEND_IP="$(kubectl get service frontend-external \ + -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" +export APP_URL="http://${FRONTEND_IP}" + +echo "${APP_URL}" +``` + +The output should be similar to: + +```output +http://34.x.x.x +``` + +If the external IP is empty, wait a minute and run the commands again. + +## Verify that the storefront responds + +Send a header request to the storefront: + +```bash +curl --max-time 30 -I "${APP_URL}" +``` + +The output should include an HTTP response header. A `200 OK` or `302 Found` response confirms that the baseline storefront is reachable. + +Open the printed URL in your browser. The Online Boutique storefront should load. + +![Online Boutique storefront page showing product cards and the Google Cloud label, confirming that the baseline storefront is reachable before the assistant is deployed#center](images/storefront-running-on-google-axion.jpeg "Online Boutique storefront running on Google Axion") + +## What you've accomplished + +You've confirmed that the storefront runs on N4A and that the assistant tier is still absent. This gives you a clean baseline before you add the AI service. + +Next, you'll inspect the assistant implementation and confirm that its source files are ready to build.