Skip to content

Commit 13938ce

Browse files
adding a takeaway
1 parent 3329bf2 commit 13938ce

7 files changed

Lines changed: 21 additions & 19 deletions

File tree

content/learning-paths/cross-platform/ai-agent-cpu-orchestration/1-overview.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,9 @@ layout: learningpathall
99

1010
## What you will run
1111

12-
You'll run a local concierge agent: a terminal application that answers research-style questions, such as finding restaurants, comparing products, or summarizing a topic, by searching the web, reading multiple pages, and writing a fact-checked summary.
12+
You'll run a local concierge agent: a terminal application that answers research-style questions, such as finding restaurants, comparing products, or summarizing a topic. The agent answers these questions by searching the web, reading multiple pages, and writing a fact-checked summary.
1313

14-
You'll run everything on your local Arm machine. The agent uses a large language model (LLM) served locally by [Ollama](https://ollama.com/), so your prompts and the pages you browse never leave the device. The same code runs on an Apple silicon MacBook, an Arm Linux laptop, or an NVIDIA DGX Spark.
14+
You'll run everything on your local Arm machine. The agent uses an LLM served locally by [Ollama](https://ollama.com/), so your prompts and the pages you browse never leave the device. The same code runs on an Apple silicon MacBook, an Arm Linux laptop, or an NVIDIA DGX Spark.
1515

1616
The agent is interesting not just because it runs locally, but because of *how the work is divided*. A common assumption is that an AI agent is "just the model." In practice, the model is only one stage in a longer pipeline, and most of the surrounding work runs on the CPU.
1717

@@ -30,7 +30,7 @@ Your question
3030
-> Write a fact-checked summary (GPU: model reasoning)
3131
```
3232

33-
The model is called several times, but between every model call the CPU does a large amount of orchestration: generating query variants, dispatching parallel network requests, merging and deduplicating results, scoring pages for relevance, and extracting structured data.
33+
The model is called several times, but between every model call, the CPU does a large amount of orchestration. The CPU generates query variants, dispatches parallel network requests, merges and deduplicates results, scores pages for relevance, and extracts structured data.
3434

3535
## Why the CPU matters in an agentic workflow
3636

@@ -59,6 +59,6 @@ By the end of this Learning Path, you'll be able to look at a single query and s
5959

6060
## What you've learned and what's next
6161

62-
You've now learned about the agent you'll build and how the steps it executes are split between the CPU and the GPU.
62+
You've now learned about the agent you'll build and the role the CPU plays in orchestrating the agent's workflow.
6363

6464
Next, you'll set up your environment: create a Python virtual environment, install the required packages, and get a Serper API key for web search.

content/learning-paths/cross-platform/ai-agent-cpu-orchestration/2-setup.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@ These steps are identical on an Apple silicon MacBook, an Arm Linux machine, and
1515

1616
### Get a Serper API key
1717

18-
The agent searches the web through [Serper](https://serper.dev/), a Google Search API. The free tier is enough to complete this Learning Path.
18+
The agent searches the web through [Serper](https://serper.dev/), a Google Search API. The free tier is enough for you to complete the Learning Path.
19+
20+
To get an API key:
1921

2022
1. Go to [serper.dev](https://serper.dev/) and create a free account.
2123
2. Open your dashboard and copy your API key.
@@ -61,7 +63,7 @@ export SERPER_API_KEY="your-serper-api-key"
6163
```
6264

6365
{{% notice Tip %}}
64-
This variable lasts only for the current terminal session. To keep it across sessions, add the same line to your shell profile, for example `~/.zshrc` on macOS or `~/.bashrc` on Linux.
66+
This variable lasts only for the current terminal session. To keep the variable across sessions, add the same line to your shell profile, for example `~/.zshrc` on macOS or `~/.bashrc` on Linux.
6567
{{% /notice %}}
6668

6769
### Download the agent script

content/learning-paths/cross-platform/ai-agent-cpu-orchestration/3-ollama.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ layout: learningpathall
99

1010
## Run AI models locally with Ollama
1111

12-
The agent's reasoning steps, such as choosing search terms, selecting URLs, and writing the final summary, are handled by a Large Language Model (LLM). Instead of calling a cloud API, you'll serve the model locally with [Ollama](https://ollama.com/).
12+
The agent's reasoning steps, such as choosing search terms, selecting URLs, and writing the final summary, are handled by an LLM. Instead of calling a cloud API, you'll serve the model locally with [Ollama](https://ollama.com/).
1313

1414
Ollama is a lightweight runtime that downloads open models, loads them into memory, and exposes a local HTTP API at `http://localhost:11434`. When the agent calls that endpoint, the model runs directly on your machine: the CPU and GPU on your MacBook, Arm Linux laptop, or NVIDIA DGX Spark.
1515

@@ -100,6 +100,6 @@ Type `/bye` to exit the model session.
100100

101101
## What you've accomplished and what's next
102102

103-
You've now set up Ollama and served a Google `gemma3:4b` model locally.
103+
You've now set up Ollama, served a Google `gemma3:4b` model locally, and tested the model.
104104

105105
Next, you'll look at how the agent code uses the model.

content/learning-paths/cross-platform/ai-agent-cpu-orchestration/4-code-walkthrough.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Understand the agent code
2+
title: Understand the concierge agent code
33
description: Review the concierge agent code to see where web search, page scraping, Ollama model calls, and CPU orchestration happen in the workflow.
44
weight: 5
55

@@ -54,7 +54,7 @@ def call_gemma_ollama(prompt, output_format="json", timing=None,
5454
...
5555
```
5656

57-
Two details are worth highlighting:
57+
Two details are worth noting:
5858

5959
- `keep_alive: -1` tells Ollama to keep the model resident in memory between calls. The agent calls the model several times per query, so this avoids reloading the weights each time.
6060
- The function records timing as it streams. The time until the *first* token arrives is the model's input-token processing (prefill); the time spent streaming the rest is token generation. Separating these two values is what lets the agent show you the GPU breakdown later.
@@ -63,7 +63,7 @@ Because every model call goes through this one function, the agent only ever use
6363

6464
### The CPU orchestration pipeline
6565

66-
Between the model calls, the CPU prepares the context. Each stage below runs entirely on the CPU:
66+
Between the model calls, the CPU prepares the context. Each of the following stages runs entirely on the CPU:
6767

6868
| Stage | Function | What it does |
6969
|---|---|---|
@@ -77,7 +77,7 @@ Between the model calls, the CPU prepares the context. Each stage below runs ent
7777

7878
### The agentic chain
7979

80-
`run_concierge_agent()` ties the pipeline together. Reading it top to bottom shows how CPU and GPU steps alternate:
80+
`run_concierge_agent()` ties the pipeline together. When you read the function from top to bottom, you can see how CPU and GPU steps alternate:
8181

8282
1. GPU – `call_gemma_ollama()` turns your question into a search query.
8383
2. CPU – `generate_search_queries()` and `parallel_search()` expand and run the searches.

content/learning-paths/cross-platform/ai-agent-cpu-orchestration/5-run-and-examples.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ layout: learningpathall
1111

1212
Make sure the following are in place before you start:
1313

14-
- Ollama is running and serving `gemma3:4b` (from the previous section).
14+
- Ollama is running and serving `gemma3:4b`.
1515
- Your virtual environment is active and `SERPER_API_KEY` is set in the current terminal.
1616

1717
From your project directory, start the agent:
@@ -40,7 +40,7 @@ Type a question that benefits from searching and reading several pages. For exam
4040
Find three highly rated ramen restaurants in San Francisco that are open late
4141
```
4242

43-
As the agent works, watch the log lines. Cyan `[CPU]` lines show orchestration and web I/O; yellow `[GPU]` lines show each model call:
43+
As the agent works, watch the log lines. Cyan <code style="color:#00aaaa"><strong>[CPU]</strong></code> lines show orchestration and web I/O; yellow <code style="color:#aaaa00"><strong>[GPU]</strong></code> lines show each model call:
4444

4545
```output
4646
[GPU] Thinking with local Gemma model...

content/learning-paths/cross-platform/ai-agent-cpu-orchestration/6-cpu-orchestration.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Most agentic systems share the same shape, regardless of the task. Between model
2424

2525
These stages are mostly classical computing: concurrency, networking, parsing, and text processing. They run continuously throughout a query, while the GPU is used in focused bursts only when the model reasons. As agents call more tools and handle more data, this CPU-side work grows.
2626

27-
## Why this is a good fit for Arm
27+
## Why orchestration is a good fit for Arm
2828

2929
This kind of broad, concurrent orchestration and I/O is exactly the workload Arm CPUs handle efficiently. The same pattern scales across Arm platforms:
3030

@@ -45,6 +45,6 @@ Each experiment shifts the balance between CPU and GPU work and makes the orches
4545

4646
## What you've accomplished
4747

48-
You've now connected the agent's timeline to the CPU and GPU work behind each query. You've also seen how query fan-out, browsing concurrency, and model size change that balance.
48+
You've now connected the agent's timeline to the CPU and GPU work behind each query. You've also seen how the number of query variants, the size of the model, and browsing concurrency change that balance.
4949

50-
You can use this pattern to adapt the local agent for your own tasks on Arm.
50+
You can use what you've learned in this Learning Path to update your own agentic workflows so that CPU orchestration is more efficient.

content/learning-paths/cross-platform/ai-agent-cpu-orchestration/_index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@ description: Run a local AI concierge agent with Ollama on Arm and visualize how
55

66
minutes_to_complete: 45
77

8-
who_is_this_for: This is an introductory topic for developers who want to build a local, privacy-friendly AI agent on Arm hardware and visualize how the CPU orchestrates an agentic workflow around a locally served LLM.
8+
who_is_this_for: This is an introductory topic for developers who want to build a local, privacy-friendly AI agent on Arm hardware and visualize how the CPU orchestrates an agentic workflow around a locally served Large Language Model (LLM).
99

1010
learning_objectives:
1111
- Set up a Python environment and obtain a Serper web search API key for the agent
1212
- Serve an LLM locally with Ollama and select a model that fits your hardware
13-
- Explain how the CPU orchestrates an agentic workflow while the GPU handles model inference
13+
- Understand how the CPU orchestrates an agentic workflow while the GPU handles model inference
1414
- Run the agent and interpret the CPU and GPU timeline it produces for each query
1515

1616
prerequisites:

0 commit comments

Comments
 (0)