You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/ai-agent-cpu-orchestration/1-overview.md
+5-4Lines changed: 5 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,8 @@ layout: learningpathall
10
10
11
11
In this Learning Path, you'll build and run a *local concierge agent*: a terminal application that answers research-style questions, such as finding restaurants, comparing products, or summarizing a topic, by searching the web, reading multiple pages, and writing a fact-checked summary.
12
12
13
-
Everything runs on your own Arm machine. The agent uses a large language model (LLM) served locally by [Ollama](https://ollama.com/), so your prompts and the pages you browse never leave the device. The same code runs on an Apple silicon MacBook, an Arm Linux laptop, or an NVIDIA DGX Spark.
13
+
Everything runs on your own Arm machine. The agent uses an LLM served locally by [Ollama](https://ollama.com/), so your prompts and the pages you browse never leave the device. The same code runs on an Apple silicon MacBook, an Arm Linux laptop, or an NVIDIA DGX Spark.
14
+
14
15
15
16
The agent is interesting not just because it runs locally, but because of *how the work is divided*. A common assumption is that an AI agent is "just the model." In practice, the model is only one stage in a longer pipeline, and most of the surrounding work runs on the CPU.
16
17
@@ -29,7 +30,7 @@ Your question
29
30
-> Write a fact-checked summary (GPU: model reasoning)
30
31
```
31
32
32
-
The model is called several times, but between every model call the CPU does a large amount of orchestration: generating query variants, dispatching parallel network requests, merging and deduplicating results, scoring pages for relevance, and extracting structured data such as phone numbers and opening hours.
33
+
The model is called several times, but between every model call the CPU does a large amount of orchestration: generating query variants, dispatching parallel network requests, merging and deduplicating results, scoring pages for relevance, and extracting structured data.
33
34
34
35
## Why the CPU matters in an agentic workflow
35
36
@@ -48,8 +49,8 @@ This division is a natural fit for Arm platforms. On an Apple silicon MacBook or
48
49
49
50
The program is instrumented so you can see this division directly. As it runs, it prints:
50
51
51
-
-`[CPU]` log lines (in cyan) for every orchestration and processing step
52
-
-`[GPU]` log lines (in yellow) for every model call
52
+
-<codestyle="color:#00aaaa"><strong>[CPU]</strong></code> log lines for every orchestration and processing step
53
+
-<codestyle="color:#aaaa00"><strong>[GPU]</strong></code> log lines for every model call
53
54
- A timing breakdown of CPU time versus GPU input-token processing and token generation
54
55
- A text-based timeline that visualizes when the CPU and GPU are each active
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/ai-agent-cpu-orchestration/2-setup.md
+4-12Lines changed: 4 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,10 +21,6 @@ The agent searches the web through [Serper](https://serper.dev/), a Google Searc
21
21
22
22
You'll set this key as an environment variable later in this section, so the agent can read it without the key being written into the code.
23
23
24
-
{{% notice Note %}}
25
-
Never paste your API key directly into source code that you share or commit. Always read it from an environment variable, as shown below.
26
-
{{% /notice %}}
27
-
28
24
## Create a project directory and virtual environment
29
25
30
26
A virtual environment keeps the agent's dependencies isolated from your system Python, so you always run against the right package versions.
@@ -47,7 +43,7 @@ Your shell prompt now shows `(.venv)`, which confirms the environment is active.
47
43
48
44
## Install the required packages
49
45
50
-
The agent uses `requests` for HTTP calls and `beautifulsoup4` to parse web pages. The remaining libraries it uses, such as `smtplib`, `json`, and `concurrent.futures`, are part of the Python standard library.
46
+
The agent uses `requests` for HTTP calls and `beautifulsoup4` to parse web pages.
51
47
52
48
Install the two packages:
53
49
@@ -69,12 +65,12 @@ This variable only lasts for the current terminal session. To keep it across ses
69
65
70
66
## Download the agent script
71
67
72
-
Download the complete agent script, [concierge_agent.py](concierge_agent.py), into your project directory. You'll run it at the end of this Learning Path, and the next sections walk through how the important parts work.
68
+
Download the complete agent script, <ahref="/learning-paths/cross-platform/ai-agent-cpu-orchestration/concierge_agent.py"download>concierge_agent.py</a>, and move it into your `concierge-agent` directory. You'll run it at the end of this Learning Path, and the next sections walk through how the important parts work.
73
69
74
-
Download it directly from the command line:
70
+
Alternatively, download it directly from the command line:
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/ai-agent-cpu-orchestration/3-ollama.md
+15-23Lines changed: 15 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,46 +8,38 @@ layout: learningpathall
8
8
9
9
## Run AI models locally with Ollama
10
10
11
-
The agent's reasoning steps, such as choosing search terms, selecting URLs, and writing the final summary, are handled by a large language model. Instead of calling a cloud API, this Learning Path serves the model locally with [Ollama](https://ollama.com/).
11
+
The agent's reasoning steps, such as choosing search terms, selecting URLs, and writing the final summary, are handled by an LLM. Instead of calling a cloud API, this Learning Path serves the model locally with [Ollama](https://ollama.com/).
12
12
13
-
Ollama is a lightweight runtime that downloads open models, loads them into memory, and exposes a local HTTP API at `http://localhost:11434`. When the agent calls that endpoint, the model runs directly on your machine: the CPU and GPU on your MacBook, Arm Linux laptop, or NVIDIA DGX Spark. Nothing is sent to an external service.
13
+
Ollama is a lightweight runtime that downloads open models, loads them into memory, and exposes a local HTTP API at `http://localhost:11434`. When the agent calls that endpoint, the model runs directly on your machine: the CPU and GPU on your MacBook, Arm Linux laptop, or NVIDIA DGX Spark. Nothing is sent to an external service or cloud.
14
14
15
15
Running locally has three benefits that matter for an agent:
16
16
17
17
-**Privacy**: your prompts and the web content the agent reads stay on the device.
18
18
-**Cost**: there are no per-token API charges, so you can run many queries freely.
19
19
-**Control**: you choose the exact model and keep it resident in memory for fast repeated calls.
20
20
21
-
## Install Ollama
21
+
## Install and start Ollama
22
22
23
-
Install Ollama for your operating system.
23
+
Install [Ollama](https://ollama.com/) and start its server using one of the following options. The server exposes a local API at `http://localhost:11434` that the agent connects to.
On macOS you can also download the Ollama desktop app from [ollama.com/download](https://ollama.com/download). On an NVIDIA DGX Spark, follow the Linux instructions; Ollama uses the Blackwell GPU for inference automatically.
36
-
{{% /notice %}}
37
-
38
-
## Start the Ollama server
39
36
40
-
Start the Ollama background service, which hosts the local API the agent connects to:
41
-
42
-
```bash
37
+
# Start the server (leave this running, and open a second terminal)
43
38
ollama serve
44
-
```
45
-
46
-
Leave this running in its own terminal. Open a second terminal for the remaining commands.
39
+
{{< /tab >}}
40
+
{{< /tabpane >}}
47
41
48
-
{{% notice Tip %}}
49
-
On macOS, starting the Ollama desktop app already runs the server in the background, so you can skip `ollama serve`.
50
-
{{% /notice %}}
42
+
With [Homebrew](https://brew.sh/), `brew services` runs Ollama in the background, so you can use the same terminal throughout. With the install script, `ollama serve` runs in the foreground, so keep that terminal open and use a second terminal for the remaining commands.
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/ai-agent-cpu-orchestration/_index.md
+4-6Lines changed: 4 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
title: Build a CPU-orchestrated local AI agent with Ollama and Gemma
2
+
title: Build and run a local AI agent with Ollama
3
3
4
4
description: Learn how to build a local AI concierge agent that runs entirely on your Arm machine, using the CPU to orchestrate web search, scraping, and text processing while a local Gemma model handles reasoning on the GPU.
5
5
@@ -9,18 +9,17 @@ cascade:
9
9
10
10
minutes_to_complete: 45
11
11
12
-
who_is_this_for: This is an introductory topic for developers who want to build a local, privacy-friendly AI agent on Arm hardware and understand how the CPU orchestrates an agentic workflow around a locally served large language model.
12
+
who_is_this_for: This is an introductory topic for developers who want to build a local, privacy-friendly AI agent on Arm hardware and visualize how the CPU orchestrates an agentic workflow around a locally served LLM.
13
13
14
14
learning_objectives:
15
-
- Set up a Python environment and obtain the API access needed to run a local AI agent
16
-
- Serve a large language model locally with Ollama and select a model that fits your hardware
15
+
- Set up a Python environment and obtain a Serper web search API key for the agent
16
+
- Serve an LLM locally with Ollama and select a model that fits your hardware
17
17
- Explain how the CPU orchestrates an agentic workflow while the GPU handles model inference
18
18
- Run the agent and interpret the CPU and GPU timeline it produces for each query
19
19
20
20
prerequisites:
21
21
- An Arm-based computer running Linux or macOS, such as an Apple silicon MacBook or an NVIDIA DGX Spark
22
22
- Familiarity with running Python scripts from the terminal
0 commit comments