Skip to content

Commit a36cc25

Browse files
authored
Merge pull request #3366 from anupras-mohapatra-arm/laptops-and-desktops
DGXSpark persistent agent LP editorial review
2 parents 9885971 + 56397fe commit a36cc25

8 files changed

Lines changed: 348 additions & 340 deletions

File tree

content/learning-paths/laptops-and-desktops/dgx_persistent_agent/1_introduce.md

Lines changed: 22 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,25 @@
11
---
2-
title: Understand persistent AI runtime architecture
2+
title: Explore persistent AI runtime architecture on NVIDIA DGX Spark
3+
description: Explore the persistent AI runtime architecture for NVIDIA DGX Spark, including Hermes Agent orchestration, Ollama inference, Qdrant memory, and CPU/GPU responsibilities.
34
weight: 2
45

56
### FIXED, DO NOT MODIFY
67
layout: learningpathall
78
---
89

9-
## Understand persistent AI runtime architecture
10+
## Persistent AI runtime capabilities
1011

11-
You will build a persistent local AI runtime on NVIDIA [DGX Spark](https://www.nvidia.com/en-gb/products/workstations/dgx-spark/). The implementation is validated on DGX Spark, but the architecture also applies to other Arm Cortex-A platforms that can run containerized services and local AI runtimes.
12+
In this Learning Path, you'll build a persistent local AI runtime on NVIDIA [DGX Spark](https://www.nvidia.com/en-gb/products/workstations/dgx-spark/). The implementation is validated on DGX Spark, but the architecture can also apply to other Arm Cortex-A platforms that can run containerized services and local AI runtimes.
1213

13-
The final system is not a single chatbot process. It is a set of local services that run continuously, share a workspace, react to file events, generate summaries, create embeddings, store vector memory, retrieve context, and periodically reason about the state of the workspace.
14+
The final system isn't a single chatbot process. It's a set of local services that run continuously, share a workspace, react to file events, generate summaries, create embeddings, store vector memory, retrieve context, and periodically reason about the state of the workspace.
1415

15-
The core idea is: **AI systems are orchestration systems, not just inference systems.**
16+
The core idea is that AI systems are orchestration systems, not just inference systems.
1617

17-
DGX Spark is well suited to this type of workload because it combines Arm CPU orchestration with local GPU acceleration. In the [Grace Blackwell architecture](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/1_gb10_introduction/), the Arm Grace CPU coordinates background services, filesystem events, scheduling, document processing, metadata handling, and service-to-service communication. The Blackwell GPU accelerates local LLM inference, token generation, summarization, and embedding generation.
18+
DGX Spark is well suited to this type of workload because it combines Arm CPU orchestration with local GPU acceleration. In the [Grace Blackwell architecture](/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/1_gb10_introduction/), the Arm Grace CPU coordinates background services, filesystem events, scheduling, document processing, metadata handling, and service-to-service communication.
1819

19-
By the end of this Learning Path, you will have a local runtime with these capabilities:
20+
The Blackwell GPU accelerates local LLM inference, token generation, summarization, and embedding generation.
21+
22+
By the end of this Learning Path, you'll have a local runtime with these capabilities:
2023

2124
| Capability | Runtime component |
2225
|---|---|
@@ -27,7 +30,7 @@ By the end of this Learning Path, you will have a local runtime with these capab
2730
| Semantic retrieval | Hermes Agent + Qdrant + Ollama |
2831
| Autonomous workspace cognition | Hermes Agent + Ollama |
2932

30-
## Runtime architecture overview
33+
## Persistent AI runtime components
3134

3235
The runtime uses four containerized services:
3336

@@ -70,21 +73,19 @@ The important architectural pattern is separation of responsibilities. Each serv
7073
| Memory layer | Qdrant | Stores and searches vector memory |
7174
| Orchestration layer | Hermes Agent | Watches files, schedules work, coordinates services |
7275

73-
## Runtime components
74-
7576
### Hermes runtime
7677

77-
Hermes is the orchestration runtime you will build.
78+
Hermes is the orchestration runtime you'll build.
7879

79-
It runs as a persistent Python service inside a container. It watches the shared workspace, detects new files, reads documents, sends requests to Ollama, stores memory in Qdrant, performs semantic retrieval, and later generates autonomous workspace summaries. Hermes doesn't run the language model itself. It coordinates AI workflows across local services.
80+
Hermes runs as a persistent Python service inside a container. It watches the shared workspace, detects new files, and reads documents. It sends requests to Ollama, stores memory in Qdrant, performs semantic retrieval, and later generates autonomous workspace summaries. Hermes doesn't run the language model itself. Instead, it coordinates AI workflows across local services.
8081

81-
This is the main CPU-side workload in the system. The Arm CPU keeps the runtime alive, schedules background loops, tracks file events, moves data between services, and manages runtime state.
82+
The Hermes runtime is the main CPU-side workload in the system. The Arm CPU keeps the runtime alive, schedules background loops, tracks file events, moves data between services, and manages runtime state.
8283

8384
### Ollama runtime
8485

8586
Ollama provides the local inference runtime. It's a convenient way to run local models and expose a simple API, but the architecture isn't limited to Ollama.
8687

87-
Conceptually, Ollama is one possible inference backend. Hermes can orchestrate any local or remote inference service that exposes a compatible API, such as llama.cpp server, vLLM, a custom PyTorch service, or another model runtime.
88+
Conceptually, Ollama is one possible inference backend. Hermes can orchestrate any local or remote inference service that exposes a compatible API, such as `llama.cpp` server, vLLM, a custom PyTorch service, or another model runtime.
8889

8990
Hermes uses Ollama for two types of model calls:
9091

@@ -106,7 +107,7 @@ Open WebUI provides a local browser interface for interacting with the Ollama ru
106107

107108
Use it to confirm that local models are available, test prompts, or explore the inference runtime in a browser. The persistent AI runtime is still coordinated by Hermes.
108109

109-
## Shared workspace
110+
## Shared workspace structure
110111

111112
The services use a shared workspace mounted into the containers.
112113

@@ -169,7 +170,7 @@ This is different from storing plain text files and searching for keywords. Vect
169170

170171
## Autonomous workspace cognition
171172

172-
The final stage of this Learning Path adds autonomous workspace cognition.
173+
In the final stage of this Learning Path, you'll add autonomous workspace cognition.
173174

174175
Instead of responding only when a new file appears or when a query is submitted, Hermes periodically reviews the accumulated semantic memory and generates a workspace-level summary.
175176

@@ -185,7 +186,7 @@ Runtime behavior is controlled by a configuration file:
185186
/workspace/config/runtime.json
186187
```
187188

188-
This allows the runtime to adjust settings such as supported file extensions, retrieval depth, summary interval, and summary output path without hardcoding every behavior into the agent.
189+
This file allows the runtime to adjust settings such as supported file extensions, retrieval depth, summary interval, and summary output path without hardcoding every behavior into the agent.
189190

190191
## CPU and GPU responsibilities
191192

@@ -195,6 +196,8 @@ The Arm Grace CPU handles the persistent runtime side: watching the filesystem,
195196

196197
This separation is central to the architecture. The GPU accelerates model-heavy operations, while the CPU keeps the runtime organized and continuously operating.
197198

198-
## Next step
199+
## What you've learned and what's next
200+
201+
You've now learned about the persistent AI runtime architecture that you'll be building on NVIDIA DGX Spark. You've explored the different components and workflows of this architecture, and how the CPU and GPU perform different roles.
199202

200-
Next, you will build the DGX Spark runtime foundation: Docker, GPU-enabled containers, the shared workspace, and the initial Ollama, Qdrant, and Open WebUI services.
203+
Next, you'll build the DGX Spark runtime foundation: Docker, GPU-enabled containers, the shared workspace, and the initial Ollama, Qdrant, and Open WebUI services.

0 commit comments

Comments
 (0)