Skip to content

Commit 56397fe

Browse files
final pass
1 parent d35479c commit 56397fe

7 files changed

Lines changed: 50 additions & 58 deletions

File tree

content/learning-paths/laptops-and-desktops/dgx_persistent_agent/1_introduce.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Expplore persistent AI runtime architecture on NVIDIA DGX Spark
2+
title: Explore persistent AI runtime architecture on NVIDIA DGX Spark
33
description: Explore the persistent AI runtime architecture for NVIDIA DGX Spark, including Hermes Agent orchestration, Ollama inference, Qdrant memory, and CPU/GPU responsibilities.
44
weight: 2
55

content/learning-paths/laptops-and-desktops/dgx_persistent_agent/2_build.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ You'll see a message confirming that Docker is installed and working.
102102

103103
### Install NVIDIA Container Toolkit
104104

105-
The NVIDIA Container Toolkit enables Docker to expose the GPU to containers using the `--gpus` flag. Without it, containers can't access the GPU regardless of the driver version installed on the host.
105+
The NVIDIA Container Toolkit allows Docker to expose the GPU to containers using the `--gpus` flag. Without it, containers can't access the GPU regardless of the driver version installed on the host.
106106

107107
Add the NVIDIA Container Toolkit GPG key:
108108

@@ -148,7 +148,7 @@ nvcr.io/nvidia/cuda:13.0.1-devel-ubuntu24.04 \
148148
nvidia-smi
149149
```
150150

151-
If you have not pulled this image before, Docker downloads it before running `nvidia-smi`. This can take a few minutes depending on your network connection.
151+
If you have not pulled this image before, Docker downloads it before running `nvidia-smi`. The download can take a few minutes depending on your network connection.
152152

153153
The output is similar to:
154154

@@ -354,7 +354,7 @@ docker compose up -d
354354
```
355355

356356
{{% notice Note %}}
357-
The first `docker compose up -d` run can take several minutes because Docker needs to pull the service images. The time depends on your network speed.
357+
The first `docker compose up -d` run can take several minutes, depending on your network speed, because Docker needs to pull the service images.
358358
{{% /notice %}}
359359

360360
Verify that the containers are running:
@@ -435,13 +435,13 @@ exit
435435

436436
These model names are used throughout the examples, so make note of them. The architecture supports other suitable models.
437437

438-
Because the `models/` directory is mounted into the container as a volume, downloaded models are stored on the host at `~/dgx-hermes-agent/models/` and persist even if the container is removed or recreated.
439-
440438
| Model | Purpose |
441439
|---|---|
442440
| `qwen2.5:7b` | Local chat, summarization, reasoning |
443441
| `nomic-embed-text` | Embedding generation for semantic memory |
444442

443+
Because the `models/` directory is mounted into the container as a volume, downloaded models are stored on the host at `~/dgx-hermes-agent/models/`. The models persist even if the container is removed or recreated.
444+
445445
### Verify local inference
446446

447447
Open a shell in the Ollama container:

content/learning-paths/laptops-and-desktops/dgx_persistent_agent/3_deploy_orch_runtime.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,7 @@ layout: "learningpathall"
99

1010
In this section, you'll add Hermes Agent to the runtime stack.
1111

12-
Hermes Agent acts as the orchestration layer for the local AI runtime. It watches the workspace, detects runtime events, and coordinates the next action without requiring a user to manually run each step.
13-
14-
Hermes is the CPU-side orchestration runtime. It runs continuously, watches the shared workspace, and reacts when new files are created. This is the first step toward a persistent local AI agent.
12+
Hermes Agent acts as the CPU-side orchestration layer for the local AI runtime. It watches the workspace, detects runtime events, and coordinates the next action without requiring a user to manually run each step. This is the first step toward a persistent local AI agent.
1513

1614
In this section, Hermes doesn't call a language model yet. You'll first build the event-driven runtime foundation, where a new file in `workspace/inbox/` triggers a filesystem event, Hermes handles it, and prints a content preview to the logs.
1715

@@ -252,7 +250,7 @@ mv /tmp/runtime-test.txt \
252250
```
253251
Doing this avoids triggering the filesystem event before the file content has finished writing.
254252

255-
Return to the terminal that is following Hermes logs.
253+
Return to the terminal that's following Hermes logs.
256254

257255
The output is similar to:
258256

@@ -278,7 +276,9 @@ This shared mount is what allows the host, Hermes, Ollama, and later memory work
278276

279277
## Understand CPU orchestration responsibilities
280278

281-
This section demonstrates the CPU-side work required by persistent AI systems. The Arm CPU runs the long-lived service process, monitors filesystem events, schedules runtime activity, processes files, and manages the containerized service lifecycle. The GPU becomes important when model inference is added, but the persistent runtime is coordinated entirely by the CPU.
279+
This section shows the CPU-side work required by persistent AI systems. The Arm CPU runs the long-lived service process and monitors filesystem events. It schedules runtime activity, processes files, and manages the containerized service lifecycle.
280+
281+
The GPU becomes important when model inference is added, but the persistent runtime is coordinated entirely by the CPU.
282282

283283
## What you've accomplished and what's next
284284

@@ -288,4 +288,4 @@ You also verified that creating a new file in `workspace/inbox/` triggers Hermes
288288

289289
Next, you'll connect Hermes to Ollama for local LLM summarization.
290290

291-
Before moving to the next section, press `Ctrl+C` in terminal 1 to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.
291+
Before moving to the next section, press `Ctrl+C` in the first terminal running Hermes logs to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.

content/learning-paths/laptops-and-desktops/dgx_persistent_agent/4_local_llm.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ workspace/inbox document
2222
-> Local LLM summary
2323
```
2424

25-
This introduces the first GPU-accelerated step in the persistent runtime.
25+
Connecting Hermes Agent to Ollama introduces the first GPU-accelerated step in the persistent runtime.
2626

2727
### Configure Ollama runtime access
2828

@@ -215,9 +215,9 @@ mv /tmp/ai-runtime-note.txt \
215215
~/dgx-hermes-agent/workspace/inbox/ai-runtime-note.txt
216216
```
217217

218-
Return to terminal 1 to see the Hermes log output.
218+
Return to the first terminal running Hermes logs to see the Hermes log output.
219219

220-
The output similar to:
220+
The output is similar to:
221221

222222
```output
223223
[Agent] New file detected:
@@ -252,13 +252,15 @@ mv /tmp/gpu-inference-test.txt \
252252
nvtop
253253
```
254254

255-
The background command creates the file after five seconds, giving `nvtop` time to start before Ollama begins inference. During summarization, `nvtop` shows GPU activity from the Ollama model runtime. Watch the terminal running Hermes logs to see the Hermes log output as inference runs.
255+
The background command creates the file after five seconds, giving `nvtop` time to start before Ollama begins inference. During summarization, `nvtop` shows GPU activity from the Ollama model runtime. Watch the first terminal running Hermes logs to see the Hermes log output as inference runs.
256256

257257
Press `q` to quit `nvtop` after reviewing the GPU activity.
258258

259259
## Understand CPU and GPU responsibilities
260260

261-
The Arm Grace CPU coordinates the full workflow: watching the workspace, handling filesystem events, reading files, preparing model requests, and sending API calls to Ollama. The Blackwell GPU accelerates the model workload, running LLM inference, generating tokens, and producing the summary. This pattern repeats throughout the Learning Path. Hermes orchestrates and Ollama executes.
261+
The Arm Grace CPU coordinates the full workflow: watching the workspace, handling filesystem events, reading files, preparing model requests, and sending API calls to Ollama.
262+
263+
The Blackwell GPU accelerates the model workload, running LLM inference, generating tokens, and producing the summary. This pattern repeats throughout the Learning Path. Hermes orchestrates and Ollama executes.
262264

263265
## What you've accomplished and what's next
264266

@@ -268,4 +270,4 @@ The runtime has moved from file detection to event-driven AI summarization.
268270

269271
Next, you'll add persistent semantic memory with embeddings and Qdrant.
270272

271-
Before moving to the next section, press `Ctrl+C` in the terminal running Hermes logs to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.
273+
Before moving to the next section, press `Ctrl+C` in the first terminal running Hermes logs to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.

content/learning-paths/laptops-and-desktops/dgx_persistent_agent/5_persistent_memory.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ workspace/inbox document
2323
-> persistent semantic memory
2424
```
2525

26-
This turns Hermes from an event-driven summarizer into a local AI runtime with long-term memory.
26+
Adding persistent memory turns Hermes from an event-driven summarizer into a local AI runtime with long-term memory.
2727

2828
### Understand persistent memory architecture
2929

@@ -237,7 +237,7 @@ if __name__ == "__main__":
237237

238238
#### Understand the code trace
239239

240-
Hermes connects to Qdrant using the Docker service name `qdrant`, read from the `QDRANT_HOST` environment variable with a default fallback. On startup, `ensure_collection()` creates the `workspace_memory` collection if it does not already exist, configured with a vector size of 768 and cosine distance to match the `nomic-embed-text` model output.
240+
Hermes connects to Qdrant using the Docker service name `qdrant`, read from the `QDRANT_HOST` environment variable with a default fallback. On startup, `ensure_collection()` creates the `workspace_memory` collection if it doesn't already exist, configured with a vector size of 768 and cosine distance to match the `nomic-embed-text` model output.
241241

242242
When a file is detected, `process_file()` calls `generate_summary()` and then `generate_embedding()` in sequence. The embedding uses `client.embed()` from the current Ollama Python SDK, and the result is read from `response["embeddings"][0]`. Hermes stores each document as a Qdrant `PointStruct` with the vector and a payload containing the file path, generated summary, and source content excerpt. The payload makes sure that future retrieval results include the document context needed to answer workspace queries.
243243

@@ -276,7 +276,7 @@ vectors_config=VectorParams(
276276
)
277277
```
278278

279-
If you change the embedding model later, update the Qdrant collection dimension to match the new model output. If the dimensions do not match, Qdrant will reject the inserted vectors.
279+
If you change the embedding model later, update the Qdrant collection dimension to match the new model output. If the dimensions don't match, Qdrant will reject the inserted vectors.
280280

281281
### Rebuild Hermes
282282

@@ -327,7 +327,7 @@ mv /tmp/memory-test.txt \
327327
~/dgx-hermes-agent/workspace/inbox/memory-test.txt
328328
```
329329

330-
Watch the terminal running Hermes logs for the Hermes log output.
330+
Watch the first terminal running Hermes logs for the Hermes log output.
331331

332332
The memory ingestion output is similar to:
333333

@@ -361,9 +361,9 @@ http://localhost:6333/dashboard
361361

362362
Confirm that the `workspace_memory` collection exists:
363363

364-
![Qdrant dashboard showing the workspace_memory collection#center](qdrant_dashboard_2.png "Qdrant Dashboard")
364+
![Qdrant dashboard showing the workspace_memory collection, confirming that Hermes created the collection#center](qdrant_dashboard_2.png "Qdrant Dashboard")
365365

366-
The dashboard shows the `workspace_memory` collection after Hermes starts and runs `ensure_collection()`. If the collection does not appear, check the Hermes logs for Qdrant connection errors and confirm that the `qdrant` container is running.
366+
The dashboard shows the `workspace_memory` collection after Hermes starts and runs `ensure_collection()`. If the collection doesn't appear, check the Hermes logs for Qdrant connection errors and confirm that the `qdrant` container is running.
367367

368368
Open the collection and verify that points are being stored. Each point represents one ingested workspace document and contains:
369369

@@ -372,13 +372,13 @@ Open the collection and verify that points are being stored. Each point represen
372372
- A `summary` payload field
373373
- A `content` payload field
374374

375-
![Qdrant workspace_memory collection showing stored vectors and payload fields#center](qdrant_dashboard_3.png "Qdrant workspace_memory")
375+
![Qdrant workspace_memory collection showing stored vectors and payload fields#center](qdrant_dashboard_3.png "Qdrant workspace_memory vector and payload view")
376376

377377
Use this view to confirm that Qdrant has stored both the vector and payload metadata. The payload fields are important because later retrieval steps need the path and summary to assemble useful context for the LLM.
378378

379379
You can also inspect collection storage and memory usage:
380380

381-
![Qdrant collection storage view showing persistent memory usage#center](qdrant_dashboard_4.png "Qdrant workspace_memory")
381+
![Qdrant collection storage view showing persistent memory usage#center](qdrant_dashboard_4.png "Qdrant workspace_memory memory usage view")
382382

383383
The memory usage view confirms that Qdrant is maintaining persistent collection state on disk. This matters because the vector memory survives container restarts as long as the `../qdrant:/qdrant/storage` volume remains mounted.
384384

@@ -388,7 +388,7 @@ You can also inspect collections from the host:
388388
curl http://localhost:6333/collections
389389
```
390390

391-
The response is similar to:
391+
The output is similar to:
392392

393393
```output
394394
{"result":{"collections":[{"name":"workspace_memory"}]},"status":"ok","time":0.0001}
@@ -408,4 +408,4 @@ The runtime can now ingest documents, summarize them, generate embeddings, and p
408408

409409
Next, you'll add semantic retrieval and contextual question answering.
410410

411-
Before moving to the next section, press `Ctrl+C` in terminal 1 to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.
411+
Before moving to the next section, press `Ctrl+C` in the first terminal running Hermes logs to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.

content/learning-paths/laptops-and-desktops/dgx_persistent_agent/6_semantic_retrieval.md

Lines changed: 13 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,7 @@ The main runtime loop checks for `/workspace/query.txt` on each iteration. When
276276

277277
#### Understand runtime compatibility considerations
278278

279-
Use `qdrant.query_points()` with `query=embedding` when calling the vector search API. Older examples use `query_vector=embedding`, which is not compatible with current versions of the Qdrant Python client. Read each result's payload with `result.payload` before assembling the retrieval context.
279+
Use `qdrant.query_points()` with `query=embedding` when calling the vector search API. Older examples use `query_vector=embedding`, which isn't compatible with current versions of the Qdrant Python client. Read each result's payload with `result.payload` before assembling the retrieval context.
280280

281281
The `limit=3` value in `search_memory()` is hardcoded to keep the retrieval behavior easy to inspect. You can make it configurable later.
282282

@@ -358,7 +358,7 @@ mv /tmp/semantic-memory-note.txt \
358358
~/dgx-hermes-agent/workspace/inbox/semantic-memory-note.txt
359359
```
360360

361-
Watch the terminal running Hermes logs until each document is summarized, embedded, and stored.
361+
Watch the first terminal running Hermes logs until each document is summarized, embedded, and stored.
362362

363363
The output is similar to:
364364

@@ -405,7 +405,7 @@ The output is similar to:
405405

406406
### Test semantic retrieval
407407

408-
In terminal 2, create a query file:
408+
In the second terminal, create a query file:
409409

410410
```bash
411411
echo "How do CPUs help persistent AI systems?" \
@@ -414,13 +414,17 @@ echo "How do CPUs help persistent AI systems?" \
414414

415415
Hermes checks for `/workspace/query.txt` in the runtime loop. When it sees the file, it reads the question and removes the file. It then embeds the question, searches Qdrant, and sends the retrieved context to Ollama.
416416

417-
In terminal 1, confirm that semantic search started:
417+
In the first terminal running Hermes logs, confirm that semantic search started.
418+
419+
The output is similar to:
418420

419421
```output
420422
[Memory] Searching semantic memory...
421423
```
422424

423-
Then confirm that Hermes printed the question and the retrieved memory context:
425+
Then confirm that Hermes printed the question and the retrieved memory context.
426+
427+
The output is similar to:
424428

425429
```output
426430
[Workspace Query]
@@ -469,7 +473,7 @@ The exact answer will vary, but it should refer to retrieved memory about CPU or
469473

470474
### Verify contextual reasoning
471475

472-
In terminal 2, ask a second question:
476+
In the second terminal, ask another question:
473477

474478
```bash
475479
echo "Why does the runtime need semantic memory?" \
@@ -511,28 +515,14 @@ Summary:
511515
The runtime needs semantic memory because it retains embeddings and metadata that store relevant context from past workspace activities. This allows the system to effectively reason by retrieving pertinent information, enhancing its ability to understand and respond to new inputs more intelligently. Unlike simple keyword matching, semantic memory uses a vector database which captures the meaning of words or concepts, enabling more accurate and contextualized recall of past events or knowledge.
512516
```
513517

514-
## Understand the retrieval workflow
515-
516-
The full retrieval workflow is:
517-
518-
```output
519-
query.txt question
520-
-> Ollama query embedding
521-
-> Qdrant workspace_memory search
522-
-> retrieved summaries
523-
-> Hermes context prompt
524-
-> Ollama contextual response
525-
-> Hermes log output
526-
```
527-
528-
This creates a local contextual reasoning loop using persistent memory.
529-
530518
## Understand CPU and GPU responsibilities
531519

532520
The Arm Grace CPU coordinates the retrieval workflow: watching for `query.txt`, reading and deleting the query file, calling Ollama for the query embedding, searching Qdrant for relevant memories, parsing result payloads, assembling the retrieved context, and calling Ollama again for contextual reasoning.
533521

534522
The Blackwell GPU accelerates query embedding generation, contextual LLM inference, and response generation. Qdrant performs the vector similarity search and returns the most relevant memory payloads.
535523

524+
This retrieval workflow creates a local contextual reasoning loop using persistent memory.
525+
536526
## What you've accomplished and what's next
537527

538528
You've added semantic retrieval and contextual reasoning to Hermes Agent. The runtime now turns a question into an embedding, searches Qdrant with `query_points(...)`, assembles retrieved memory, and sends that context to `qwen2.5:7b`.
@@ -541,4 +531,4 @@ The runtime can now store memory and reason over it through the local `/workspac
541531

542532
Next, you'll add autonomous workspace cognition.
543533

544-
Before moving to the next section, press `Ctrl+C` in terminal 1 to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.
534+
Before moving to the next section, press `Ctrl+C` in the first terminal running Hermes logs to stop the Hermes log stream. In the next section, you'll rebuild the Hermes container and run `docker logs -f hermes` again.

0 commit comments

Comments
 (0)