Skip to content

Commit d288659

Browse files
authored
docs: add agent runtime primer (#145)
#### Overview Adds a first-time Agent Runtime Primer so users can understand what NeMo Relay adds before running Quick Start. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Adds `docs/getting-started/agent-runtime-primer.md` to explain scopes, managed tool and LLM calls, middleware, events, subscribers/exporters, and plugins. - Clarifies that NeMo Relay does not replace agent frameworks, model providers, application logic, observability backends, or NeMo Agent Toolkit. - Links the page from the docs overview and Getting Started navigation. Validation: `just docs`, `just docs-linkcheck`, `git diff --check`. `uv run pre-commit run --all-files` was also run; all unrelated hooks passed. #### Where should the reviewer start? `docs/getting-started/agent-runtime-primer.md` #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - N/A ## Summary by CodeRabbit * **Documentation** * Added an Agent Runtime Primer detailing the agent runtime execution model: scopes, managed tool/LLM calls, middleware, lifecycle events, and plugins. * Clarifies what NeMo Relay does not replace and offers guidance on selecting instrumentation boundaries (application-owned, framework-owned, or reusable plugin behavior). * Updated docs navigation to include the new Getting Started entry for the primer. [![Review Change Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/NVIDIA/NeMo-Relay/pull/145?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) Authors: - https://github.com/mnajafian-nv Approvers: - Will Killian (https://github.com/willkill07) URL: #145
1 parent 7fbe239 commit d288659

2 files changed

Lines changed: 104 additions & 0 deletions

File tree

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
<!--
2+
SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
SPDX-License-Identifier: Apache-2.0
4+
-->
5+
6+
# Agent Runtime Primer
7+
8+
NeMo Relay is a portable runtime layer for agent systems that already have an
9+
application, framework, or model provider. Use this primer when you need to
10+
understand what NeMo Relay adds before running [Quick Start](quick-start.md).
11+
12+
Agent applications usually cross several boundaries in one request: an entry
13+
point starts work, the agent calls a model, the model asks for tools, tools call
14+
services, and tracing or policy systems need to understand the result. Without a
15+
shared runtime layer, each boundary tends to grow its own wrappers, callback
16+
shape, trace vocabulary, and cleanup rules.
17+
18+
NeMo Relay gives those boundaries one execution model.
19+
20+
## What NeMo Relay Adds
21+
22+
NeMo Relay does not decide what your agent should do. It describes and manages
23+
what happens when your agent crosses runtime boundaries.
24+
25+
The core runtime model has five parts:
26+
27+
- **Scopes** describe where work belongs. They preserve parent-child
28+
relationships across requests, agent runs, tools, LLM calls, background work,
29+
and nested functions.
30+
- **Managed tool and LLM calls** attach work to the active scope, run middleware
31+
in a consistent order, and emit lifecycle events. The application result is
32+
preserved unless registered intercepts or guardrails intentionally change the
33+
execution path.
34+
- **Middleware** runs around managed execution. Intercepts can transform or wrap
35+
real calls. Guardrails can block execution or sanitize emitted observability
36+
payloads.
37+
- **Events** record what happened. NeMo Relay emits Agent Trajectory
38+
Observability Format (ATOF) lifecycle records that subscribers and exporters
39+
can consume.
40+
- **Plugins** package reusable runtime behavior so teams can install middleware,
41+
subscribers, exporters, or adaptive behavior from configuration instead of
42+
repeating setup code in every application.
43+
44+
The simplest mental model is:
45+
46+
```text
47+
app or framework boundary
48+
-> NeMo Relay scope
49+
-> managed tool or LLM call
50+
-> middleware
51+
-> lifecycle event
52+
-> subscriber or exporter
53+
```
54+
55+
## What NeMo Relay Does Not Replace
56+
57+
NeMo Relay sits below the choices your application already makes.
58+
59+
It does not replace:
60+
61+
- your agent framework or orchestration logic
62+
- your model provider or provider SDK
63+
- your application business logic
64+
- your production observability backend
65+
- NeMo Agent Toolkit
66+
67+
Instead, it gives those systems a shared runtime contract for call boundaries,
68+
policy hooks, event emission, and export.
69+
70+
## Choose The Boundary You Own
71+
72+
Where you start depends on who owns the call boundary.
73+
74+
If your application directly calls tools or model providers, start by
75+
instrumenting the application boundary. Add scopes first, then wrap the tool and
76+
LLM calls your code owns.
77+
78+
If a framework owns scheduling, retries, callbacks, or provider payloads, use a
79+
framework integration. The integration should preserve framework behavior while
80+
adding NeMo Relay scopes, managed calls, codecs, middleware, and events at stable
81+
framework boundaries.
82+
83+
If you need the same behavior across multiple services or teams, package it as a
84+
plugin. Plugins are the configuration-driven path for reusable middleware,
85+
subscribers, exporters, and adaptive components.
86+
87+
## Read Next
88+
89+
The following pages help you choose the next step for your integration.
90+
91+
- Use [Quick Start](quick-start.md) for the smallest binding-specific example.
92+
- Use [Instrument Applications](../instrument-applications/about.md) when you
93+
own the tool or LLM call site.
94+
- Use [Integrate into Frameworks](../integrate-frameworks/about.md) when a
95+
framework owns invocation, scheduling, retries, callbacks, or provider
96+
payloads.
97+
- Use [Build Plugins](../build-plugins/about.md) when behavior should be
98+
reusable and activated from configuration.
99+
- Use [Observability](../plugins/observability/about.md) when you need to export
100+
runtime events to ATIF, OpenTelemetry, or OpenInference.
101+
- Use [Adaptive](../plugins/adaptive/about.md) after baseline instrumentation is
102+
working and you want to tune behavior from observed runtime signals.

docs/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ Use the reading path that matches your task:
5757

5858
| Task | Start With |
5959
|---|---|
60+
| Understand what NeMo Relay adds | [Agent Runtime Primer](getting-started/agent-runtime-primer.md) |
6061
| Run a minimal example | [Quick Start](getting-started/quick-start.md) |
6162
| Install packages | [Installation](getting-started/installation.md) |
6263
| Develop from source | [Development Setup](contribute/development-setup.md) |
@@ -143,6 +144,7 @@ about/release-notes/index
143144
:caption: Getting Started
144145
:maxdepth: 2
145146
147+
Agent Runtime Primer <getting-started/agent-runtime-primer>
146148
getting-started/prerequisites
147149
getting-started/installation
148150
Configuration / Setup <getting-started/configuration>

0 commit comments

Comments
 (0)