Skip to content

Commit 30245c3

Browse files
authored
feat: add NeMo Guardrails example plugin (NVIDIA#61)
## Summary - Add an external Python NeMo Guardrails example plugin for NeMo Flow LLM and tool calls. - Add runnable example assets and native Guardrails config examples under `examples/nemoguardrails`. - Document configuration, runtime behavior, supported codecs, and limitations. - Add focused pytest coverage with stubbed `nemoguardrails` modules so CI does not need the optional dependency. Closes NVIDIA#26 Closes NMF-16 ## Testing - `uv run pytest python/tests/test_nemoguardrails_example_plugin.py` - `uv run pre-commit run --files docs/build-plugins/about.md docs/build-plugins/nemoguardrails.md docs/index.md examples/nemoguardrails/README.md examples/nemoguardrails/example/agent_example.py examples/nemoguardrails/example/example_config.yml examples/nemoguardrails/example/plugin.py examples/nemoguardrails/example/rails/config.yml python/tests/test_nemoguardrails_example_plugin.py` ## Summary by CodeRabbit * **New Features** * Added a NeMo Guardrails example plugin with a runnable agent demo, CLI modes for guardrails source, example tools, and a sample configuration for input/output safety checks. * **Documentation** * New guide and README documenting plugin usage, configuration options, example workflows, and run instructions; index updated. * **Tests** * Added comprehensive tests covering plugin validation and runtime behaviors (pass/block/modify scenarios). Authors: - https://github.com/afourniernv Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#61
1 parent 349a5bc commit 30245c3

9 files changed

Lines changed: 1898 additions & 0 deletions

File tree

docs/build-plugins/about.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ Use these guide links to move from the overview into task-specific instructions.
3535
- [Basic Guide: Validate Plugin Configuration](validate-configuration.md) covers JSON-compatible config, validation rules, and structured diagnostics.
3636
- [Basic Guide: Register Plugin Behavior](register-behavior.md) shows how to initialize config and install subscribers or middleware through `PluginContext`.
3737
- [Advanced Guide: Design Plugin Configuration](advanced-configuration.md) covers validation rules, advanced configuration patterns, rollout controls, and `PluginContext` usage.
38+
- [NeMo Guardrails Example Plugin](nemoguardrails.md) shows an external Python plugin that applies NeMo Guardrails checks around NeMo Flow LLM and tool calls.
3839
- [Code Examples](code-examples.md) provides patterns for dynamic header injection, subscriber-oriented export, multi-surface bundles, and framework-facing plugins.
3940

4041
Start by deciding which runtime surfaces the plugin owns: middleware,
Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
<!--
2+
SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
SPDX-License-Identifier: Apache-2.0
4+
-->
5+
6+
# NeMo Guardrails Example Plugin
7+
8+
This example shows how to write a Python NeMo Flow plugin that calls the NeMo
9+
Guardrails Python API.
10+
11+
The example lives under `examples/nemoguardrails`. The single-file plugin
12+
implementation, runnable agent, and Guardrails config artifacts are under
13+
`example`.
14+
It is not part of the
15+
`nemo_flow` Python package, and NeMo Flow does not depend on `nemoguardrails`.
16+
Applications that use the example install NeMo Guardrails in their own
17+
environment and import or vendor the example plugin.
18+
19+
## Install
20+
21+
Install NeMo Flow normally, then install NeMo Guardrails in the application or
22+
example environment that activates the plugin:
23+
24+
```bash
25+
pip install nemoguardrails
26+
```
27+
28+
The bundled example config uses NeMo Guardrails' `nvidia_ai_endpoints` model
29+
engine. Install the NVIDIA LangChain provider when you want to run that config
30+
as-is:
31+
32+
```bash
33+
pip install langchain-nvidia-ai-endpoints
34+
```
35+
36+
## Configure
37+
38+
Guardrails stay in native NeMo Guardrails config. Point the plugin at a
39+
Guardrails config directory, or pass inline YAML content.
40+
41+
```python
42+
import asyncio
43+
44+
import nemo_flow
45+
import plugin as nemoguardrails_plugin
46+
47+
48+
async def main() -> None:
49+
nemoguardrails_plugin.register()
50+
try:
51+
config = nemo_flow.plugin.PluginConfig(
52+
components=[
53+
nemo_flow.plugin.ComponentSpec(
54+
kind=nemoguardrails_plugin.DEFAULT_KIND,
55+
config={
56+
"config_path": "./rails",
57+
"codec": "openai_chat",
58+
},
59+
)
60+
]
61+
)
62+
await nemo_flow.plugin.initialize(config)
63+
finally:
64+
nemo_flow.plugin.clear()
65+
nemoguardrails_plugin.deregister()
66+
67+
68+
asyncio.run(main())
69+
```
70+
71+
The `config_path` directory is a normal NeMo Guardrails config directory. For
72+
example:
73+
74+
```yaml
75+
# rails/config.yml
76+
models:
77+
- type: main
78+
engine: nvidia_ai_endpoints
79+
model: meta/llama-3.1-8b-instruct
80+
81+
rails:
82+
input:
83+
flows:
84+
- self check input
85+
output:
86+
flows:
87+
- self check output
88+
89+
prompts:
90+
- task: self_check_input
91+
content: |-
92+
You are checking whether a NeMo Flow request should be allowed.
93+
The input may be plain user text or a JSON object with tool_name and
94+
arguments fields.
95+
User input: {{ user_input }}
96+
Should this request be blocked? Answer only Yes or No.
97+
98+
- task: self_check_output
99+
content: |-
100+
You are checking whether a NeMo Flow response should be returned.
101+
The output may be assistant text or a JSON object with tool_name,
102+
arguments, and result fields.
103+
Model output: {{ bot_response }}
104+
Should this response be blocked? Answer only Yes or No.
105+
```
106+
107+
The plugin config accepts these fields:
108+
109+
- `config_path`: Path to a NeMo Guardrails config directory.
110+
- `config_yaml`: Inline NeMo Guardrails YAML config.
111+
- `colang_content`: Optional inline Colang content. This can only be used with
112+
`config_yaml`.
113+
- `codec`: One of `openai_chat`, `openai_responses`, or
114+
`anthropic_messages`. This is required when `input` or `output` is enabled.
115+
- `input`: Whether to run input rails around LLM calls. Defaults to `true`.
116+
- `output`: Whether to run output rails around LLM calls. Defaults to `true`.
117+
- `tool_input`: Whether to check managed tool arguments before execution.
118+
Defaults to `false`.
119+
- `tool_output`: Whether to check managed tool results after execution.
120+
Defaults to `false`.
121+
- `priority`: Execution-intercept priority. Defaults to `100`.
122+
123+
Exactly one of `config_path` or `config_yaml` is required.
124+
125+
## Example Agent
126+
127+
The example includes
128+
[`agent_example.py`](../../examples/nemoguardrails/example/agent_example.py), a
129+
concrete example agent that initializes the plugin, checks a managed
130+
`tools.execute(...)` call, and checks a managed `llm.execute(...)` call against
131+
live NVIDIA-hosted inference.
132+
133+
Run it from a checkout where NeMo Flow and NeMo Guardrails are installed. The
134+
default lane uses a passthrough Guardrails config and the `current_time` tool.
135+
This is the fastest live validation path because it exercises the real plugin,
136+
real `nemoguardrails` initialization, tool execution, and LLM execution without
137+
running model-backed self-check rails:
138+
139+
```bash
140+
export NVIDIA_API_KEY="<your-key>"
141+
python examples/nemoguardrails/example/agent_example.py
142+
```
143+
144+
To run the inline self-check rails example, load
145+
[`example_config.yml`](../../examples/nemoguardrails/example/example_config.yml)
146+
from `example` and pass it as inline `config_yaml`:
147+
148+
```bash
149+
python examples/nemoguardrails/example/agent_example.py --guardrails-config inline
150+
```
151+
152+
The config directory lane uses the bundled
153+
`examples/nemoguardrails/example/rails/config.yml` by default. It
154+
contains the same input and output self-check rails as `example/example_config.yml`:
155+
156+
```bash
157+
python examples/nemoguardrails/example/agent_example.py --guardrails-config path
158+
```
159+
160+
Use `--tool weather` when you want the example to use a weather tool instead
161+
of the default `current_time` tool:
162+
163+
```bash
164+
python examples/nemoguardrails/example/agent_example.py --tool weather
165+
```
166+
167+
Pass `--config-path` when you want the example agent to use your own native
168+
NeMo Guardrails config directory:
169+
170+
```bash
171+
python examples/nemoguardrails/example/agent_example.py \
172+
--guardrails-config path \
173+
--config-path ./rails
174+
```
175+
176+
## Runtime Behavior
177+
178+
For non-streaming `llm.execute(...)` calls, the plugin checks the user input
179+
before the model call and checks the assistant text after the model call.
180+
Guardrails can pass, block, or rewrite input. For output, this example supports
181+
pass and block; modified output raises because NeMo Flow response codecs are
182+
decode-only and the example does not rewrite provider-shaped responses.
183+
184+
For managed `tools.execute(...)` calls, the plugin can also check serialized
185+
tool arguments before execution and serialized tool results after execution.
186+
When Guardrails rewrites tool arguments or results, the rewritten content must
187+
be valid JSON.
188+
189+
The bundled config uses the same NeMo Guardrails input and output self-check
190+
rails for both LLM messages and tool payloads. The plugin makes tool calls
191+
visible to Guardrails by serializing managed tool arguments and results as JSON
192+
message content.
193+
194+
This behavior changes the real execution path. It is not an observability-only
195+
sanitize guardrail.
196+
197+
## Supported Codecs
198+
199+
The example is intentionally limited to NeMo Flow's built-in LLM codec shapes:
200+
201+
- `openai_chat` for OpenAI Chat Completions-style requests and responses.
202+
- `openai_responses` for OpenAI Responses API-style requests and responses.
203+
- `anthropic_messages` for Anthropic Messages-style requests and responses.
204+
205+
Provider-specific payloads outside those codecs need a NeMo Flow codec and a
206+
response text replacement strategy before a production plugin can apply
207+
modified output safely.
208+
209+
## Limitations
210+
211+
This example calls NeMo Guardrails `check_async`, not `generate_async`. It
212+
checks around NeMo Flow LLM and tool execution calls, but it does not let NeMo
213+
Guardrails take over generation or agent orchestration.
214+
215+
The example does not support:
216+
217+
- Streaming LLM calls.
218+
- Dialog rails, retrieval rails, execution rails, or generation rails that
219+
require NeMo Guardrails to orchestrate the full generation flow.
220+
- Arbitrary provider payloads beyond the three supported NeMo Flow codecs.
221+
- Applying modified LLM output back into provider responses.
222+
- Rewriting tool-call arguments inside model responses before an application
223+
turns those model tool calls into managed `tools.execute(...)` calls.
224+
225+
Tool checks use serialized JSON and NeMo Guardrails input/output checks. They
226+
are NeMo Flow tool middleware checks powered by Guardrails, not a full
227+
`generate_async` agent-loop integration.
228+
229+
`config_path` points at native NeMo Guardrails configuration. Guardrails config
230+
can load project code such as actions, so treat that path as trusted
231+
application code.

docs/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,7 @@ Basic Guide: Define a Plugin <build-plugins/basic-guide>
178178
Basic Guide: Validate Plugin Configuration <build-plugins/validate-configuration>
179179
Basic Guide: Register Plugin Behavior <build-plugins/register-behavior>
180180
Advanced Guide: Design Plugin Configuration <build-plugins/advanced-configuration>
181+
NeMo Guardrails Example Plugin <build-plugins/nemoguardrails>
181182
Code Examples <build-plugins/code-examples>
182183
```
183184

0 commit comments

Comments
 (0)