|
| 1 | +<!-- |
| 2 | +SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. |
| 3 | +SPDX-License-Identifier: Apache-2.0 |
| 4 | +--> |
| 5 | + |
| 6 | +# NeMo Guardrails Example Plugin |
| 7 | + |
| 8 | +This example shows how to write a Python NeMo Flow plugin that calls the NeMo |
| 9 | +Guardrails Python API. |
| 10 | + |
| 11 | +The example lives under `examples/nemoguardrails`. The single-file plugin |
| 12 | +implementation, runnable agent, and Guardrails config artifacts are under |
| 13 | +`example`. |
| 14 | +It is not part of the |
| 15 | +`nemo_flow` Python package, and NeMo Flow does not depend on `nemoguardrails`. |
| 16 | +Applications that use the example install NeMo Guardrails in their own |
| 17 | +environment and import or vendor the example plugin. |
| 18 | + |
| 19 | +## Install |
| 20 | + |
| 21 | +Install NeMo Flow normally, then install NeMo Guardrails in the application or |
| 22 | +example environment that activates the plugin: |
| 23 | + |
| 24 | +```bash |
| 25 | +pip install nemoguardrails |
| 26 | +``` |
| 27 | + |
| 28 | +The bundled example config uses NeMo Guardrails' `nvidia_ai_endpoints` model |
| 29 | +engine. Install the NVIDIA LangChain provider when you want to run that config |
| 30 | +as-is: |
| 31 | + |
| 32 | +```bash |
| 33 | +pip install langchain-nvidia-ai-endpoints |
| 34 | +``` |
| 35 | + |
| 36 | +## Configure |
| 37 | + |
| 38 | +Guardrails stay in native NeMo Guardrails config. Point the plugin at a |
| 39 | +Guardrails config directory, or pass inline YAML content. |
| 40 | + |
| 41 | +```python |
| 42 | +import asyncio |
| 43 | + |
| 44 | +import nemo_flow |
| 45 | +import plugin as nemoguardrails_plugin |
| 46 | + |
| 47 | + |
| 48 | +async def main() -> None: |
| 49 | + nemoguardrails_plugin.register() |
| 50 | + try: |
| 51 | + config = nemo_flow.plugin.PluginConfig( |
| 52 | + components=[ |
| 53 | + nemo_flow.plugin.ComponentSpec( |
| 54 | + kind=nemoguardrails_plugin.DEFAULT_KIND, |
| 55 | + config={ |
| 56 | + "config_path": "./rails", |
| 57 | + "codec": "openai_chat", |
| 58 | + }, |
| 59 | + ) |
| 60 | + ] |
| 61 | + ) |
| 62 | + await nemo_flow.plugin.initialize(config) |
| 63 | + finally: |
| 64 | + nemo_flow.plugin.clear() |
| 65 | + nemoguardrails_plugin.deregister() |
| 66 | + |
| 67 | + |
| 68 | +asyncio.run(main()) |
| 69 | +``` |
| 70 | + |
| 71 | +The `config_path` directory is a normal NeMo Guardrails config directory. For |
| 72 | +example: |
| 73 | + |
| 74 | +```yaml |
| 75 | +# rails/config.yml |
| 76 | +models: |
| 77 | + - type: main |
| 78 | + engine: nvidia_ai_endpoints |
| 79 | + model: meta/llama-3.1-8b-instruct |
| 80 | + |
| 81 | +rails: |
| 82 | + input: |
| 83 | + flows: |
| 84 | + - self check input |
| 85 | + output: |
| 86 | + flows: |
| 87 | + - self check output |
| 88 | + |
| 89 | +prompts: |
| 90 | + - task: self_check_input |
| 91 | + content: |- |
| 92 | + You are checking whether a NeMo Flow request should be allowed. |
| 93 | + The input may be plain user text or a JSON object with tool_name and |
| 94 | + arguments fields. |
| 95 | + User input: {{ user_input }} |
| 96 | + Should this request be blocked? Answer only Yes or No. |
| 97 | +
|
| 98 | + - task: self_check_output |
| 99 | + content: |- |
| 100 | + You are checking whether a NeMo Flow response should be returned. |
| 101 | + The output may be assistant text or a JSON object with tool_name, |
| 102 | + arguments, and result fields. |
| 103 | + Model output: {{ bot_response }} |
| 104 | + Should this response be blocked? Answer only Yes or No. |
| 105 | +``` |
| 106 | +
|
| 107 | +The plugin config accepts these fields: |
| 108 | +
|
| 109 | +- `config_path`: Path to a NeMo Guardrails config directory. |
| 110 | +- `config_yaml`: Inline NeMo Guardrails YAML config. |
| 111 | +- `colang_content`: Optional inline Colang content. This can only be used with |
| 112 | + `config_yaml`. |
| 113 | +- `codec`: One of `openai_chat`, `openai_responses`, or |
| 114 | + `anthropic_messages`. This is required when `input` or `output` is enabled. |
| 115 | +- `input`: Whether to run input rails around LLM calls. Defaults to `true`. |
| 116 | +- `output`: Whether to run output rails around LLM calls. Defaults to `true`. |
| 117 | +- `tool_input`: Whether to check managed tool arguments before execution. |
| 118 | + Defaults to `false`. |
| 119 | +- `tool_output`: Whether to check managed tool results after execution. |
| 120 | + Defaults to `false`. |
| 121 | +- `priority`: Execution-intercept priority. Defaults to `100`. |
| 122 | + |
| 123 | +Exactly one of `config_path` or `config_yaml` is required. |
| 124 | + |
| 125 | +## Example Agent |
| 126 | + |
| 127 | +The example includes |
| 128 | +[`agent_example.py`](../../examples/nemoguardrails/example/agent_example.py), a |
| 129 | +concrete example agent that initializes the plugin, checks a managed |
| 130 | +`tools.execute(...)` call, and checks a managed `llm.execute(...)` call against |
| 131 | +live NVIDIA-hosted inference. |
| 132 | + |
| 133 | +Run it from a checkout where NeMo Flow and NeMo Guardrails are installed. The |
| 134 | +default lane uses a passthrough Guardrails config and the `current_time` tool. |
| 135 | +This is the fastest live validation path because it exercises the real plugin, |
| 136 | +real `nemoguardrails` initialization, tool execution, and LLM execution without |
| 137 | +running model-backed self-check rails: |
| 138 | + |
| 139 | +```bash |
| 140 | +export NVIDIA_API_KEY="<your-key>" |
| 141 | +python examples/nemoguardrails/example/agent_example.py |
| 142 | +``` |
| 143 | + |
| 144 | +To run the inline self-check rails example, load |
| 145 | +[`example_config.yml`](../../examples/nemoguardrails/example/example_config.yml) |
| 146 | +from `example` and pass it as inline `config_yaml`: |
| 147 | + |
| 148 | +```bash |
| 149 | +python examples/nemoguardrails/example/agent_example.py --guardrails-config inline |
| 150 | +``` |
| 151 | + |
| 152 | +The config directory lane uses the bundled |
| 153 | +`examples/nemoguardrails/example/rails/config.yml` by default. It |
| 154 | +contains the same input and output self-check rails as `example/example_config.yml`: |
| 155 | + |
| 156 | +```bash |
| 157 | +python examples/nemoguardrails/example/agent_example.py --guardrails-config path |
| 158 | +``` |
| 159 | + |
| 160 | +Use `--tool weather` when you want the example to use a weather tool instead |
| 161 | +of the default `current_time` tool: |
| 162 | + |
| 163 | +```bash |
| 164 | +python examples/nemoguardrails/example/agent_example.py --tool weather |
| 165 | +``` |
| 166 | + |
| 167 | +Pass `--config-path` when you want the example agent to use your own native |
| 168 | +NeMo Guardrails config directory: |
| 169 | + |
| 170 | +```bash |
| 171 | +python examples/nemoguardrails/example/agent_example.py \ |
| 172 | + --guardrails-config path \ |
| 173 | + --config-path ./rails |
| 174 | +``` |
| 175 | + |
| 176 | +## Runtime Behavior |
| 177 | + |
| 178 | +For non-streaming `llm.execute(...)` calls, the plugin checks the user input |
| 179 | +before the model call and checks the assistant text after the model call. |
| 180 | +Guardrails can pass, block, or rewrite input. For output, this example supports |
| 181 | +pass and block; modified output raises because NeMo Flow response codecs are |
| 182 | +decode-only and the example does not rewrite provider-shaped responses. |
| 183 | + |
| 184 | +For managed `tools.execute(...)` calls, the plugin can also check serialized |
| 185 | +tool arguments before execution and serialized tool results after execution. |
| 186 | +When Guardrails rewrites tool arguments or results, the rewritten content must |
| 187 | +be valid JSON. |
| 188 | + |
| 189 | +The bundled config uses the same NeMo Guardrails input and output self-check |
| 190 | +rails for both LLM messages and tool payloads. The plugin makes tool calls |
| 191 | +visible to Guardrails by serializing managed tool arguments and results as JSON |
| 192 | +message content. |
| 193 | + |
| 194 | +This behavior changes the real execution path. It is not an observability-only |
| 195 | +sanitize guardrail. |
| 196 | + |
| 197 | +## Supported Codecs |
| 198 | + |
| 199 | +The example is intentionally limited to NeMo Flow's built-in LLM codec shapes: |
| 200 | + |
| 201 | +- `openai_chat` for OpenAI Chat Completions-style requests and responses. |
| 202 | +- `openai_responses` for OpenAI Responses API-style requests and responses. |
| 203 | +- `anthropic_messages` for Anthropic Messages-style requests and responses. |
| 204 | + |
| 205 | +Provider-specific payloads outside those codecs need a NeMo Flow codec and a |
| 206 | +response text replacement strategy before a production plugin can apply |
| 207 | +modified output safely. |
| 208 | + |
| 209 | +## Limitations |
| 210 | + |
| 211 | +This example calls NeMo Guardrails `check_async`, not `generate_async`. It |
| 212 | +checks around NeMo Flow LLM and tool execution calls, but it does not let NeMo |
| 213 | +Guardrails take over generation or agent orchestration. |
| 214 | + |
| 215 | +The example does not support: |
| 216 | + |
| 217 | +- Streaming LLM calls. |
| 218 | +- Dialog rails, retrieval rails, execution rails, or generation rails that |
| 219 | + require NeMo Guardrails to orchestrate the full generation flow. |
| 220 | +- Arbitrary provider payloads beyond the three supported NeMo Flow codecs. |
| 221 | +- Applying modified LLM output back into provider responses. |
| 222 | +- Rewriting tool-call arguments inside model responses before an application |
| 223 | + turns those model tool calls into managed `tools.execute(...)` calls. |
| 224 | + |
| 225 | +Tool checks use serialized JSON and NeMo Guardrails input/output checks. They |
| 226 | +are NeMo Flow tool middleware checks powered by Guardrails, not a full |
| 227 | +`generate_async` agent-loop integration. |
| 228 | + |
| 229 | +`config_path` points at native NeMo Guardrails configuration. Guardrails config |
| 230 | +can load project code such as actions, so treat that path as trusted |
| 231 | +application code. |
0 commit comments