trustyai-explainability · m-misiura · Feb 10, 2026 · Dec 3, 2025 · Dec 10, 2025 · Dec 10, 2025
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -9,6 +9,31 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 >
 > The changes related to the Colang language and runtime have moved to [CHANGELOG-Colang](./CHANGELOG-Colang.md) file.
 
+## [0.19.0] - 2025-12-03
+
+### 🚀 Features
+
+- Support langchain v1 ([#1472](https://github.com/NVIDIA/NeMo-Guardrails/issues/1472))
+- *(llm)* Add LangChain 1.x content blocks support for reasoning and tool calls ([#1496](https://github.com/NVIDIA/NeMo-Guardrails/issues/1496))
+- *(benchmark)* Add Procfile to run Guardrails and mock LLMs ([#1490](https://github.com/NVIDIA/NeMo-Guardrails/issues/1490))
+- *(benchmark)*: Add AIPerf run script (([#1501](https://github.com/NVIDIA/NeMo-Guardrails/issues/1501)))
+
+### 🐛 Bug Fixes
+
+- *(llm)* Add async streaming support to ChatNVIDIA provider patch ([#1504](https://github.com/NVIDIA/NeMo-Guardrails/issues/1504))
+- ensure stream_async background task completes before exit ([#1508](https://github.com/NVIDIA/NeMo-Guardrails/issues/1508))
+- *(cli)* Fix TypeError in v2.x chat due to incorrect State/dict conversion ([#1509](https://github.com/NVIDIA/NeMo-Guardrails/issues/1509))
+- *(llmrails)*: skip output rails when dialog disabled and no bot_message provided ([#1518](https://github.com/NVIDIA/NeMo-Guardrails/issues/1518))
+- *(llm)*: ensure that stop token is not ignored if llm_params is None ([#1529](https://github.com/NVIDIA/NeMo-Guardrails/issues/1529))
+
+### ⚙️ Miscellaneous Tasks
+
+- *(llm)* Remove deprecated llm_params module ([#1475](https://github.com/NVIDIA/NeMo-Guardrails/issues/1475))
+
+### ◀️ Revert
+
+- *(llm)* Remove custom HTTP headers patch now in langchain-nvidia-ai-endpoints v0.3.19 ([#1503](https://github.com/NVIDIA/NeMo-Guardrails/issues/1503))
+
 ## [0.18.0] - 2025-11-06
 
 ### 🚀 Features

diff --git a/Dockerfile b/Dockerfile
@@ -15,10 +15,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM python:3.10
+FROM python:3.12-slim
 
-# Install git and gcc/g++ for annoy
-RUN apt-get update && apt-get install -y git gcc g++
+RUN apt-get update && apt-get install -y --no-install-recommends git gcc g++ \
+    && rm -rf /var/lib/apt/lists/*
 
 # Set POETRY_VERSION environment variable
 ENV POETRY_VERSION=1.8.2

diff --git a/Makefile b/Makefile
@@ -1,4 +1,4 @@
-.PHONY: all test tests test_watch test_coverage test_profile docs pre_commit help
+.PHONY: all test tests test_watch test_coverage test_profile docs docs-serve docs-update-cards docs-check-cards docs-watch-cards pre_commit help
 
 # Default target executed when no specific target is provided to make.
 all: help
@@ -24,6 +24,18 @@ test_profile:
 docs:
 	poetry run sphinx-build -b html docs _build/docs
 
+docs-serve:
+	cd docs && poetry run sphinx-autobuild . _build/html --port 8000 --open-browser
+
+docs-update-cards:
+	cd docs && poetry run python scripts/update_cards/update_cards.py
+
+docs-check-cards:
+	cd docs && poetry run python scripts/update_cards/update_cards.py --dry-run
+
+docs-watch-cards:
+	cd docs && poetry run python scripts/update_cards/update_cards.py watch
+
 pre_commit:
 	pre-commit install
 	pre-commit run --all-files
@@ -39,4 +51,8 @@ help:
 	@echo 'test_watch                   - run unit tests in watch mode'
 	@echo 'test_coverage                - run unit tests with coverage'
 	@echo 'docs                         - build docs, if you installed the docs dependencies'
+	@echo 'docs-serve                   - serve docs locally with auto-rebuild on changes'
+	@echo 'docs-update-cards            - update grid cards in index files from linked pages'
+	@echo 'docs-check-cards             - check if grid cards are up to date (dry run)'
+	@echo 'docs-watch-cards             - watch for file changes and auto-update cards'
 	@echo 'pre_commit                   - run pre-commit hooks'
diff --git a/README.md b/README.md
@@ -1,13 +1,19 @@
 # NeMo Guardrails
 
-[![Tests](https://img.shields.io/badge/Tests-passing-green)](#)
-[![License](https://img.shields.io/badge/License-Apache%202.0-brightgreen.svg)](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/LICENSE.md)
-[![PyPI version](https://badge.fury.io/py/nemoguardrails.svg)](https://badge.fury.io/py/nemoguardrails)
-[![Python 3.8+](https://img.shields.io/badge/python-3.8%2B-green)](https://www.python.org/downloads/)
+[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+[![PyPI](https://img.shields.io/pypi/v/nemoguardrails)](https://pypi.org/project/nemoguardrails)
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/nemoguardrails)](https://pypi.org/project/nemoguardrails)
+[![Tests/Linux](https://img.shields.io/github/actions/workflow/status/NVIDIA/NeMo-Guardrails/pr-tests.yml?logo=github&label=Tests%2FLinux)](https://github.com/NVIDIA/NeMo-Guardrails/actions/workflows/pr-tests.yml)
+[![Tests/Windows](https://img.shields.io/github/actions/workflow/status/NVIDIA/NeMo-Guardrails/full-tests.yml?logo=github&label=Tests%2FWindows)](https://github.com/NVIDIA/NeMo-Guardrails/actions/workflows/full-tests.yml)
+[![Tests/macOS](https://img.shields.io/github/actions/workflow/status/NVIDIA/NeMo-Guardrails/full-tests.yml?logo=github&label=Tests%2FmacOS)](https://github.com/NVIDIA/NeMo-Guardrails/actions/workflows/full-tests.yml)
+[![Lint](https://img.shields.io/github/actions/workflow/status/NVIDIA/NeMo-Guardrails/lint.yml?logo=github&label=Lint)](https://github.com/NVIDIA/NeMo-Guardrails/actions/workflows/lint.yml)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
-[![arXiv](https://img.shields.io/badge/arXiv-2310.10501-b31b1b.svg)](https://arxiv.org/abs/2310.10501)
+[![Documentation](https://img.shields.io/badge/docs-nvidia.com-blue.svg)](https://docs.nvidia.com/nemo/guardrails)
+[![arXiv](https://img.shields.io/badge/cs.CL-arXiv%3A2310.10501-b31b1b.svg)](https://arxiv.org/abs/2310.10501)
+[![Downloads](https://static.pepy.tech/badge/nemoguardrails)](https://pepy.tech/project/nemoguardrails)
+[![Downloads](https://static.pepy.tech/badge/nemoguardrails/month)](https://pepy.tech/project/nemoguardrails)
 
-> **LATEST RELEASE / DEVELOPMENT VERSION**: The [main](https://github.com/NVIDIA/NeMo-Guardrails/tree/main) branch tracks the latest released beta version: [0.18.0](https://github.com/NVIDIA/NeMo-Guardrails/tree/v0.18.0). For the latest development version, checkout the [develop](https://github.com/NVIDIA/NeMo-Guardrails/tree/develop) branch.
+> **LATEST RELEASE / DEVELOPMENT VERSION**: The [main](https://github.com/NVIDIA/NeMo-Guardrails/tree/main) branch tracks the latest released beta version: [0.19.0](https://github.com/NVIDIA/NeMo-Guardrails/tree/v0.19.0). For the latest development version, checkout the [develop](https://github.com/NVIDIA/NeMo-Guardrails/tree/develop) branch.
 
 ✨✨✨
 

diff --git a/benchmark/Procfile b/benchmark/Procfile
@@ -0,0 +1,8 @@
+# Procfile
+
+# NeMo Guardrails server
+gr: poetry run nemoguardrails server --config ../examples/configs/content_safety_local --default-config-id content_safety_local --port 9000
+
+# Guardrails NIMs for inference. PYTHONPATH is set to the project root so absolute imports work
+app_llm: PYTHONPATH=.. python mock_llm_server/run_server.py --workers 4 --port 8000 --config-file mock_llm_server/configs/meta-llama-3.3-70b-instruct.env
+cs_llm: PYTHONPATH=.. python mock_llm_server/run_server.py --workers 4 --port 8001 --config-file mock_llm_server/configs/nvidia-llama-3.1-nemoguard-8b-content-safety.env
diff --git a/nemoguardrails/benchmark/README.md → benchmark/README.md b/nemoguardrails/benchmark/README.md → benchmark/README.md
@@ -14,29 +14,40 @@ All models use the [Mock LLM Server](mock_llm_server), which is a simplified mod
 The aim of this benchmark is to detect performance-regressions as quickly as running unit-tests.
 
 ## Quickstart: Running Guardrails with Mock LLMs
+
 To run Guardrails with mocks for both the content-safety and main LLM, follow the steps below.
-All commands must be run in the `nemoguardrails/benchmark` directory.
-These assume you already have a working environment after following the steps in [CONTRIBUTING.md](../../CONTRIBUTING.md).
+All commands must be run in the `benchmark` directory.
+
+### 1. Set up benchmarking virtual environment
+
+The benchmarking tools have their own dependencies, which are managed using a virtual environment, pip, and the [requirements.txt](requirements.txt) file.
+In this section, you'll create a new virtual environment, activate it, and install all the dependencies needed to benchmark Guardrails.
 
-First, we need to install the honcho and langchain-nvidia-ai-endpoints packages.
-The `honcho` package is used to run Procfile-based applications, and is a Python port of [Foreman](https://github.com/ddollar/foreman).
-The `langchain-nvidia-ai-endpoints` package is used to communicate with Mock LLMs via Langchain.
+First you'll create the virtual environment and install dependencies.
 
 ```shell
-# Install dependencies
-$ poetry run pip install honcho langchain-nvidia-ai-endpoints
+# Create a virtual environment under ~/env/benchmark_env and activate it
+
+$ cd benchmark
+$ mkdir ~/env
+$ python -m venv ~/env/benchmark_env
+$ pip install -r requirements.txt
 ...
-Successfully installed filetype-1.2.0 honcho-2.0.0 langchain-nvidia-ai-endpoints-0.3.19
+Successfully installed fastapi-0.128.0 honcho-2.0.0 httpx-0.28.1 langchain-core-1.2.5 numpy-2.4.0 pydantic-2.12.5 pydantic-core-2.41.5 pydantic-settings-2.12.0 pyyaml-6.0.3 typer-0.21.0 typing-inspection-0.4.2 uuid-utils-0.12.0 uvicorn-0.40.0
+$ source ~/env/benchmark_env/bin/activate
+(benchmark_env) $
 ```
 
+### 2. Run Guardrails with Mock LLMs for Content-Safety and Application LLM
+
 Now we can start up the processes that are part of the [Procfile](Procfile).
 As the Procfile processes spin up, they log to the console with a prefix. The `system` prefix is used by Honcho, `app_llm` is the Application or Main LLM mock, `cs_llm` is the content-safety mock, and `gr` is the Guardrails service. We'll explore the Procfile in more detail below.
 Once the three 'Uvicorn running on ...' messages are printed, you can move to the next step. Note these messages are likely not on consecutive lines.
 
-```
-# All commands must be run in the nemoguardrails/benchmark directory
-$ cd nemoguardrails/benchmark
-$ poetry run honcho start
+```shell
+# These commands must be run in the benchmark directory after activating the benchmark_env virtual environment
+
+(benchmark_env) $ honcho start
 13:40:33 system    | gr.1 started (pid=93634)
 13:40:33 system    | app_llm.1 started (pid=93635)
 13:40:33 system    | cs_llm.1 started (pid=93636)
@@ -48,34 +59,34 @@ $ poetry run honcho start
 13:40:45 gr.1      | INFO:     Uvicorn running on http://0.0.0.0:9000 (Press CTRL+C to quit)
 ```
 
-Once Guardrails and the mock servers are up, we can use the `validate_mocks.py` script to check they're healthy and serving the correct models.
+### 3. Validate services are running correctly
+
+Once Guardrails and the mock servers are up, we'll use the [validate_mocks.sh](scripts/validate_mocks.sh) script to validate everything is working.
+This doesn't require the `benchmark_env` virtual environment since we're running curl commands in the script.
 
 ```shell
-$ cd nemoguardrails/benchmark
-$ poetry run python validate_mocks.py
+# In a new shell, change into the benchmark directory and run these commands.
+
+$ cd benchmark
+$ scripts/validate_mocks.sh
 Starting LLM endpoint health check...
 
 --- Checking Port: 8000 ---
 Checking http://localhost:8000/health ...
-HTTP Request: GET http://localhost:8000/health "HTTP/1.1 200 OK"
 Health Check PASSED: Status is 'healthy'.
 Checking http://localhost:8000/v1/models for 'meta/llama-3.3-70b-instruct'...
-HTTP Request: GET http://localhost:8000/v1/models "HTTP/1.1 200 OK"
 Model Check PASSED: Found 'meta/llama-3.3-70b-instruct' in model list.
 --- Port 8000: ALL CHECKS PASSED ---
 
 --- Checking Port: 8001 ---
 Checking http://localhost:8001/health ...
-HTTP Request: GET http://localhost:8001/health "HTTP/1.1 200 OK"
 Health Check PASSED: Status is 'healthy'.
 Checking http://localhost:8001/v1/models for 'nvidia/llama-3.1-nemoguard-8b-content-safety'...
-HTTP Request: GET http://localhost:8001/v1/models "HTTP/1.1 200 OK"
 Model Check PASSED: Found 'nvidia/llama-3.1-nemoguard-8b-content-safety' in model list.
 --- Port 8001: ALL CHECKS PASSED ---
 
 --- Checking Port: 9000 (Rails Config) ---
 Checking http://localhost:9000/v1/rails/configs ...
-HTTP Request: GET http://localhost:9000/v1/rails/configs "HTTP/1.1 200 OK"
 HTTP Status PASSED: Got 200.
 Body Check PASSED: Response is an array with at least one entry.
 --- Port 9000: ALL CHECKS PASSED ---
@@ -88,10 +99,12 @@ Port 9000 (Rails Config): PASSED
 Overall Status: All endpoints are healthy!
 ```
 
+### 4. Make Guardrails requests
+
 Once the mocks and Guardrails are running and the script passes, we can issue curl requests against the Guardrails `/chat/completions` endpoint to generate a response and test the system end-to-end.
 
 ```shell
-curl -s -X POST http://0.0.0.0:9000/v1/chat/completions \
+ $ curl -s -X POST http://0.0.0.0:9000/v1/chat/completions \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
@@ -104,6 +117,7 @@ curl -s -X POST http://0.0.0.0:9000/v1/chat/completions \
       ],
       "stream": false
     }' | jq
+
 {
   "messages": [
     {
@@ -112,7 +126,6 @@ curl -s -X POST http://0.0.0.0:9000/v1/chat/completions \
     }
   ]
 }
-
 ```
 
 ------
@@ -123,20 +136,22 @@ In this section, we'll examine the configuration files used in the quickstart ab
 
 ### Procfile
 
-The [Procfile](Procfile?raw=true) contains all the processes that make up the application.
+The [Procfile](Procfile) contains all the processes that make up the application.
 The Honcho package reads in this file, starts all the processes, and combines their logs to the console
-The `gr` line runs the Guardrails server on port 9000 and sets the default Guardrails configuration as [content_safety_colang1](configs/guardrail_configs/content_safety_colang1?raw=true).
+The `gr` line runs the Guardrails server on port 9000 and sets the default Guardrails configuration as [content_safety_local](../examples/configs/content_safety_local).
 The `app_llm` line runs the Application or Main Mock LLM. Guardrails calls this LLM to generate a response to the user's query. This server uses 4 uvicorn workers and runs on port 8000. The configuration file here is a Mock LLM configuration, not a Guardrails configuration.
 The `cs_llm` line runs the Content-Safety Mock LLM. This uses 4 uvicorn workers and runs on port 8001.
 
 ### Guardrails Configuration
-The [Guardrails Configuration](configs/guardrail_configs/content_safety_colang1/config.yml) is used by the Guardrails server.
+
+The [Guardrails Configuration](../examples/configs/content_safety_local/config.yml) is used by the Guardrails server.
 Under the `models` section, the `main` model is used to generate responses to the user queries. The base URL for this model is the `app_llm` Mock LLM from the Procfile, running on port 8000. The `model` field has to match the Mock LLM model name.
 The `content_safety` model is configured for use in an input and output rail. The `type` field matches the `$model` used in the input and output flows.
 
 ### Mock LLM Endpoints
+
 The Mock LLM implements a subset of the OpenAI LLM API.
-There are two Mock LLM configurations, one for the Mock [main model](configs/mock_configs/meta-llama-3.3-70b-instruct.env), and another for the Mock [content-safety](configs/mock_configs/nvidia-llama-3.1-nemoguard-8b-content-safety.env) model.
+There are two Mock LLM configurations, one for the Mock [main model](mock_llm_server/configs/meta-llama-3.3-70b-instruct.env), and another for the Mock [content-safety](mock_llm_server/configs/nvidia-llama-3.1-nemoguard-8b-content-safety.env) model.
 The Mock LLM has the following OpenAI-compatible endpoints:
 
 * `/health`: Returns a JSON object with status set to healthy and timestamp in seconds-since-epoch. For example `{"status":"healthy","timestamp":1762781239}`
@@ -145,6 +160,7 @@ The Mock LLM has the following OpenAI-compatible endpoints:
 * `/v1/chat/completions`: Returns an [OpenAI chat completion object](https://platform.openai.com/docs/api-reference/chat/object) using the Mock configuration (see below).
 
 ### Mock LLM Configuration
+
 Mock LLMs are configured using the `.env` file format. These files are passed to the Mock LLM using the `--config-file` argument.
 The Mock LLMs return either a `SAFE_TEXT` or `UNSAFE_TEXT` response to `/v1/completions` or `/v1/chat/completions` inference requests.
 The probability of the `UNSAFE_TEXT` being returned if given by `UNSAFE_PROBABILITY`.
@@ -155,8 +171,9 @@ The latency of each response is also controllable, and works as follows:
 * If the sampled value is less than `LATENCY_MAX_SECONDS`, it is set to `LATENCY_MAX_SECONDS`.
 
 The full list of configuration fields is shown below:
+
 * `MODEL`: The Model name served by the Mock LLM. This will be returned on the `/v1/models` endpoint.
-* `UNSAFE_PROBABILITY`: Probability of an unsafe response. This is a probability, and must be in the range [0, 1].
+* `UNSAFE_PROBABILITY`: Probability of an unsafe response. This must be in the range [0, 1].
 * `UNSAFE_TEXT`: String returned as an unsafe response.
 * `SAFE_TEXT`: String returned as a safe response.
 * `LATENCY_MIN_SECONDS`: Minimum latency in seconds.

diff --git a/benchmark/aiperf/README.md b/benchmark/aiperf/README.md
@@ -20,9 +20,9 @@ Instead of manually running AIPerf multiple times with different parameters, you
 ### Prerequisites
 
 These steps have been tested with Python 3.11.11.
-To use the provided configurations, you need to create accounts at https://build.nvidia.com/ and [Huggingface](https://huggingface.co/).
-* The provided configurations use models hosted at https://build.nvidia.com/, you'll need to create a Personal API Key to access the models.
-* The provided AIperf configurations require the [Meta Llama 3.3 70B Instruct tokenizer](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) to calculate token-counts.
+To use the provided configurations, you need to create accounts at <https://build.nvidia.com/> and [Huggingface](https://huggingface.co/).
+- The provided configurations use models hosted at <https://build.nvidia.com/>, you'll need to create a Personal API Key to access the models.
+- The provided AIperf configurations require the [Meta Llama 3.3 70B Instruct tokenizer](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) to calculate token-counts.
 
 1. **Create a virtual environment in which to install AIPerf**
 
@@ -37,13 +37,13 @@ To use the provided configurations, you need to create accounts at https://build
    $ pip install aiperf huggingface_hub typer
    ```
 
-3. ** Login to Hugging Face:**
+3. **Login to Hugging Face:**
 
    ```bash
    huggingface-cli login
    ```
 
-4. ** Set NVIDIA API Key:**
+4. **Set NVIDIA API Key:**
 
    The provided configs use models hosted on [build.nvidia.com](https://build.nvidia.com/).
    To access these, [create an account](https://build.nvidia.com/), and create a Personal API Key.

diff --git a/benchmark/aiperf/__init__.py b/benchmark/aiperf/__init__.py
@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");

diff --git a/benchmark/aiperf/__main__.py b/benchmark/aiperf/__main__.py
@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");

diff --git a/benchmark/aiperf/aiperf_models.py b/benchmark/aiperf/aiperf_models.py
@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");

diff --git a/benchmark/aiperf/run_aiperf.py b/benchmark/aiperf/run_aiperf.py
@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 #
 # Licensed under the Apache License, Version 2.0 (the "License");