Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 18 additions & 14 deletions docs/source/build-workflows/advanced/middleware.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Create a configuration class inheriting from `DynamicMiddlewareConfig`:

```python
from pydantic import Field
from nat.middleware.dynamic.dynamic_middleware_config import DynamicMiddlewareConfig
from nat.plugin_api import DynamicMiddlewareConfig


class LoggingMiddlewareConfig(DynamicMiddlewareConfig, name="logging_middleware"):
Expand Down Expand Up @@ -168,8 +168,8 @@ Create the middleware class inheriting from `DynamicFunctionMiddleware`:
```python
import logging

from nat.middleware.dynamic.dynamic_function_middleware import DynamicFunctionMiddleware
from nat.middleware.middleware import InvocationContext
from nat.plugin_api import DynamicFunctionMiddleware
from nat.plugin_api import InvocationContext

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -244,8 +244,8 @@ Key benefits of extending `DynamicFunctionMiddleware`:
Create a registration module following the idiomatic pattern:

```python
from nat.builder.builder import Builder
from nat.cli.register_workflow import register_middleware
from nat.plugin_api import Builder
from nat.plugin_api import register_middleware
from .logging_middleware import LoggingMiddleware, LoggingMiddlewareConfig


Expand Down Expand Up @@ -289,8 +289,8 @@ functions:
Register your function without needing to specify middleware in the decorator:

```python
from nat.cli.register_workflow import register_function
from nat.builder.builder import Builder
from nat.plugin_api import register_function
from nat.plugin_api import Builder


@register_function(config_type=MyAPIFunctionConfig)
Expand Down Expand Up @@ -429,6 +429,9 @@ async def caching_middleware(config: CachingMiddlewareConfig, builder: Builder):
Final middleware can short-circuit execution:

```python
from nat.plugin_api import FunctionMiddleware
from nat.plugin_api import FunctionMiddlewareBaseConfig

class ValidationMiddlewareConfig(FunctionMiddlewareBaseConfig, name="validation"):
strict_mode: bool = Field(default=True)

Expand Down Expand Up @@ -521,9 +524,9 @@ function_groups:
```

```python
from nat.cli.register_workflow import register_function_group
from nat.builder.function import FunctionGroup
from nat.data_models.function import FunctionGroupBaseConfig
from nat.plugin_api import register_function_group
from nat.plugin_api import FunctionGroup
from nat.plugin_api import FunctionGroupBaseConfig


class WeatherAPIGroupConfig(FunctionGroupBaseConfig, name="weather_api_group"):
Expand Down Expand Up @@ -603,7 +606,8 @@ Test middleware in isolation:
```python
import pytest
from unittest.mock import MagicMock
from nat.middleware.middleware import FunctionMiddlewareContext, InvocationContext
from nat.plugin_api import FunctionMiddlewareContext
from nat.plugin_api import InvocationContext


@pytest.mark.asyncio
Expand Down Expand Up @@ -823,12 +827,12 @@ Solution: Ensure the register module is imported. NeMo Agent Toolkit automatical

## API Reference

- {py:class}`~nat.middleware.function_middleware.FunctionMiddleware`: Base class
- {py:class}`~nat.middleware.function_middleware.FunctionMiddlewareContext`: Context info
- {py:class}`~nat.plugin_api.FunctionMiddleware`: Base class
- {py:class}`~nat.plugin_api.FunctionMiddlewareContext`: Context info
- {py:class}`~nat.middleware.function_middleware.FunctionMiddlewareChain`: Chain management
- {py:class}`~nat.middleware.cache.cache_middleware_config.CacheMiddlewareConfig`: Cache configuration
- {py:class}`~nat.middleware.cache.cache_middleware.CacheMiddleware`: Cache implementation
- {py:func}`~nat.cli.register_workflow.register_middleware`: Registration decorator
- {py:func}`~nat.plugin_api.register_middleware`: Registration decorator

## See Also

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ workflow:

- **Multiple functions need the same connection** (database, API client, cache)
- **Functions share configuration** (credentials, endpoints, settings)
- **You want to namespace related functions** (`math.add`, `math.multiply`)
- **You want to namespace related functions** (`math__add`, `math__multiply`)
- **Functions need to share state** (session data, counters, caches)
- **You have a family of operations** (CRUD operations, data transformations)

Expand Down Expand Up @@ -365,7 +365,7 @@ This will return a list of all accessible functions in the function group that a

Functions inside a group are automatically namespaced by the group instance name. This creates a clear hierarchy and prevents naming conflicts.

To maintain compatibility with third-party libraries, the namespace separator switched from `.` (period) to `__` (double underscore).
Function groups expose functions with fully qualified names that use `__` (double underscore) between the group instance name and the function name. This keeps names compatible with third-party frameworks that reject or reinterpret `.` in tool names.

**Pattern**: `instance_name__function_name`

Expand Down Expand Up @@ -495,7 +495,7 @@ workflow:
llm_name: my_llm
```

All functions in the `math` group (`math.add`, `math.multiply`) become available as tools for the agent.
All functions in the `math` group (`math__add`, `math__multiply`) become available as tools for the agent.

#### Example 2: Including Specific Functions

Expand Down Expand Up @@ -602,8 +602,8 @@ async with WorkflowBuilder() as builder:
To wrap all accessible functions in a group for a specific agent framework:

```python
from nat.data_models.component_ref import FunctionGroupRef
from nat.builder.framework_enum import LLMFrameworkEnum
from nat.plugin_api import FunctionGroupRef
from nat.plugin_api import LLMFrameworkEnum

async with WorkflowBuilder() as builder:
await builder.add_function_group("math", MathGroupConfig(include=["add", "multiply"]))
Expand Down Expand Up @@ -640,7 +640,7 @@ async with WorkflowBuilder() as builder:

# Now only "add" functions are accessible
accessible = await math_group.get_accessible_functions()
# Returns: ["math.add"]
# Returns: ["math__add"]
```

#### Per-Function Filters
Expand Down Expand Up @@ -758,11 +758,11 @@ Instance names become part of function names, so keep them concise:
```python
# Good
group = FunctionGroup(config=config, instance_name="db")
# Results in: db.query, db.insert
# Results in: db__query, db__insert

# Less ideal
group = FunctionGroup(config=config, instance_name="database_operations")
# Results in: database_operations.query, database_operations.insert
# Results in: database_operations__query, database_operations__insert
```

#### Use Environment Variables for Secrets
Expand Down
2 changes: 1 addition & 1 deletion docs/source/build-workflows/mcp-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Example:
workflows:
_type: react_agent
tool_names:
- mcp_tools.tool_a
- mcp_tools__tool_a
```

An additional case to note is when a function group is served by an MCP server, the tools within the function group must still be accessed by their full name. This is the same as the prior case, but there is an important difference. Consider the following example:
Expand Down
2 changes: 2 additions & 0 deletions docs/source/build-workflows/retrievers.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ Retrievers are configured similarly to other NeMo Agent Toolkit components, such

Below is an example config object for the NeMo Retriever:
```python
from nat.plugin_api import RetrieverBaseConfig

class NemoRetrieverConfig(RetrieverBaseConfig, name="nemo_retriever"):
"""
Configuration for a Retriever which pulls data from a Nemo Retriever service.
Expand Down
7 changes: 5 additions & 2 deletions docs/source/components/sharing-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ requirements.
```python
from pydantic import Field

from nat.data_models.function import FunctionBaseConfig
from nat.plugin_api import FunctionBaseConfig

class MyFnConfig(FunctionBaseConfig, name="my_fn_name"): # includes a name
"""The docstring should provide a description of the components utility.""" # includes a docstring
Expand Down Expand Up @@ -104,10 +104,13 @@ When building the `pyproject.toml` file, there are two critical sections:
* Entrypoints: Provide the path to your plugins so they are registered with NeMo Agent Toolkit when installed.
An example is provided below:
```
[project.entry-points.'nat.components']
[project.entry-points.'nat.plugins']
nat_notional_pkg_name = "nat_notional_pkg_name.register"
```

The runtime continues to load `nat.components` entry points for backward compatibility, but new external packages
should use `nat.plugins`.

### Building a Wheel Package

After completing development and creating a `pyproject.toml` file that includes the necessary sections, the simplest
Expand Down
49 changes: 34 additions & 15 deletions docs/source/extend/custom-components/adding-a-retriever.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,36 +16,45 @@ limitations under the License.
-->

# Adding a Retriever Provider
New [retrievers](../../build-workflows/retrievers.md) can be added to NeMo Agent Toolkit by creating a plugin. The general process is the same as for most plugins, but the retriever-specific steps are outlined here.
New [retrievers](../../build-workflows/retrievers.md) can be added to NVIDIA NeMo Agent Toolkit by creating a plugin. The general process is the same as for most plugins, but the retriever-specific steps are outlined here.

First, create a retriever for the provider that implements the Retriever interface:
```python
class Retriever(ABC):
"""
Abstract interface for interacting with data stores.
from nat.plugin_api import Document
from nat.plugin_api import Retriever
from nat.plugin_api import RetrieverOutput

A Retriever is resposible for retrieving data from a configured data store.

Implemntations may integrate with vector stores or other indexing backends that allow for text-based search.
"""
class ExampleRetriever(Retriever):

@abstractmethod
async def search(self, query: str, **kwargs) -> RetrieverOutput:
"""
Retireve max(top_k) items from the data store based on vector similarity search (implementation dependent).
def __init__(self, client):
self._client = client

"""
raise NotImplementedError
async def search(self, query: str, **kwargs) -> RetrieverOutput:
result = await self._client.search(query=query, **kwargs)
return RetrieverOutput(
results=[
Document(page_content=item.text, metadata=item.metadata, document_id=item.id)
for item in result.items
]
)
```

Next, create the config for the provider and register it with NeMo Agent Toolkit:

```python
from nat.plugin_api import Builder
from nat.plugin_api import RetrieverBaseConfig
from nat.plugin_api import RetrieverProviderInfo
from nat.plugin_api import register_retriever_provider
from pydantic import Field
from pydantic import HttpUrl

class ExampleRetrieverConfig(RetrieverBaseConfig, name="example_retriever"):
"""
Configuration for a Retriever provider. The parameters will depend on the particular provider. These are examples.
"""
uri: HttpUrl = Field(description="The uri of the Nemo Retriever service.")
uri: HttpUrl = Field(description="The URI of the retriever service.")
collection_name: str = Field(description="The name of the collection to search")
top_k: int = Field(description="The number of results to return", gt=0, le=50, default=5)
output_fields: list[str] | None = Field(
Expand All @@ -61,11 +70,21 @@ async def example_retriever(retriever_config: ExampleRetrieverConfig, builder: B
Lastly, implement and register the retriever client:

```python
from nat.plugin_api import Builder
from nat.plugin_api import register_retriever_client

@register_retriever_client(config_type=ExampleRetrieverConfig, wrapper_type=None)
async def nemo_retriever_client(config: ExampleRetrieverConfig, builder: Builder):
from example_plugin.client import ExampleClient
from example_plugin.retriever import ExampleRetriever

retriever = ExampleRetriever(**config.model_dump())
client = ExampleClient(
uri=str(config.uri),
collection_name=config.collection_name,
top_k=config.top_k,
output_fields=config.output_fields,
)
retriever = ExampleRetriever(client=client)

yield retriever
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ authenticate with the target API resource.
The following example shows how to define and register a custom evaluator and can be found here:
{py:class}`~nat.authentication.oauth2.oauth2_auth_code_flow_provider_config.OAuth2AuthCodeFlowProviderConfig` class:
```python
from nat.data_models.authentication import AuthProviderBaseConfig

class OAuth2AuthCodeFlowProviderConfig(AuthProviderBaseConfig, name="oauth2_auth_code_flow"):

client_id: str = Field(description="The client ID for OAuth 2.0 authentication.")
Expand Down Expand Up @@ -75,6 +77,9 @@ An asynchronous function decorated with {py:func}`~nat.cli.register_workflow.reg

The `OAuth2AuthCodeFlowProviderConfig` from the previous section is registered as follows:
```python
from nat.plugin_api import Builder
from nat.cli.register_workflow import register_auth_provider

@register_auth_provider(config_type=OAuth2AuthCodeFlowProviderConfig)
async def oauth2_client(authentication_provider: OAuth2AuthCodeFlowProviderConfig, builder: Builder):
from nat.authentication.oauth2.oauth2_auth_code_flow_provider import OAuth2AuthCodeFlowProvider
Expand All @@ -91,7 +96,7 @@ After implementing a new authentication provider, it’s important to verify tha
## Packaging the Provider

The provider will need to be bundled into a Python package, which in turn will be registered with the toolkit as a [plugin](../plugins.md). In the `pyproject.toml` file of the package the
`project.entry-points.'nat.components'` section, defines a Python module as the entry point of the plugin. Details on how this is defined are found in the [Entry Point](../plugins.md#entry-point) section of the plugins document. By convention, the entry point module is named `register.py`, but this is not a requirement.
`project.entry-points.'nat.plugins'` section defines a Python module as the entry point of the plugin. Details on how this is defined are found in the [Entry Point](../plugins.md#entry-point) section of the plugins document. By convention, the entry point module is named `register.py`, but this is not a requirement.

In the entry point module, the registration of provider, that is the function decorated with `register_auth_provider`, needs to be defined, either directly or imported from another module. A hypothetical `register.py` file could be defined as follows:

Expand Down
Loading
Loading