Skip to content

Commit 91e5837

Browse files
authored
plugin docs: Stabilize the public plugin authoring API (#1959)
# Stabilize the public plugin authoring API Closes #1952. ## Summary This PR defines `nat.plugin_api` as the stable public import surface for third-party plugin authors and updates the author-facing docs/examples to use that surface instead of deeper implementation modules. It also adds a third-party plugin package guide for partner-owned repositories, including package naming, project layout, `nat.plugins` entry points, compatibility expectations, listing requirements, and testing guidance. The intent is to make the common external plugin contract explicit without redesigning provider-specific capabilities such as web search results, embeddings, or retrieval payloads, and without prematurely promoting every runtime extension point into the stable facade. ## What Changed - Adds `nat.plugin_api` as the public facade for plugin authoring APIs: - common registration decorators - builders and function/function-group authoring types - configuration bases and refs for promoted plugin surfaces - provider info helpers - secret helpers - small middleware, memory, and object-store implementation contracts needed by registered components - Adds contract tests for the exact `nat.plugin_api.__all__` export map so public API drift is intentional. - Tracks deferred public API candidates in the contract test, with source modules and rationale, so intentionally unpromoted surfaces are documented and do not silently rot. - Adds a public Plugin API docs page that distinguishes stable authoring APIs, trusted-plugin surfaces, deferred candidates, and private implementation modules. - Adds third-party plugin packaging docs based on the external repository process: repository ownership, package layout, dependency boundaries, version compatibility, licensing expectations, plugin listing metadata, and contribution checklist. - Documents the entry-point policy explicitly: new external packages should use the `nat.plugins` entry-point group, while `nat.components` remains supported for backward compatibility. - Clarifies that telemetry facade support covers exporter registration and configuration; exporter runtime implementation APIs remain subsystem-specific until promoted deliberately. - Updates custom component docs, workflow templates, and examples to import from `nat.plugin_api`. - Keeps docs for deferred extension points on their existing subsystem imports instead of presenting them as stable `nat.plugin_api` exports. - Extends `ToolTestRunner` with function-group helpers so external packages can test registered function groups and assert `group__function` names without constructing a full workflow. - Fixes stale function-group docs/examples that still referenced legacy group.function names; runtime dot-compatibility remains unchanged. - Clarifies that the existing Tavily and Exa integrations are LangChain-backed tools, while external integrations should depend on the framework-agnostic NeMo Agent Toolkit plugin authoring API. ## Scope Boundaries In scope: - Public import stability for plugin authors. - Documentation of which APIs third-party packages should use. - Third-party plugin package guidance for external repository maintainers. - Regression tests that protect the public facade. - Documented deferred candidates for advanced extension points that should be reviewed separately before promotion. - Function-group testing support needed by external packages using the `group__function` convention. Out of scope: - A shared web search result schema. - New capability-specific interfaces for search, embeddings, retrieval, or other managed-provider features. - Removing existing legacy imports from implementation modules. - Breaking users that still import from older module paths or still publish `nat.components` entry points. - Promoting advanced subsystem extension points that need more review, including front ends, logging methods, registry handlers, optimizer hooks, finetuning components, and TTC strategies. Web search result fields remain provider-specific in this PR. Third-party packages should use the public plugin authoring API, but their result payloads should stay unconstrained unless a separate capability contract is designed later. ## Motivation External packages such as `nemo-agent-toolkit-tavily` need a clearer contract for which NeMo Agent Toolkit APIs are public and stable. Today, plugin examples and downstream packages can end up importing from implementation modules such as `nat.builder.*`, `nat.cli.register_workflow`, and `nat.data_models.*`. This PR creates a conservative facade that external packages can rely on across minor and patch releases, while leaving private implementation modules free to evolve as long as the public contract remains intact. Installed plugins still execute as trusted Python code in the application environment. This PR stabilizes import paths and authoring contracts; it does not make untrusted plugins safe to install or run. ## Testing - `uv run ruff check packages/nvidia_nat_core/src/nat/plugin_api.py packages/nvidia_nat_core/tests/nat/test_plugin_api.py packages/nvidia_nat_test/src/nat/test/tool_test_runner.py packages/nvidia_nat_core/tests/nat/tools/test_tool_test_runner.py packages/nvidia_nat_langchain/src/nat/plugins/langchain/tools/tavily_internet_search.py packages/nvidia_nat_langchain/src/nat/plugins/langchain/tools/exa_internet_search.py` - `uv run --project packages/nvidia_nat_core --extra test pytest packages/nvidia_nat_core/tests/nat/test_plugin_api.py packages/nvidia_nat_core/tests/nat/tools/test_tool_test_runner.py -q` - `11 passed` - `uv run ruff check packages/nvidia_nat_core/src/nat/plugin_api.py packages/nvidia_nat_core/tests/nat/test_plugin_api.py docs/source/extend/third-party-plugins.md` - `uv run --project packages/nvidia_nat_core --extra test pytest packages/nvidia_nat_core/tests/nat/test_plugin_api.py packages/nvidia_nat_core/tests/nat/tools/test_tool_test_runner.py -q` - `14 passed` - `git diff --check` - `python3 ci/scripts/license_diff.py develop > license_diff_output.txt` - output file is empty - `python3 ci/scripts/path_checks.py --check-paths-in-files` - plugin API and third-party plugin docs path issues are cleared; local full run still reports unrelated missing `external/lc-deepagents-quickstarts` example paths from the checkout ## Follow-Ups - Migrate downstream external repos, starting with `nemo-agent-toolkit-tavily`, to import from `nat.plugin_api`. - Defer breaking import removals until external packages have had a release cycle to move to the facade. - Review deferred extension points individually before promoting them into `nat.plugin_api`. - Consider future capability contracts only where there is a concrete interoperability need. Web search result payloads are intentionally left provider-specific here. ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing/index.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **Documentation** * Updated many examples to use unified nat.plugin_api imports; added a Plugin API guide and third‑party plugin guidance; switched function-group/tool naming examples to double‑underscore separators. * **New Features** * Introduced nat.plugin_api as the stable public plugin authoring surface. * Added function‑group testing support to the test tooling to verify and invoke grouped tools. * **Tests** * Added tests enforcing the plugin API export contract and preferred import conventions. * Added tests covering function‑group exposure and invocation. Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - https://github.com/mnajafian-nv URL: #1959
1 parent 3424a44 commit 91e5837

47 files changed

Lines changed: 1739 additions & 244 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/source/build-workflows/advanced/middleware.md

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ Create a configuration class inheriting from `DynamicMiddlewareConfig`:
8484

8585
```python
8686
from pydantic import Field
87-
from nat.middleware.dynamic.dynamic_middleware_config import DynamicMiddlewareConfig
87+
from nat.plugin_api import DynamicMiddlewareConfig
8888

8989

9090
class LoggingMiddlewareConfig(DynamicMiddlewareConfig, name="logging_middleware"):
@@ -168,8 +168,8 @@ Create the middleware class inheriting from `DynamicFunctionMiddleware`:
168168
```python
169169
import logging
170170

171-
from nat.middleware.dynamic.dynamic_function_middleware import DynamicFunctionMiddleware
172-
from nat.middleware.middleware import InvocationContext
171+
from nat.plugin_api import DynamicFunctionMiddleware
172+
from nat.plugin_api import InvocationContext
173173

174174
logger = logging.getLogger(__name__)
175175

@@ -244,8 +244,8 @@ Key benefits of extending `DynamicFunctionMiddleware`:
244244
Create a registration module following the idiomatic pattern:
245245

246246
```python
247-
from nat.builder.builder import Builder
248-
from nat.cli.register_workflow import register_middleware
247+
from nat.plugin_api import Builder
248+
from nat.plugin_api import register_middleware
249249
from .logging_middleware import LoggingMiddleware, LoggingMiddlewareConfig
250250

251251

@@ -289,8 +289,8 @@ functions:
289289
Register your function without needing to specify middleware in the decorator:
290290
291291
```python
292-
from nat.cli.register_workflow import register_function
293-
from nat.builder.builder import Builder
292+
from nat.plugin_api import register_function
293+
from nat.plugin_api import Builder
294294

295295

296296
@register_function(config_type=MyAPIFunctionConfig)
@@ -429,6 +429,9 @@ async def caching_middleware(config: CachingMiddlewareConfig, builder: Builder):
429429
Final middleware can short-circuit execution:
430430

431431
```python
432+
from nat.plugin_api import FunctionMiddleware
433+
from nat.plugin_api import FunctionMiddlewareBaseConfig
434+
432435
class ValidationMiddlewareConfig(FunctionMiddlewareBaseConfig, name="validation"):
433436
strict_mode: bool = Field(default=True)
434437
@@ -521,9 +524,9 @@ function_groups:
521524
```
522525

523526
```python
524-
from nat.cli.register_workflow import register_function_group
525-
from nat.builder.function import FunctionGroup
526-
from nat.data_models.function import FunctionGroupBaseConfig
527+
from nat.plugin_api import register_function_group
528+
from nat.plugin_api import FunctionGroup
529+
from nat.plugin_api import FunctionGroupBaseConfig
527530
528531
529532
class WeatherAPIGroupConfig(FunctionGroupBaseConfig, name="weather_api_group"):
@@ -603,7 +606,8 @@ Test middleware in isolation:
603606
```python
604607
import pytest
605608
from unittest.mock import MagicMock
606-
from nat.middleware.middleware import FunctionMiddlewareContext, InvocationContext
609+
from nat.plugin_api import FunctionMiddlewareContext
610+
from nat.plugin_api import InvocationContext
607611
608612
609613
@pytest.mark.asyncio
@@ -823,12 +827,12 @@ Solution: Ensure the register module is imported. NeMo Agent Toolkit automatical
823827
824828
## API Reference
825829
826-
- {py:class}`~nat.middleware.function_middleware.FunctionMiddleware`: Base class
827-
- {py:class}`~nat.middleware.function_middleware.FunctionMiddlewareContext`: Context info
830+
- {py:class}`~nat.plugin_api.FunctionMiddleware`: Base class
831+
- {py:class}`~nat.plugin_api.FunctionMiddlewareContext`: Context info
828832
- {py:class}`~nat.middleware.function_middleware.FunctionMiddlewareChain`: Chain management
829833
- {py:class}`~nat.middleware.cache.cache_middleware_config.CacheMiddlewareConfig`: Cache configuration
830834
- {py:class}`~nat.middleware.cache.cache_middleware.CacheMiddleware`: Cache implementation
831-
- {py:func}`~nat.cli.register_workflow.register_middleware`: Registration decorator
835+
- {py:func}`~nat.plugin_api.register_middleware`: Registration decorator
832836
833837
## See Also
834838

docs/source/build-workflows/functions-and-function-groups/function-groups.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ workflow:
206206
207207
- **Multiple functions need the same connection** (database, API client, cache)
208208
- **Functions share configuration** (credentials, endpoints, settings)
209-
- **You want to namespace related functions** (`math.add`, `math.multiply`)
209+
- **You want to namespace related functions** (`math__add`, `math__multiply`)
210210
- **Functions need to share state** (session data, counters, caches)
211211
- **You have a family of operations** (CRUD operations, data transformations)
212212

@@ -365,7 +365,7 @@ This will return a list of all accessible functions in the function group that a
365365

366366
Functions inside a group are automatically namespaced by the group instance name. This creates a clear hierarchy and prevents naming conflicts.
367367

368-
To maintain compatibility with third-party libraries, the namespace separator switched from `.` (period) to `__` (double underscore).
368+
Function groups expose functions with fully qualified names that use `__` (double underscore) between the group instance name and the function name. This keeps names compatible with third-party frameworks that reject or reinterpret `.` in tool names.
369369

370370
**Pattern**: `instance_name__function_name`
371371

@@ -495,7 +495,7 @@ workflow:
495495
llm_name: my_llm
496496
```
497497

498-
All functions in the `math` group (`math.add`, `math.multiply`) become available as tools for the agent.
498+
All functions in the `math` group (`math__add`, `math__multiply`) become available as tools for the agent.
499499

500500
#### Example 2: Including Specific Functions
501501

@@ -602,8 +602,8 @@ async with WorkflowBuilder() as builder:
602602
To wrap all accessible functions in a group for a specific agent framework:
603603

604604
```python
605-
from nat.data_models.component_ref import FunctionGroupRef
606-
from nat.builder.framework_enum import LLMFrameworkEnum
605+
from nat.plugin_api import FunctionGroupRef
606+
from nat.plugin_api import LLMFrameworkEnum
607607
608608
async with WorkflowBuilder() as builder:
609609
await builder.add_function_group("math", MathGroupConfig(include=["add", "multiply"]))
@@ -640,7 +640,7 @@ async with WorkflowBuilder() as builder:
640640
641641
# Now only "add" functions are accessible
642642
accessible = await math_group.get_accessible_functions()
643-
# Returns: ["math.add"]
643+
# Returns: ["math__add"]
644644
```
645645

646646
#### Per-Function Filters
@@ -758,11 +758,11 @@ Instance names become part of function names, so keep them concise:
758758
```python
759759
# Good
760760
group = FunctionGroup(config=config, instance_name="db")
761-
# Results in: db.query, db.insert
761+
# Results in: db__query, db__insert
762762
763763
# Less ideal
764764
group = FunctionGroup(config=config, instance_name="database_operations")
765-
# Results in: database_operations.query, database_operations.insert
765+
# Results in: database_operations__query, database_operations__insert
766766
```
767767

768768
#### Use Environment Variables for Secrets

docs/source/build-workflows/mcp-client.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ Example:
7777
workflows:
7878
_type: react_agent
7979
tool_names:
80-
- mcp_tools.tool_a
80+
- mcp_tools__tool_a
8181
```
8282

8383
An additional case to note is when a function group is served by an MCP server, the tools within the function group must still be accessed by their full name. This is the same as the prior case, but there is an important difference. Consider the following example:

docs/source/build-workflows/retrievers.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,8 @@ Retrievers are configured similarly to other NeMo Agent Toolkit components, such
8181

8282
Below is an example config object for the NeMo Retriever:
8383
```python
84+
from nat.plugin_api import RetrieverBaseConfig
85+
8486
class NemoRetrieverConfig(RetrieverBaseConfig, name="nemo_retriever"):
8587
"""
8688
Configuration for a Retriever which pulls data from a Nemo Retriever service.

docs/source/components/sharing-components.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ requirements.
5858
```python
5959
from pydantic import Field
6060

61-
from nat.data_models.function import FunctionBaseConfig
61+
from nat.plugin_api import FunctionBaseConfig
6262

6363
class MyFnConfig(FunctionBaseConfig, name="my_fn_name"): # includes a name
6464
"""The docstring should provide a description of the components utility.""" # includes a docstring
@@ -104,10 +104,13 @@ When building the `pyproject.toml` file, there are two critical sections:
104104
* Entrypoints: Provide the path to your plugins so they are registered with NeMo Agent Toolkit when installed.
105105
An example is provided below:
106106
```
107-
[project.entry-points.'nat.components']
107+
[project.entry-points.'nat.plugins']
108108
nat_notional_pkg_name = "nat_notional_pkg_name.register"
109109
```
110110
111+
The runtime continues to load `nat.components` entry points for backward compatibility, but new external packages
112+
should use `nat.plugins`.
113+
111114
### Building a Wheel Package
112115
113116
After completing development and creating a `pyproject.toml` file that includes the necessary sections, the simplest

docs/source/extend/custom-components/adding-a-retriever.md

Lines changed: 34 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -16,36 +16,45 @@ limitations under the License.
1616
-->
1717

1818
# Adding a Retriever Provider
19-
New [retrievers](../../build-workflows/retrievers.md) can be added to NeMo Agent Toolkit by creating a plugin. The general process is the same as for most plugins, but the retriever-specific steps are outlined here.
19+
New [retrievers](../../build-workflows/retrievers.md) can be added to NVIDIA NeMo Agent Toolkit by creating a plugin. The general process is the same as for most plugins, but the retriever-specific steps are outlined here.
2020

2121
First, create a retriever for the provider that implements the Retriever interface:
2222
```python
23-
class Retriever(ABC):
24-
"""
25-
Abstract interface for interacting with data stores.
23+
from nat.plugin_api import Document
24+
from nat.plugin_api import Retriever
25+
from nat.plugin_api import RetrieverOutput
2626

27-
A Retriever is resposible for retrieving data from a configured data store.
2827

29-
Implemntations may integrate with vector stores or other indexing backends that allow for text-based search.
30-
"""
28+
class ExampleRetriever(Retriever):
3129

32-
@abstractmethod
33-
async def search(self, query: str, **kwargs) -> RetrieverOutput:
34-
"""
35-
Retireve max(top_k) items from the data store based on vector similarity search (implementation dependent).
30+
def __init__(self, client):
31+
self._client = client
3632

37-
"""
38-
raise NotImplementedError
33+
async def search(self, query: str, **kwargs) -> RetrieverOutput:
34+
result = await self._client.search(query=query, **kwargs)
35+
return RetrieverOutput(
36+
results=[
37+
Document(page_content=item.text, metadata=item.metadata, document_id=item.id)
38+
for item in result.items
39+
]
40+
)
3941
```
4042

4143
Next, create the config for the provider and register it with NeMo Agent Toolkit:
4244

4345
```python
46+
from nat.plugin_api import Builder
47+
from nat.plugin_api import RetrieverBaseConfig
48+
from nat.plugin_api import RetrieverProviderInfo
49+
from nat.plugin_api import register_retriever_provider
50+
from pydantic import Field
51+
from pydantic import HttpUrl
52+
4453
class ExampleRetrieverConfig(RetrieverBaseConfig, name="example_retriever"):
4554
"""
4655
Configuration for a Retriever provider. The parameters will depend on the particular provider. These are examples.
4756
"""
48-
uri: HttpUrl = Field(description="The uri of the Nemo Retriever service.")
57+
uri: HttpUrl = Field(description="The URI of the retriever service.")
4958
collection_name: str = Field(description="The name of the collection to search")
5059
top_k: int = Field(description="The number of results to return", gt=0, le=50, default=5)
5160
output_fields: list[str] | None = Field(
@@ -61,11 +70,21 @@ async def example_retriever(retriever_config: ExampleRetrieverConfig, builder: B
6170
Lastly, implement and register the retriever client:
6271

6372
```python
73+
from nat.plugin_api import Builder
74+
from nat.plugin_api import register_retriever_client
75+
6476
@register_retriever_client(config_type=ExampleRetrieverConfig, wrapper_type=None)
6577
async def nemo_retriever_client(config: ExampleRetrieverConfig, builder: Builder):
78+
from example_plugin.client import ExampleClient
6679
from example_plugin.retriever import ExampleRetriever
6780

68-
retriever = ExampleRetriever(**config.model_dump())
81+
client = ExampleClient(
82+
uri=str(config.uri),
83+
collection_name=config.collection_name,
84+
top_k=config.top_k,
85+
output_fields=config.output_fields,
86+
)
87+
retriever = ExampleRetriever(client=client)
6988

7089
yield retriever
7190
```

docs/source/extend/custom-components/adding-an-authentication-provider.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@ authenticate with the target API resource.
4848
The following example shows how to define and register a custom evaluator and can be found here:
4949
{py:class}`~nat.authentication.oauth2.oauth2_auth_code_flow_provider_config.OAuth2AuthCodeFlowProviderConfig` class:
5050
```python
51+
from nat.data_models.authentication import AuthProviderBaseConfig
52+
5153
class OAuth2AuthCodeFlowProviderConfig(AuthProviderBaseConfig, name="oauth2_auth_code_flow"):
5254

5355
client_id: str = Field(description="The client ID for OAuth 2.0 authentication.")
@@ -75,6 +77,9 @@ An asynchronous function decorated with {py:func}`~nat.cli.register_workflow.reg
7577

7678
The `OAuth2AuthCodeFlowProviderConfig` from the previous section is registered as follows:
7779
```python
80+
from nat.plugin_api import Builder
81+
from nat.cli.register_workflow import register_auth_provider
82+
7883
@register_auth_provider(config_type=OAuth2AuthCodeFlowProviderConfig)
7984
async def oauth2_client(authentication_provider: OAuth2AuthCodeFlowProviderConfig, builder: Builder):
8085
from nat.authentication.oauth2.oauth2_auth_code_flow_provider import OAuth2AuthCodeFlowProvider
@@ -91,7 +96,7 @@ After implementing a new authentication provider, it’s important to verify tha
9196
## Packaging the Provider
9297

9398
The provider will need to be bundled into a Python package, which in turn will be registered with the toolkit as a [plugin](../plugins.md). In the `pyproject.toml` file of the package the
94-
`project.entry-points.'nat.components'` section, defines a Python module as the entry point of the plugin. Details on how this is defined are found in the [Entry Point](../plugins.md#entry-point) section of the plugins document. By convention, the entry point module is named `register.py`, but this is not a requirement.
99+
`project.entry-points.'nat.plugins'` section defines a Python module as the entry point of the plugin. Details on how this is defined are found in the [Entry Point](../plugins.md#entry-point) section of the plugins document. By convention, the entry point module is named `register.py`, but this is not a requirement.
95100

96101
In the entry point module, the registration of provider, that is the function decorated with `register_auth_provider`, needs to be defined, either directly or imported from another module. A hypothetical `register.py` file could be defined as follows:
97102

0 commit comments

Comments
 (0)