Skip to content

Commit a328e8a

Browse files
yczhang-nvyczhang-nv
authored
Add ATIF trajectory exporter for Phoenix visualization and debugging (#1869)
Implement `ATIFTrajectoryExporter` to visualize the complete ATIF trajectory. This exporter/viewer is for debugging purpose only. - Add `ATIFTrajectorySpanExporter` that converts complete ATIF trajectory JSON dicts into NAT Span objects, supporting agent steps (LLM), system/pipeline steps (FUNCTION), nested tool chains, and subagent trajectories - Add `ATIFTrajectoryPhoenixExporter` that wraps the converter to batch-export spans to a Phoenix server via OpenTelemetry HTTP - Add CLI script (`export_atif_trajectory_to_phoenix.py`) for exporting one or more ATIF JSON files to Phoenix with configurable options ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing/index.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **New Features** * Added ATIF trajectory export to Phoenix, converting trajectories into OpenTelemetry-compatible spans. * New CLI to export one or multiple ATIF JSON files with endpoint, project, and verbose options. * **Documentation** * Added a comprehensive README documenting setup, running a Phoenix server, export CLI usage, span hierarchy expectations, and behavior notes for trajectory processing. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - David Gardner (https://github.com/dagardner-nv) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: #1869
1 parent 3f159aa commit a328e8a

6 files changed

Lines changed: 1077 additions & 0 deletions

File tree

‎packages/nvidia_nat_phoenix/src/nat/plugins/phoenix/scripts/__init__.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.

‎packages/nvidia_nat_phoenix/src/nat/plugins/phoenix/scripts/export_trajectory_to_phoenix/README.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
<!--
2+
SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
SPDX-License-Identifier: Apache-2.0
4+
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
-->
17+
18+
# ATIF Trajectory Scripts
19+
20+
Scripts for converting and exporting ATIF trajectory JSON files to
21+
OpenTelemetry-compatible visualization tools such as Phoenix.
22+
23+
## Prerequisites
24+
25+
### Installing the Required Packages
26+
27+
From the repository root, install the required packages from source:
28+
29+
```bash
30+
uv pip install -e packages/nvidia_nat_phoenix
31+
```
32+
33+
### Starting the Phoenix Server
34+
35+
Run a Phoenix instance through Docker:
36+
37+
```bash
38+
docker run -it --rm -p 4317:4317 -p 6006:6006 arizephoenix/phoenix:13.22
39+
```
40+
41+
Once running, the Phoenix UI is available at `http://localhost:6006`.
42+
43+
## Exporting trajectories to Phoenix
44+
45+
```bash
46+
# Single file
47+
python -m nat.plugins.phoenix.scripts.export_trajectory_to_phoenix.export_atif_trajectory_to_phoenix trajectory.json
48+
49+
# Multiple files
50+
python -m nat.plugins.phoenix.scripts.export_trajectory_to_phoenix.export_atif_trajectory_to_phoenix *.json
51+
52+
# Custom endpoint and project
53+
python -m nat.plugins.phoenix.scripts.export_trajectory_to_phoenix.export_atif_trajectory_to_phoenix trajectory.json \
54+
--endpoint http://localhost:6006/v1/traces \
55+
--project my-project
56+
57+
# Enable debug logging
58+
python -m nat.plugins.phoenix.scripts.export_trajectory_to_phoenix.export_atif_trajectory_to_phoenix -v trajectory.json
59+
```
60+
61+
### CLI options
62+
63+
| Flag | Default | Description |
64+
|------|---------|-------------|
65+
| `files` (positional) | *(required)* | One or more ATIF trajectory JSON files |
66+
| `--endpoint` | `http://localhost:6006/v1/traces` | Phoenix server endpoint URL |
67+
| `--project` | `atif-trajectories` | Phoenix project name for trace grouping |
68+
| `--verbose`, `-v` | off | Enable debug logging |
69+
70+
## Architecture
71+
72+
### Processing flow
73+
74+
```text
75+
ATIF Trajectory (dict)
76+
-> ATIFTrajectorySpanExporter.convert() -> list[Span]
77+
-> convert_spans_to_otel_batch() -> list[OtelSpan]
78+
-> HTTPSpanExporter.export() -> Phoenix
79+
```
80+
81+
### Span hierarchy
82+
83+
Each trajectory produces the following span tree:
84+
85+
```text
86+
WORKFLOW span (root -- covers entire trajectory duration)
87+
|-- LLM span (agent step with tool_calls)
88+
| |-- TOOL span (tool call 1)
89+
| +-- TOOL span (tool call 2)
90+
|-- LLM span (next agent step)
91+
| +-- TOOL span
92+
|-- LLM span (terminal agent step -- final answer)
93+
|-- FUNCTION span (system step with tool_calls, no LLM)
94+
| |-- TOOL span (tool call 1)
95+
| +-- TOOL span (tool call 2, may nest under tool 1)
96+
+-- FUNCTION span (terminal system step)
97+
```
98+
99+
- **WORKFLOW** spans wrap the entire trajectory and carry the first
100+
user message as input and the last agent or system response as output.
101+
- **LLM** spans represent agent steps backed by a language model.
102+
- **FUNCTION** spans represent system or pipeline steps with no LLM involvement.
103+
- **TOOL** spans represent individual tool calls. Nested tool ancestry
104+
(tool A calls tool B) is preserved via parent-child relationships.
105+
106+
### Design notes
107+
108+
- User steps in external ATIF files typically lack `extra.ancestry`.
109+
The converter synthesises a root WORKFLOW span from trajectory
110+
metadata and re-parents orphaned agent spans under it.
111+
- `subagent_trajectories` are processed recursively and share
112+
the parent trace ID so all spans appear in a single Phoenix trace.
113+
- Tool spans that delegate to sub-agents are linked via
114+
`subagent_trajectory_ref` in observation results.
115+
- Each call to `export()` creates a new trace (unique trace ID),
116+
so re-exporting the same file will not produce duplicate spans
117+
within a single trace.
118+
- Timestamps prefer `extra.invocation` epoch timestamps when available
119+
and fall back to ISO `step.timestamp` fields only when no invocation
120+
timestamps exist.
121+
122+
## Modules
123+
124+
| Module | Description |
125+
|--------|-------------|
126+
| `atif_trajectory_exporter.py` | Core converter: ATIF trajectory dict to NeMo Agent Toolkit `Span` objects |
127+
| `atif_trajectory_phoenix_exporter.py` | Phoenix wrapper: converts spans to OTel and exports via HTTP |
128+
| `export_atif_trajectory_to_phoenix.py` | CLI entry point |

‎packages/nvidia_nat_phoenix/src/nat/plugins/phoenix/scripts/export_trajectory_to_phoenix/__init__.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.

0 commit comments

Comments
 (0)