Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/hub/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,8 @@
title: Embed the Dataset Viewer in a webpage
- local: datasets-viewer-sql-console
title: "SQL Console"
- local: agent-traces
title: Agent Traces
- local: datasets-download-stats
title: Datasets Download Stats

Expand Down
57 changes: 57 additions & 0 deletions docs/hub/agent-traces.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Agent Traces

<div class="flex justify-center">
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/agent-traces-preview-light.webp" alt="Agent trace preview on a dataset page in light mode."/>
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/agent-traces-preview-dark.webp" alt="Agent trace preview on a dataset page in dark mode."/>
</div>

Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a [Dataset](https://huggingface.co/datasets?format=format%3Aagent-traces) or a [Storage Bucket](./storage-buckets) to open them in a dedicated trace viewer. Datasets show traces in Data Studio; buckets let you open individual `.jsonl` files directly.

## Find your traces

Each supported agent writes JSONL sessions to the following directories:

| Agent | Local session directory |
| ----------- | ----------------------- |
| Claude Code | `~/.claude/projects` |
| Codex | `~/.codex/sessions` |
| Pi | `~/.pi/agent/sessions` |

These trace files are supported out of the box, so you can upload them without modifying or converting them first.

Trace files can include prompts, tool inputs, command output, local paths, screenshots, secrets, private code, and personal data. Review and redact traces before publishing them publicly, or keep the dataset or bucket private if you are not sure what is inside.

For Pi Agent sessions, [`pi-share-hf`](https://github.com/badlogic/pi-share-hf) can help collect project sessions, redact known secrets, run TruffleHog and LLM review, and upload only sessions that pass checks.

<Tip>

The easiest way to upload is to ask your agent itself: point it at the directory above and tell it to upload the `.jsonl` files to a Hub dataset or bucket.
Comment thread
cfahlgren1 marked this conversation as resolved.
Comment thread
cfahlgren1 marked this conversation as resolved.

Buckets are especially useful if you want to keep syncing traces as new sessions land.

</Tip>

## Upload your traces

After creating a dataset or bucket, install the `hf` CLI with the [recommended standalone installer](/docs/huggingface_hub/guides/cli#getting-started) and log in. If you want your coding agent to run `hf` commands for you, install the Hugging Face CLI skill with `hf skills add`.

```bash
curl -LsSf https://hf.co/cli/install.sh | bash
hf auth login
hf skills add

hf upload <username>/<dataset-name> ~/.codex/sessions . --repo-type dataset
hf buckets sync ~/.codex/sessions hf://buckets/<username>/<bucket-name>/codex
```

Replace `~/.codex/sessions` with the matching session directory for your agent. Use `hf upload` for datasets and `hf buckets sync` for buckets you want to update as new traces are written. The shorter `hf sync` command is an alias for `hf buckets sync`.

## View your traces

Once you have traces in a dataset, open Data Studio and click a row. For traces in a Storage Bucket, navigate to the `.jsonl` file you want to inspect and open it. The trace viewer shows the session timeline, prompts, assistant messages, tool calls, and results.

<div class="flex justify-center">
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/agent-traces-session.webp" alt="Agent trace session viewer showing user and assistant turns with expanded tool calls."/>
</div>

For a public example, open [`TeichAI/DeepSeek-v4-Pro-Agent`](https://huggingface.co/datasets/TeichAI/DeepSeek-v4-Pro-Agent). You can also browse more datasets tagged as [`traces`](https://huggingface.co/datasets?format=format%3Aagent-traces).
1 change: 1 addition & 0 deletions docs/hub/agents-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Hugging Face provides tools and protocols that connect AI agents directly to the
| [Skills](./agents-skills) | Task-specific guidance for AI/ML workflows |
| [SDK](./agents-sdk) | Build agents programmatically with Python or JavaScript |
| [Local Agents](./agents-local) | Run fully local agents with llama.cpp and Pi |
| [Agent Traces](./agent-traces) | View Claude Code, Codex, and Pi sessions on the Hub |

## Chat with Hugging Face

Expand Down
1 change: 1 addition & 0 deletions docs/hub/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ You can connect agents via the HF MCP Server, install pre-built Skills for codin
- [Building agents with the HF SDK](./agents-sdk)
- [Local Agents with llama.cpp and Pi](./agents-local)
- [Agent Libraries](./agents-libraries)
- [Agent Traces](./agent-traces)
8 changes: 6 additions & 2 deletions docs/hub/data-studio.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,14 @@ You can search for a word in the dataset by typing it in the search bar at the t

## Run SQL queries on the dataset

You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files).
You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files).

<div class="flex justify-center">
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/sql-ai.png" />
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/sql-ai-dark.png"/>
</div>

For more information see our guide on [SQL Console](./datasets-viewer-sql-console).
For more information see our guide on [SQL Console](./datasets-viewer-sql-console).

## Share a specific row

Expand All @@ -45,6 +45,10 @@ You can share a specific row by clicking on it, and then copying the URL in the
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/datastudio-row-dark.png"/>
</div>

## Agent traces

Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset to browse them in Data Studio, or sync them to a [Storage Bucket](./storage-buckets) and open individual `.jsonl` files there. Both paths use the same trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./agent-traces) for supported agents and viewing traces.

## Large scale datasets

The Dataset Viewer supports large scale datasets, but depending on the data format it may only show the first 5GB of the dataset:
Expand Down
1 change: 1 addition & 0 deletions docs/hub/datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,5 @@ This documentation focuses on the datasets functionality in the Hugging Face Hub
- [Downloading Datasets](./datasets-downloading)
- [Libraries](./datasets-libraries)
- [Dataset Viewer](./datasets-viewer)
- [Agent Traces](./agent-traces)
- [Data files Configuration](./datasets-data-files-configuration)
Loading