Skip to content

Commit bf2a50f

Browse files
cfahlgren1Pierrcijulien-c
authored
add agent traces docs (#2492)
* add agent traces docs * Update docs/hub/data-studio.md Co-authored-by: Pierric Cistac <Pierrci@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Caleb Fahlgren <cfahlgren1@gmail.com> * address agent traces review feedback * Apply suggestions from code review Co-authored-by: Julien Chaumond <julien@huggingface.co> * rename datasets-agent-traces to agent-traces to match toctree * Update agent-traces.md * link agent traces page from agents overview and index --------- Co-authored-by: Pierric Cistac <Pierrci@users.noreply.github.com> Co-authored-by: Julien Chaumond <julien@huggingface.co>
1 parent 6d54527 commit bf2a50f

6 files changed

Lines changed: 68 additions & 2 deletions

File tree

docs/hub/_toctree.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,8 @@
311311
title: Embed the Dataset Viewer in a webpage
312312
- local: datasets-viewer-sql-console
313313
title: "SQL Console"
314+
- local: agent-traces
315+
title: Agent Traces
314316
- local: datasets-download-stats
315317
title: Datasets Download Stats
316318

docs/hub/agent-traces.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Agent Traces
2+
3+
<div class="flex justify-center">
4+
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/agent-traces-preview-light.webp" alt="Agent trace preview on a dataset page in light mode."/>
5+
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/agent-traces-preview-dark.webp" alt="Agent trace preview on a dataset page in dark mode."/>
6+
</div>
7+
8+
Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a [Dataset](https://huggingface.co/datasets?format=format%3Aagent-traces) or a [Storage Bucket](./storage-buckets) to open them in a dedicated trace viewer. Datasets show traces in Data Studio; buckets let you open individual `.jsonl` files directly.
9+
10+
## Find your traces
11+
12+
Each supported agent writes JSONL sessions to the following directories:
13+
14+
| Agent | Local session directory |
15+
| ----------- | ----------------------- |
16+
| Claude Code | `~/.claude/projects` |
17+
| Codex | `~/.codex/sessions` |
18+
| Pi | `~/.pi/agent/sessions` |
19+
20+
These trace files are supported out of the box, so you can upload them without modifying or converting them first.
21+
22+
Trace files can include prompts, tool inputs, command output, local paths, screenshots, secrets, private code, and personal data. Review and redact traces before publishing them publicly, or keep the dataset or bucket private if you are not sure what is inside.
23+
24+
For Pi Agent sessions, [`pi-share-hf`](https://github.com/badlogic/pi-share-hf) can help collect project sessions, redact known secrets, run TruffleHog and LLM review, and upload only sessions that pass checks.
25+
26+
<Tip>
27+
28+
The easiest way to upload is to ask your agent itself: point it at the directory above and tell it to upload the `.jsonl` files to a Hub dataset or bucket.
29+
30+
Buckets are especially useful if you want to keep syncing traces as new sessions land.
31+
32+
</Tip>
33+
34+
## Upload your traces
35+
36+
After creating a dataset or bucket, install the `hf` CLI with the [recommended standalone installer](/docs/huggingface_hub/guides/cli#getting-started) and log in. If you want your coding agent to run `hf` commands for you, install the Hugging Face CLI skill with `hf skills add`.
37+
38+
```bash
39+
curl -LsSf https://hf.co/cli/install.sh | bash
40+
hf auth login
41+
hf skills add
42+
43+
hf upload <username>/<dataset-name> ~/.codex/sessions . --repo-type dataset
44+
hf buckets sync ~/.codex/sessions hf://buckets/<username>/<bucket-name>/codex
45+
```
46+
47+
Replace `~/.codex/sessions` with the matching session directory for your agent. Use `hf upload` for datasets and `hf buckets sync` for buckets you want to update as new traces are written. The shorter `hf sync` command is an alias for `hf buckets sync`.
48+
49+
## View your traces
50+
51+
Once you have traces in a dataset, open Data Studio and click a row. For traces in a Storage Bucket, navigate to the `.jsonl` file you want to inspect and open it. The trace viewer shows the session timeline, prompts, assistant messages, tool calls, and results.
52+
53+
<div class="flex justify-center">
54+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/agent-traces-session.webp" alt="Agent trace session viewer showing user and assistant turns with expanded tool calls."/>
55+
</div>
56+
57+
For a public example, open [`TeichAI/DeepSeek-v4-Pro-Agent`](https://huggingface.co/datasets/TeichAI/DeepSeek-v4-Pro-Agent). You can also browse more datasets tagged as [`traces`](https://huggingface.co/datasets?format=format%3Aagent-traces).

docs/hub/agents-overview.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Hugging Face provides tools and protocols that connect AI agents directly to the
99
| [Skills](./agents-skills) | Task-specific guidance for AI/ML workflows |
1010
| [SDK](./agents-sdk) | Build agents programmatically with Python or JavaScript |
1111
| [Local Agents](./agents-local) | Run fully local agents with llama.cpp and Pi |
12+
| [Agent Traces](./agent-traces) | View Claude Code, Codex, and Pi sessions on the Hub |
1213

1314
## Chat with Hugging Face
1415

docs/hub/agents.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,4 @@ You can connect agents via the HF MCP Server, install pre-built Skills for codin
1313
- [Building agents with the HF SDK](./agents-sdk)
1414
- [Local Agents with llama.cpp and Pi](./agents-local)
1515
- [Agent Libraries](./agents-libraries)
16+
- [Agent Traces](./agent-traces)

docs/hub/data-studio.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,14 +27,14 @@ You can search for a word in the dataset by typing it in the search bar at the t
2727

2828
## Run SQL queries on the dataset
2929

30-
You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files).
30+
You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files).
3131

3232
<div class="flex justify-center">
3333
<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/sql-ai.png" />
3434
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/sql-ai-dark.png"/>
3535
</div>
3636

37-
For more information see our guide on [SQL Console](./datasets-viewer-sql-console).
37+
For more information see our guide on [SQL Console](./datasets-viewer-sql-console).
3838

3939
## Share a specific row
4040

@@ -45,6 +45,10 @@ You can share a specific row by clicking on it, and then copying the URL in the
4545
<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/datastudio-row-dark.png"/>
4646
</div>
4747

48+
## Agent traces
49+
50+
Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset to browse them in Data Studio, or sync them to a [Storage Bucket](./storage-buckets) and open individual `.jsonl` files there. Both paths use the same trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./agent-traces) for supported agents and viewing traces.
51+
4852
## Large scale datasets
4953

5054
The Dataset Viewer supports large scale datasets, but depending on the data format it may only show the first 5GB of the dataset:

docs/hub/datasets.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,5 @@ This documentation focuses on the datasets functionality in the Hugging Face Hub
1313
- [Downloading Datasets](./datasets-downloading)
1414
- [Libraries](./datasets-libraries)
1515
- [Dataset Viewer](./datasets-viewer)
16+
- [Agent Traces](./agent-traces)
1617
- [Data files Configuration](./datasets-data-files-configuration)

0 commit comments

Comments
 (0)