From 87c046b85018e43e35e17f1d34b5b26d98214e0d Mon Sep 17 00:00:00 2001 From: Caleb Fahlgren Date: Thu, 21 May 2026 17:07:16 -0400 Subject: [PATCH 1/8] add agent traces docs --- docs/hub/_toctree.yml | 2 ++ docs/hub/data-studio.md | 36 ++++++++++++++------------ docs/hub/datasets-agent-traces.md | 42 +++++++++++++++++++++++++++++++ docs/hub/datasets.md | 1 + 4 files changed, 65 insertions(+), 16 deletions(-) create mode 100644 docs/hub/datasets-agent-traces.md diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index 01720e6add..309d8655a9 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -311,6 +311,8 @@ title: Embed the Dataset Viewer in a webpage - local: datasets-viewer-sql-console title: "SQL Console" + - local: datasets-agent-traces + title: Agent Traces - local: datasets-download-stats title: Datasets Download Stats diff --git a/docs/hub/data-studio.md b/docs/hub/data-studio.md index e8bc5b7230..7f46c96102 100644 --- a/docs/hub/data-studio.md +++ b/docs/hub/data-studio.md @@ -27,14 +27,14 @@ You can search for a word in the dataset by typing it in the search bar at the t ## Run SQL queries on the dataset -You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files). +You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files).
-For more information see our guide on [SQL Console](./datasets-viewer-sql-console). +For more information see our guide on [SQL Console](./datasets-viewer-sql-console). ## Share a specific row @@ -45,6 +45,10 @@ You can share a specific row by clicking on it, and then copying the URL in the +## Agent traces + +Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset (or a [Storage Bucket](./storage-buckets)), and your dataset page will show a dedicated trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./datasets-agent-traces) for supported agents and viewing traces. + ## Large scale datasets The Dataset Viewer supports large scale datasets, but depending on the data format it may only show the first 5GB of the dataset: @@ -101,12 +105,12 @@ You can specify which files to display in the Dataset Viewer by adding a YAML co ```yaml --- configs: -- config_name: default - data_files: - - split: train - path: "data.csv" - - split: test - path: "holdout.csv" + - config_name: default + data_files: + - split: train + path: "data.csv" + - split: test + path: "holdout.csv" --- ``` @@ -115,14 +119,14 @@ You can also select multiple files per split or use glob patterns: ```yaml --- configs: -- config_name: default - data_files: - - split: train - path: - - "data/train_part1.csv" - - "data/train_part2.csv" - - split: test - path: "data/*.csv" + - config_name: default + data_files: + - split: train + path: + - "data/train_part1.csv" + - "data/train_part2.csv" + - split: test + path: "data/*.csv" --- ``` diff --git a/docs/hub/datasets-agent-traces.md b/docs/hub/datasets-agent-traces.md new file mode 100644 index 0000000000..699c83485f --- /dev/null +++ b/docs/hub/datasets-agent-traces.md @@ -0,0 +1,42 @@ +# Agent traces + +
+Agent trace preview on a dataset page in light mode. + +
+ +Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a dataset or a [Storage Bucket](./storage-buckets) and Data Studio opens them in a dedicated trace viewer. + +## Find your traces + +Each supported agent writes JSONL sessions to a known directory: + +| Agent | Local session directory | +| ----------- | ----------------------- | +| Claude Code | `~/.claude/projects` | +| Codex | `~/.codex/sessions` | +| Pi | `~/.pi/agent/sessions` | + +These trace files are supported out of the box, so you can upload them without modifying or converting them first. + +Trace files can include prompts, tool inputs, command output, local paths, screenshots, secrets, private code, and personal data. Review and redact traces before publishing them publicly, or keep the dataset or bucket private if you are not sure what is inside. + +For Pi Agent sessions, [`pi-share-hf`](https://github.com/badlogic/pi-share-hf) can help collect project sessions, redact known secrets, run TruffleHog and LLM review, and upload only sessions that pass checks. + + + +The easiest way to upload is to ask your agent itself: point it at the directory above and tell it to upload the `.jsonl` files to a Hub dataset or bucket. + +Buckets are especially useful if you want to keep syncing traces as new sessions land. + + + +## View your traces + +Once you have traces in a dataset or bucket, open them in Data Studio and click a row. The trace viewer shows the session timeline, prompts, assistant messages, tool calls, and results. + +
+Agent trace session viewer showing user and assistant turns with expanded tool calls. +
+ +For a public example, open [`TeichAI/DeepSeek-v4-Pro-Agent`](https://huggingface.co/datasets/TeichAI/DeepSeek-v4-Pro-Agent). You can also browse more datasets tagged as [`traces`](https://huggingface.co/datasets?format=format%3Aagent-traces). diff --git a/docs/hub/datasets.md b/docs/hub/datasets.md index 36af847cda..4545e0bcb9 100644 --- a/docs/hub/datasets.md +++ b/docs/hub/datasets.md @@ -13,4 +13,5 @@ This documentation focuses on the datasets functionality in the Hugging Face Hub - [Downloading Datasets](./datasets-downloading) - [Libraries](./datasets-libraries) - [Dataset Viewer](./datasets-viewer) +- [Agent Traces](./datasets-agent-traces) - [Data files Configuration](./datasets-data-files-configuration) From 92378daf3c11a84574fe67e516ae1bc74ea79e01 Mon Sep 17 00:00:00 2001 From: Caleb Fahlgren Date: Thu, 21 May 2026 18:15:55 -0400 Subject: [PATCH 2/8] Update docs/hub/data-studio.md Co-authored-by: Pierric Cistac --- docs/hub/data-studio.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hub/data-studio.md b/docs/hub/data-studio.md index 7f46c96102..4cd7c43ae9 100644 --- a/docs/hub/data-studio.md +++ b/docs/hub/data-studio.md @@ -47,7 +47,7 @@ You can share a specific row by clicking on it, and then copying the URL in the ## Agent traces -Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset (or a [Storage Bucket](./storage-buckets)), and your dataset page will show a dedicated trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./datasets-agent-traces) for supported agents and viewing traces. +Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset (or a [Storage Bucket](./storage-buckets)) to automatically benefit from a dedicated trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./datasets-agent-traces) for supported agents and viewing traces. ## Large scale datasets From fcecc837807c8b14174e669cded7c61c24fc3a0b Mon Sep 17 00:00:00 2001 From: Caleb Fahlgren Date: Thu, 21 May 2026 22:59:16 -0400 Subject: [PATCH 3/8] Apply suggestions from code review Co-authored-by: Caleb Fahlgren --- docs/hub/datasets-agent-traces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hub/datasets-agent-traces.md b/docs/hub/datasets-agent-traces.md index 699c83485f..319431d208 100644 --- a/docs/hub/datasets-agent-traces.md +++ b/docs/hub/datasets-agent-traces.md @@ -9,7 +9,7 @@ Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the ## Find your traces -Each supported agent writes JSONL sessions to a known directory: +Each supported agent writes JSONL sessions to the following directories: | Agent | Local session directory | | ----------- | ----------------------- | From eeaa167d6c05d8cda2ddfe6f272027f7bd0cd338 Mon Sep 17 00:00:00 2001 From: Caleb Fahlgren Date: Fri, 22 May 2026 10:43:36 -0400 Subject: [PATCH 4/8] address agent traces review feedback --- docs/hub/data-studio.md | 30 +++++++++++++++--------------- docs/hub/datasets-agent-traces.md | 19 +++++++++++++++++-- 2 files changed, 32 insertions(+), 17 deletions(-) diff --git a/docs/hub/data-studio.md b/docs/hub/data-studio.md index 4cd7c43ae9..ade02a841f 100644 --- a/docs/hub/data-studio.md +++ b/docs/hub/data-studio.md @@ -47,7 +47,7 @@ You can share a specific row by clicking on it, and then copying the URL in the ## Agent traces -Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset (or a [Storage Bucket](./storage-buckets)) to automatically benefit from a dedicated trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./datasets-agent-traces) for supported agents and viewing traces. +Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset to browse them in Data Studio, or sync them to a [Storage Bucket](./storage-buckets) and open individual `.jsonl` files there. Both paths use the same trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./datasets-agent-traces) for supported agents and viewing traces. ## Large scale datasets @@ -105,12 +105,12 @@ You can specify which files to display in the Dataset Viewer by adding a YAML co ```yaml --- configs: - - config_name: default - data_files: - - split: train - path: "data.csv" - - split: test - path: "holdout.csv" +- config_name: default + data_files: + - split: train + path: "data.csv" + - split: test + path: "holdout.csv" --- ``` @@ -119,14 +119,14 @@ You can also select multiple files per split or use glob patterns: ```yaml --- configs: - - config_name: default - data_files: - - split: train - path: - - "data/train_part1.csv" - - "data/train_part2.csv" - - split: test - path: "data/*.csv" +- config_name: default + data_files: + - split: train + path: + - "data/train_part1.csv" + - "data/train_part2.csv" + - split: test + path: "data/*.csv" --- ``` diff --git a/docs/hub/datasets-agent-traces.md b/docs/hub/datasets-agent-traces.md index 319431d208..bd04464cb3 100644 --- a/docs/hub/datasets-agent-traces.md +++ b/docs/hub/datasets-agent-traces.md @@ -5,7 +5,7 @@ -Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a dataset or a [Storage Bucket](./storage-buckets) and Data Studio opens them in a dedicated trace viewer. +Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a dataset or a [Storage Bucket](./storage-buckets) to open them in a dedicated trace viewer. Datasets show traces in Data Studio; buckets let you open individual `.jsonl` files directly. ## Find your traces @@ -31,9 +31,24 @@ Buckets are especially useful if you want to keep syncing traces as new sessions +## Upload your traces + +After creating a dataset or bucket, install the `hf` CLI with the [recommended standalone installer](/docs/huggingface_hub/guides/cli#getting-started) and log in. If you want your coding agent to run `hf` commands for you, install the Hugging Face CLI skill with `hf skills add`. + +```bash +curl -LsSf https://hf.co/cli/install.sh | bash +hf auth login +hf skills add + +hf upload / ~/.codex/sessions . --repo-type dataset +hf buckets sync ~/.codex/sessions hf://buckets///codex +``` + +Replace `~/.codex/sessions` with the matching session directory for your agent. Use `hf upload` for datasets and `hf buckets sync` for buckets you want to update as new traces are written. The shorter `hf sync` command is an alias for `hf buckets sync`. + ## View your traces -Once you have traces in a dataset or bucket, open them in Data Studio and click a row. The trace viewer shows the session timeline, prompts, assistant messages, tool calls, and results. +Once you have traces in a dataset, open Data Studio and click a row. For traces in a Storage Bucket, navigate to the `.jsonl` file you want to inspect and open it. The trace viewer shows the session timeline, prompts, assistant messages, tool calls, and results.
Agent trace session viewer showing user and assistant turns with expanded tool calls. From ce8486d3b9ddf5fea2fd07315ea6a9a1a8751b06 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Sat, 30 May 2026 16:53:16 +0200 Subject: [PATCH 5/8] Apply suggestions from code review Co-authored-by: Julien Chaumond --- docs/hub/_toctree.yml | 2 +- docs/hub/data-studio.md | 2 +- docs/hub/datasets.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index 309d8655a9..135e60cec1 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -311,7 +311,7 @@ title: Embed the Dataset Viewer in a webpage - local: datasets-viewer-sql-console title: "SQL Console" - - local: datasets-agent-traces + - local: agent-traces title: Agent Traces - local: datasets-download-stats title: Datasets Download Stats diff --git a/docs/hub/data-studio.md b/docs/hub/data-studio.md index ade02a841f..7477cf790e 100644 --- a/docs/hub/data-studio.md +++ b/docs/hub/data-studio.md @@ -47,7 +47,7 @@ You can share a specific row by clicking on it, and then copying the URL in the ## Agent traces -Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset to browse them in Data Studio, or sync them to a [Storage Bucket](./storage-buckets) and open individual `.jsonl` files there. Both paths use the same trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./datasets-agent-traces) for supported agents and viewing traces. +Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset to browse them in Data Studio, or sync them to a [Storage Bucket](./storage-buckets) and open individual `.jsonl` files there. Both paths use the same trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./agent-traces) for supported agents and viewing traces. ## Large scale datasets diff --git a/docs/hub/datasets.md b/docs/hub/datasets.md index 4545e0bcb9..4a9607a26e 100644 --- a/docs/hub/datasets.md +++ b/docs/hub/datasets.md @@ -13,5 +13,5 @@ This documentation focuses on the datasets functionality in the Hugging Face Hub - [Downloading Datasets](./datasets-downloading) - [Libraries](./datasets-libraries) - [Dataset Viewer](./datasets-viewer) -- [Agent Traces](./datasets-agent-traces) +- [Agent Traces](./agent-traces) - [Data files Configuration](./datasets-data-files-configuration) From 1bed14954a9145de9898c8fbcbec509e08c9461e Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Sat, 30 May 2026 14:54:48 +0000 Subject: [PATCH 6/8] rename datasets-agent-traces to agent-traces to match toctree --- docs/hub/{datasets-agent-traces.md => agent-traces.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/hub/{datasets-agent-traces.md => agent-traces.md} (100%) diff --git a/docs/hub/datasets-agent-traces.md b/docs/hub/agent-traces.md similarity index 100% rename from docs/hub/datasets-agent-traces.md rename to docs/hub/agent-traces.md From 715fda477e4d7f716dcac65876da9c5dfcdeefc9 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Sat, 30 May 2026 14:56:58 +0000 Subject: [PATCH 7/8] Update agent-traces.md --- docs/hub/agent-traces.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hub/agent-traces.md b/docs/hub/agent-traces.md index bd04464cb3..62c66639e0 100644 --- a/docs/hub/agent-traces.md +++ b/docs/hub/agent-traces.md @@ -1,11 +1,11 @@ -# Agent traces +# Agent Traces
Agent trace preview on a dataset page in light mode.
-Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a dataset or a [Storage Bucket](./storage-buckets) to open them in a dedicated trace viewer. Datasets show traces in Data Studio; buckets let you open individual `.jsonl` files directly. +Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a [Dataset](https://huggingface.co/datasets?format=format%3Aagent-traces) or a [Storage Bucket](./storage-buckets) to open them in a dedicated trace viewer. Datasets show traces in Data Studio; buckets let you open individual `.jsonl` files directly. ## Find your traces From 13296a445bf840d7f8b86e62df396c07f54a5938 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Sat, 30 May 2026 14:57:40 +0000 Subject: [PATCH 8/8] link agent traces page from agents overview and index --- docs/hub/agents-overview.md | 1 + docs/hub/agents.md | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/hub/agents-overview.md b/docs/hub/agents-overview.md index d8a07c20b4..0be3830c58 100644 --- a/docs/hub/agents-overview.md +++ b/docs/hub/agents-overview.md @@ -9,6 +9,7 @@ Hugging Face provides tools and protocols that connect AI agents directly to the | [Skills](./agents-skills) | Task-specific guidance for AI/ML workflows | | [SDK](./agents-sdk) | Build agents programmatically with Python or JavaScript | | [Local Agents](./agents-local) | Run fully local agents with llama.cpp and Pi | +| [Agent Traces](./agent-traces) | View Claude Code, Codex, and Pi sessions on the Hub | ## Chat with Hugging Face diff --git a/docs/hub/agents.md b/docs/hub/agents.md index bc36c7c0a4..69a572cb4b 100644 --- a/docs/hub/agents.md +++ b/docs/hub/agents.md @@ -13,3 +13,4 @@ You can connect agents via the HF MCP Server, install pre-built Skills for codin - [Building agents with the HF SDK](./agents-sdk) - [Local Agents with llama.cpp and Pi](./agents-local) - [Agent Libraries](./agents-libraries) +- [Agent Traces](./agent-traces)