diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index 01720e6add..135e60cec1 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -311,6 +311,8 @@ title: Embed the Dataset Viewer in a webpage - local: datasets-viewer-sql-console title: "SQL Console" + - local: agent-traces + title: Agent Traces - local: datasets-download-stats title: Datasets Download Stats diff --git a/docs/hub/agent-traces.md b/docs/hub/agent-traces.md new file mode 100644 index 0000000000..62c66639e0 --- /dev/null +++ b/docs/hub/agent-traces.md @@ -0,0 +1,57 @@ +# Agent Traces + +
+Agent trace preview on a dataset page in light mode. + +
+ +Agent traces from Claude Code, Codex, and Pi Agent are natively supported on the Hugging Face Hub. Upload the raw JSONL sessions to a [Dataset](https://huggingface.co/datasets?format=format%3Aagent-traces) or a [Storage Bucket](./storage-buckets) to open them in a dedicated trace viewer. Datasets show traces in Data Studio; buckets let you open individual `.jsonl` files directly. + +## Find your traces + +Each supported agent writes JSONL sessions to the following directories: + +| Agent | Local session directory | +| ----------- | ----------------------- | +| Claude Code | `~/.claude/projects` | +| Codex | `~/.codex/sessions` | +| Pi | `~/.pi/agent/sessions` | + +These trace files are supported out of the box, so you can upload them without modifying or converting them first. + +Trace files can include prompts, tool inputs, command output, local paths, screenshots, secrets, private code, and personal data. Review and redact traces before publishing them publicly, or keep the dataset or bucket private if you are not sure what is inside. + +For Pi Agent sessions, [`pi-share-hf`](https://github.com/badlogic/pi-share-hf) can help collect project sessions, redact known secrets, run TruffleHog and LLM review, and upload only sessions that pass checks. + + + +The easiest way to upload is to ask your agent itself: point it at the directory above and tell it to upload the `.jsonl` files to a Hub dataset or bucket. + +Buckets are especially useful if you want to keep syncing traces as new sessions land. + + + +## Upload your traces + +After creating a dataset or bucket, install the `hf` CLI with the [recommended standalone installer](/docs/huggingface_hub/guides/cli#getting-started) and log in. If you want your coding agent to run `hf` commands for you, install the Hugging Face CLI skill with `hf skills add`. + +```bash +curl -LsSf https://hf.co/cli/install.sh | bash +hf auth login +hf skills add + +hf upload / ~/.codex/sessions . --repo-type dataset +hf buckets sync ~/.codex/sessions hf://buckets///codex +``` + +Replace `~/.codex/sessions` with the matching session directory for your agent. Use `hf upload` for datasets and `hf buckets sync` for buckets you want to update as new traces are written. The shorter `hf sync` command is an alias for `hf buckets sync`. + +## View your traces + +Once you have traces in a dataset, open Data Studio and click a row. For traces in a Storage Bucket, navigate to the `.jsonl` file you want to inspect and open it. The trace viewer shows the session timeline, prompts, assistant messages, tool calls, and results. + +
+Agent trace session viewer showing user and assistant turns with expanded tool calls. +
+ +For a public example, open [`TeichAI/DeepSeek-v4-Pro-Agent`](https://huggingface.co/datasets/TeichAI/DeepSeek-v4-Pro-Agent). You can also browse more datasets tagged as [`traces`](https://huggingface.co/datasets?format=format%3Aagent-traces). diff --git a/docs/hub/agents-overview.md b/docs/hub/agents-overview.md index d8a07c20b4..0be3830c58 100644 --- a/docs/hub/agents-overview.md +++ b/docs/hub/agents-overview.md @@ -9,6 +9,7 @@ Hugging Face provides tools and protocols that connect AI agents directly to the | [Skills](./agents-skills) | Task-specific guidance for AI/ML workflows | | [SDK](./agents-sdk) | Build agents programmatically with Python or JavaScript | | [Local Agents](./agents-local) | Run fully local agents with llama.cpp and Pi | +| [Agent Traces](./agent-traces) | View Claude Code, Codex, and Pi sessions on the Hub | ## Chat with Hugging Face diff --git a/docs/hub/agents.md b/docs/hub/agents.md index bc36c7c0a4..69a572cb4b 100644 --- a/docs/hub/agents.md +++ b/docs/hub/agents.md @@ -13,3 +13,4 @@ You can connect agents via the HF MCP Server, install pre-built Skills for codin - [Building agents with the HF SDK](./agents-sdk) - [Local Agents with llama.cpp and Pi](./agents-local) - [Agent Libraries](./agents-libraries) +- [Agent Traces](./agent-traces) diff --git a/docs/hub/data-studio.md b/docs/hub/data-studio.md index e8bc5b7230..7477cf790e 100644 --- a/docs/hub/data-studio.md +++ b/docs/hub/data-studio.md @@ -27,14 +27,14 @@ You can search for a word in the dataset by typing it in the search bar at the t ## Run SQL queries on the dataset -You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files). +You can run SQL queries on the dataset in the browser using the SQL Console. This feature also leverages our [auto-conversion to Parquet](data-studio#access-the-parquet-files).
-For more information see our guide on [SQL Console](./datasets-viewer-sql-console). +For more information see our guide on [SQL Console](./datasets-viewer-sql-console). ## Share a specific row @@ -45,6 +45,10 @@ You can share a specific row by clicking on it, and then copying the URL in the +## Agent traces + +Push your raw JSONL traces from tools like Claude Code, Codex, or Pi into a dataset to browse them in Data Studio, or sync them to a [Storage Bucket](./storage-buckets) and open individual `.jsonl` files there. Both paths use the same trace viewer for stepping through sessions, turns, tool calls, and model responses. See [Agent Traces](./agent-traces) for supported agents and viewing traces. + ## Large scale datasets The Dataset Viewer supports large scale datasets, but depending on the data format it may only show the first 5GB of the dataset: diff --git a/docs/hub/datasets.md b/docs/hub/datasets.md index 36af847cda..4a9607a26e 100644 --- a/docs/hub/datasets.md +++ b/docs/hub/datasets.md @@ -13,4 +13,5 @@ This documentation focuses on the datasets functionality in the Hugging Face Hub - [Downloading Datasets](./datasets-downloading) - [Libraries](./datasets-libraries) - [Dataset Viewer](./datasets-viewer) +- [Agent Traces](./agent-traces) - [Data files Configuration](./datasets-data-files-configuration)