docs: add agent integration guide for AI coding tools

[kamran-rapidfireAI]

Summary

Make this repo's AI-coding-agent guidance work for every major tool — not just Claude Code — by hoisting the canonical maintainer guide to AGENTS.md and adding an end-user-facing integration guide at docs/AGENTS.md that agents discover before they help a user install or integrate the package.

This is the original docs/AGENTS.md work (commits 322fa22 and b24f5db) plus a follow-up restructure. The end-user-facing guide hasn't changed scope; what's new is where the maintainer guide lives, how each AI tool finds the right content, and how the two audiences are kept apart with zero cross-pollution risk.

What changed

Hoisted maintainer guide to AGENTS.md:

• AGENTS.md (new, root) — the full canonical maintainer guide. Holds the architecture, key commands, dev setup, common patterns, README guidelines, and the AGENTS.md sync triggers that previously lived in CLAUDE.md.
• CLAUDE.md (root) — reduced to a single line: @AGENTS.md. Claude Code's @<path> import (documented at https://code.claude.com/docs/en/memory.md) inlines the canonical content at launch. Same pattern applied to the six sub-module pairs in rapidfireai/fit/{backend,db,dispatcher,ml,utils} and rapidfireai/frontend.

Added end-user-facing integration guide (per the original PR scope):

• docs/AGENTS.md — the operational guide AI coding agents read before helping a user install or integrate rapidfireai. Placed under docs/ (not at root) so it does NOT auto-load for maintainer agents working in this repo.
• pyproject.toml — adds an AI-Agent-Guide URL so the guide surfaces in the PyPI sidebar.
• README.md — adds a collapsible <details> disclosure between the intro paragraph and ## Overview with a self-identifying header (🤖 AI coding agent installing or integrating rapidfireai? Read first). Humans see one line they can click to expand; agents reading the raw markdown see the full body unconditionally.

Per-tool MCP install instructions:

• rapidfireai/frontend/AGENTS.md — Playwright MCP install split into six tool-specific subsections (Claude Code / Cursor / OpenAI Codex CLI / GitHub Copilot / Windsurf / Aider+Junie). The original section had only the claude mcp add command, which works only for Claude Code.

Why each decision

Why hoist the maintainer guide to AGENTS.md (instead of leaving it in CLAUDE.md)?

AGENTS.md is the cross-tool convention auto-discovered by Cursor, OpenAI Codex CLI, GitHub Copilot, Windsurf, Aider, and Junie. CLAUDE.md is auto-discovered only by Claude Code. Before this PR, contributors using any non-Claude tool got zero auto-loaded guidance — they were either flying blind or skimming the end-user-oriented README. Hoisting to AGENTS.md gives every tool the same starting point with one source of truth.

Why CLAUDE.md as a one-line @AGENTS.md import (and not a duplicate)?

Two reasons:

1. Zero drift surface. A near-duplicate maintainer guide in CLAUDE.md would inevitably drift from AGENTS.md over time — someone updates one and forgets the other. A pure import cannot drift; there is only one file to maintain.
2. Documented mechanism. Claude Code's @<path> import is an officially documented feature, not a hack. The one-line file does exactly what's documented: import the referenced file into the working context at launch.

The risk is that a maintainer deletes the @AGENTS.md line thinking it's a leftover. Mitigated by the convention being applied uniformly across all seven router files (every CLAUDE.md in the repo is just @AGENTS.md), which signals the pattern is intentional.

Why is docs/AGENTS.md under docs/ and not at the repo root?

This is the load-bearing design decision. If we placed the end-user-integration guide at the root as AGENTS.md, non-Claude maintainer agents (Cursor / Codex CLI / Copilot / Windsurf) working in this repo would auto-load it and treat its content as project rules — but its content is for agents helping a user use the package, not maintain it. That cross-pollution would actively confuse contributors and is exactly the failure mode we are designing around.

Under docs/, the file is invisible to maintainer auto-loaders. End-user agents (who are NOT inside this repo — they're in the user's own project) still discover it via three independent channels: the README.md disclosure, the PyPI sidebar AI-Agent-Guide URL, and direct web search.

Why the <details> disclosure for the README pointer?

A plain blockquote adds visual weight humans don't need on every README read. A <details> block collapses to a single line — humans skip past unless they choose to expand, while agents reading the raw markdown source see the full body unconditionally (the <details> tag is just markup; the body inside is plain markdown).

The summary header 🤖 AI coding agent installing or integrating rapidfireai? Read first is tuned for three agent-discoverability signals:

• Audience identification (AI coding agent)
• Task vocabulary match (installing or integrating rapidfireai — including the project name itself, which is the strongest possible match for an agent given the task "install rapidfireai")
• Action verb (Read first — imperative with precedence)

For humans, the question form acts as a filter: "I'm not an AI coding agent" → skip past in one glance. The 🤖 emoji is a visual marker that reinforces the skip decision before words are even parsed.

Why split Playwright MCP per tool?

The original section had only claude mcp add playwright npx @playwright/mcp@latest, which works only for Claude Code. A contributor using Cursor, Codex CLI, Copilot, Windsurf, or any other MCP-capable tool would have to look up the equivalent config themselves. Per-tool subsections give each tool the correct setup (CLI command for Claude Code, JSON config snippets for Cursor / Copilot / Windsurf, TOML for Codex CLI, and a note about MCP-support variance for Aider/Junie).

Why decouple volatile install commands from docs/AGENTS.md?

README.md owns volatile install content (Python version, the pip install line, HF auth syntax, the hf-xet workaround, port numbers, kill-by-port commands, ssh -L syntax). These values change between releases.

docs/AGENTS.md owns the stable layer: workflow taxonomy (RAG vs fine-tuning), API patterns, safety rules, the workflow decision tree.

This split means a README install-step change does NOT require an AGENTS.md update — the agent guide cannot silently drift away from install reality, because it never restated the install steps to begin with. The reverse direction (when stable surfaces change) is covered by ## Keeping docs/AGENTS.md in sync inside root AGENTS.md, which lists specific triggers: Experiment public API, AutoML exports, trainer-type taxonomy, user-supplied callback signatures, CLI subcommand structure, and tutorial taxonomy.

Discovery matrix (post-PR)

Audience	Where the agent runs	What auto-loads	How they reach the canonical content
End-user agent helping a user install/use rapidfireai	User's own project (NOT this repo)	nothing from this repo	README disclosure → docs/AGENTS.md; or PyPI AI-Agent-Guide URL
Maintainer agent on Claude Code	This repo	CLAUDE.md	@AGENTS.md import inlines root AGENTS.md
Maintainer agent on Cursor / Codex CLI / Copilot / Windsurf / Aider / Junie	This repo	root AGENTS.md	Direct

No matrix cell can pollute another: maintainer auto-loaders can't see docs/AGENTS.md (wrong directory), and end-user agents can't see the maintainer files (they're not in this repo).

Test plan

• README renders correctly on GitHub — single <details> disclosure between the intro paragraph and ## Overview; collapsed by default showing only the 🤖 AI coding agent... summary line; expands to show the full pointer, quick-decision shortcut, and contributor-redirect note. No separate "For AI Coding Agents" section elsewhere in the README (deliberately merged into the disclosure).
• AGENTS.md (root) renders correctly on GitHub — full canonical maintainer guide present; scope guard for end-user audience visible at the top.
• docs/AGENTS.md renders correctly — tables and code blocks intact; contributor-redirect at the top points at root AGENTS.md.
• All CLAUDE.md routers are minimal — git diff -- '*CLAUDE.md' shows each file shrunk to a single @AGENTS.md line.
• Claude Code import works — open this repo in Claude Code, confirm canonical AGENTS.md content appears in context (not just the one-line @AGENTS.md text).
• Maintainer auto-load (positive) — open Cursor / Codex CLI / Copilot / Windsurf in this repo, ask a maintainer-flavored question, confirm root AGENTS.md content is treated as project rules.
• Maintainer auto-load (negative) — same tools must NOT auto-load docs/AGENTS.md as project rules (it should be invisible because it's under docs/, not at root).
• End-user agent smoke test — in a fresh project (NOT this repo), ask Cursor or Claude Code to install rapidfireai for fine-tuning; verify the agent fetches docs/AGENTS.md, validates the environment, picks the correct init variant, and walks the user through the §3 setup order. (Partially tested. Because this PR is not merged yet, I provided the AGENTS.md to the Cursor agent myself and it succeeded in integrating rapidfireai into a new notebook).
• Per-tool MCP install instructions render — in rapidfireai/frontend/AGENTS.md, all six tool subsections under "Installing Playwright MCP" render with their respective config snippets.
• PyPI sidebar URL appears after release (post-merge, optional) — AI-Agent-Guide link visible on the PyPI project page after the next TestPyPI deploy.

🤖 Generated with Claude Code

[kamran-rapidfireAI]

Closing in favor of four single-purpose PRs, reconstructed cleanly off main:

• Adopt AGENTS.md convention for maintainer docs #280 — Adopt the AGENTS.md convention for maintainer docs (root + per-module AGENTS.md, with each CLAUDE.md reduced to an @AGENTS.md pointer)
• Add the AI-agent setup guide and make it discoverable (docs/AGENTS.md, README, PyPI) #281 — Add the AI-agent install & setup guide (docs/AGENTS.md) + the PyPI AI-Agent-Guide URL
• Make the AI-agent setup guide discoverable from the README #282 — Add the AI coding agent notice to README.md
• Add code-integration guide for AI agents (optional, stacked on install guide) #283 — (optional, draft, stacked on Add the AI-agent setup guide and make it discoverable (docs/AGENTS.md, README, PyPI) #281) Add the code-integration layer to docs/AGENTS.md

#280 + #281 + #282 form a complete, self-contained set. #283 is purely additive — if it's never merged, the other three remain accurate and consistent (no dangling links, no over-promising). Splitting this way keeps each change reviewable on its own and lets the integration guide be accepted or dropped independently.

🤖 Generated with Claude Code