Add code-integration guide for AI agents

Extend docs/AGENTS.md with code-integration patterns for the Experiment API
(RAG/eval, SFT, DPO, GRPO), the AutoML config-group reference, and the
experiment lifecycle, so AI coding agents can wire rapidfireai into a user's
own project, not just install it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: kamran-rapidfireAI

docs/AGENTS.md

@@ -1,6 +1,6 @@
-# RapidFire AI — Agent Install & Setup Guide
+# RapidFire AI — Agent Integration Guide
 
-Operational instructions for **AI coding agents** (Claude Code, Cursor, Codex, GitHub Copilot, Windsurf, Aider, Junie, and similar) that are helping an end user install, configure, and run the `rapidfireai` Python package.
+Operational instructions for **AI coding agents** (Claude Code, Cursor, Codex, GitHub Copilot, Windsurf, Aider, Junie, and similar) that are helping an end user install, configure, and integrate the `rapidfireai` Python package into the user's own project.
 
 This file is **not** for rapidfireai contributors. If you are working *on* rapidfireai itself, stop and read the repo's root [`AGENTS.md`](https://github.com/RapidFireAI/rapidfireai/blob/main/AGENTS.md) and [`CONTRIBUTING.md`](https://github.com/RapidFireAI/rapidfireai/blob/main/CONTRIBUTING.md) instead.
 
@@ -13,6 +13,7 @@ This guide does **not** restate version-specific install commands, package versi
 What this guide *does* provide that the README does not:
 
 - A workflow decision tree (RAG vs fine-tuning vs post-training; OpenAI vs self-hosted; lite vs full).
+- Code integration patterns for the `Experiment` API and the AutoML config groups.
 - Trainer-type taxonomy for fine-tuning (`SFT` / `DPO` / `GRPO`).
 - Safety rules for handling user secrets, GPU assumptions, and gated model access.
 
@@ -39,7 +40,7 @@ Pick a branch **before** running any install commands — the two workflows inst
 ### Environment selectors
 
 - **Remote / cloud machine** → an SSH port-forward is required to view the dashboard locally. The set of ports differs by workflow (smaller for fit, larger for evals because Jupyter and MLflow are also exposed). The current port set and the exact `ssh` command are in the README §Install — read them from there, do not memorize.
-- **GPU issues** (CUDA absent, OOM, driver mismatch) → run `rapidfireai doctor` and act on its output. For OOM, switch the user to a *lite* tutorial variant (see §6).
+- **GPU issues** (CUDA absent, OOM, driver mismatch) → run `rapidfireai doctor` and act on its output. For OOM, switch the user to a *lite* tutorial variant (see §9).
 - **Hugging Face permission issues** → confirm the user has run the README's HF auth step and has been granted access on the gated model's HF page. If access is blocked, suggest an open-license substitute (TinyLlama, Qwen-0.5B/3B) where licensing permits.
 
 The exact CLI flag name for the evals init mode and the exact set of port numbers are intentionally not restated here — see the README, which is the authoritative install reference.
@@ -60,7 +61,7 @@ Run the steps in the order below. The **commands** are in the README; the **deci
 8. **Run `rapidfireai doctor`** and confirm no critical failures (Python, GPU/CUDA where applicable, ports, packages). Do not proceed past failures without diagnosing the root cause.
 9. **Start the dashboard stack** with `rapidfireai start` (frontend, MLflow, dispatcher). For the RAG/evals workflow, also start a Jupyter listener as documented in the README — the evals tutorials run inside Jupyter rather than an arbitrary IDE notebook.
 10. **(Remote/cloud machines only) Port-forward** the workflow-appropriate set with `ssh` *before* clicking the dashboard URL. The current set and SSH syntax are in README §Install. VS Code typically auto-forwards the dashboard port; other clients require the explicit `ssh -L` invocation. Skip this step on a local machine.
-11. **Open the tutorial notebook** under `./tutorial_notebooks/` (see §6 for the filename matching the workflow). Two non-obvious requirements:
+11. **Open the tutorial notebook** under `./tutorial_notebooks/` (see §9 for the filename matching the workflow). Two non-obvious requirements:
     - Notebooks **cannot** run via `python notebook.ipynb` — there is a multiprocessing restriction. Open the notebook in an IDE that supports Jupyter (VS Code, Cursor, JetBrains), or use `rapidfireai jupyter` (mandatory for the RAG/evals tutorials).
     - The notebook's kernel **must** be set to the `.venv` you created in step 2, not the system Python; otherwise `import rapidfireai` fails. In VS Code / Cursor, click the kernel selector in the upper-right of the notebook and pick the venv interpreter.
     - The dashboard URL printed by `rapidfireai start` (or by `rapidfireai jupyter` for evals) shows live run progress as cells execute.
@@ -81,7 +82,119 @@ Run the steps in the order below. The **commands** are in the README; the **deci
 
 ---
 
-## 5. Troubleshooting
+## 5. Code integration patterns
+
+The four patterns below cover every supported workflow. Each is the *minimum* a user needs.
+
+### A. RAG / evaluation
+
+```python
+from rapidfireai import Experiment
+from rapidfireai.automl import RFGridSearch, List, RFvLLMModelConfig
+
+experiment = Experiment("my-rag-eval", mode="evals")
+
+config_group = RFGridSearch({
+    "vllm_config": List([vllm_config_a, vllm_config_b]),
+    "preprocess_fn": user_preprocess,
+    "postprocess_fn": user_postprocess,
+    "compute_metrics_fn": user_compute_metrics,
+    # Optional:
+    # "accumulate_metrics_fn": user_accumulate_metrics,
+})
+
+results = experiment.run_evals(config_group, dataset, num_shards=4)
+experiment.end()
+```
+
+For an **OpenAI / Azure / Anthropic** generator, swap `RFvLLMModelConfig` for `RFAPIModelConfig`. The user-supplied callbacks are:
+
+- `preprocess_fn(batch, rag, prompt_manager) -> dict` — assemble prompts (with retrieved context for RAG).
+- `postprocess_fn(batch) -> dict` — attach ground-truth / reference labels.
+- `compute_metrics_fn(batch) -> dict[str, dict]` — per-batch metrics (Precision, Recall, NDCG@k, MRR, …).
+- `accumulate_metrics_fn(aggregated) -> dict[str, dict]` — optional, aggregates per-batch metrics across shards.
+
+`run_evals` returns `dict[run_id, (aggregated_results, cumulative_metrics)]`.
+
+### B. SFT (supervised fine-tuning)
+
+```python
+from rapidfireai import Experiment
+from rapidfireai.automl import RFGridSearch
+
+experiment = Experiment("my-sft", mode="fit")
+
+config_group = RFGridSearch(configs=config_set, trainer_type="SFT")
+
+experiment.run_fit(
+    param_config=config_group,
+    create_model_fn=user_create_model,
+    train_dataset=train_ds,
+    eval_dataset=eval_ds,
+    num_chunks=4,
+)
+
+results_df = experiment.get_results()
+experiment.end()
+```
+
+User-supplied callbacks:
+
+- `create_model_fn(model_config) -> (model, tokenizer)` — required. Loads model and tokenizer from the per-run config dict.
+- `formatting_func(row) -> dict` — optional. Returned in `RFModelConfig`. Maps a dataset row to `{"prompt": ..., "completion": ...}`.
+- `compute_metrics(preds, labels) -> dict[str, float]` — optional. Returned in `RFModelConfig`.
+
+### C. DPO (preference alignment)
+
+Same shape as SFT but `trainer_type="DPO"`. The training dataset must have `chosen` and `rejected` columns (or whatever the chosen DPO config maps to). A reference model is handled internally; the user does not have to construct one explicitly.
+
+### D. GRPO (RL-based post-training)
+
+Same shape as SFT but `trainer_type="GRPO"`. Reward functions are passed via the trainer config, not as a separate argument. Use a small model and a small dataset for first runs — GRPO samples multiple generations per prompt and is significantly slower than SFT/DPO.
+
+---
+
+## 6. Config groups and search
+
+Imports:
+
+```python
+from rapidfireai.automl import RFGridSearch, RFRandomSearch, List, Range
+```
+
+- **Fit mode**: `RFGridSearch(configs=config_set, trainer_type="SFT" | "DPO" | "GRPO")` — exhaustive sweep. `trainer_type` selects the underlying TRL trainer.
+- **Evals mode**: `RFGridSearch({"vllm_config": List([...]), "preprocess_fn": ..., ...})` — dict form. **No `trainer_type`** (evals does inference, not training).
+- `RFRandomSearch(configs, num_runs, trainer_type=...)` — sampled variant. `trainer_type` is again fit-only.
+- `List([a, b, c])` and `Range(min=..., max=...)` define sweep dimensions and work in both modes.
+
+Fit-mode config classes (from `rapidfireai.automl`): `RFModelConfig`, `RFLoraConfig`, `RFSFTConfig`, `RFDPOConfig`, `RFGRPOConfig`. Evals-mode config classes: `RFvLLMModelConfig`, `RFAPIModelConfig`, plus helpers `RFLangChainRagSpec` and `RFPromptManager`.
+
+---
+
+## 7. Experiment lifecycle
+
+`rapidfireai start` (the CLI command) and `Experiment(...)` (the Python constructor) are distinct concerns:
+
+- `rapidfireai start` brings up the **visual stack** (frontend dashboard, MLflow tracking, dispatcher API) so the user can watch runs live.
+- `Experiment(...)` separately starts an in-process dispatcher thread for the API calls it needs internally.
+
+Users typically run `rapidfireai start` first (to get the dashboard), then run their notebook or script which constructs `Experiment(...)`. The CLI `start` is **not strictly required** for `Experiment(...)` to function — but without it the dashboard UI is unavailable.
+
+API surface (from `rapidfireai.experiment`):
+
+- `Experiment(name, mode="fit" | "evals")` — `mode` defaults to `"fit"`. `name` is required.
+- `run_fit(...)` — fit mode only. Blocking on non-Colab; async on Colab.
+- `run_evals(...)` — evals mode only. Returns `dict[run_id, (results, metrics)]`.
+- `get_results()` — fit mode only. Returns a pandas `DataFrame` of metrics.
+- `get_runs_info()` — fit mode only. Returns a pandas `DataFrame` of run metadata.
+- `cancel_current()` — both modes. Cancels an in-flight task.
+- `end()` — both modes. **Required** for clean shutdown of worker processes / Ray.
+
+Only `Experiment` is exported at the top level. AutoML primitives are imported from `rapidfireai.automl`.
+
+---
+
+## 8. Troubleshooting
 
 Match the symptom to the diagnostic direction. Specific commands (kill-by-port, SSH-forward, hf-xet uninstall, etc.) are in README §Troubleshooting and §Install — do not paste them from memory.
 
@@ -100,7 +213,7 @@ Match the symptom to the diagnostic direction. Specific commands (kill-by-port,
 
 ---
 
-## 6. Tutorials
+## 9. Tutorials
 
 After `rapidfireai init`, tutorial notebooks are copied to `./tutorial_notebooks/`. Always prefer the on-disk copy — it matches the installed version.
 
@@ -117,7 +230,7 @@ Online directory: <https://github.com/RapidFireAI/rapidfireai/tree/main/tutorial
 
 ---
 
-## 7. Validation checklist (run before reporting "done")
+## 10. Validation checklist (run before reporting "done")
 
 Before telling the user setup is complete, confirm each:
 
@@ -128,12 +241,14 @@ Before telling the user setup is complete, confirm each:
 5. `rapidfireai start` brought up the stack without errors. The user can reach the dashboard URL printed by `start`.
 6. (Remote/cloud) The `ssh -L` forward is active in a separate terminal.
 7. (Tutorial path) The notebook's kernel is the project `.venv`, not the system Python. The first import cell (`from rapidfireai import Experiment`) runs without `ImportError`.
+8. (Code-integration done) `Experiment(...)` constructs without raising; the user's `create_model_fn` (fit) or `preprocess_fn` (evals) runs on a single example without error.
+9. `experiment.end()` is called at the end of the user's script/notebook.
 
 If any check fails, stop and diagnose — do not paper over with retries.
 
 ---
 
-## 8. References
+## 11. References
 
 - **Canonical install instructions and troubleshooting**: <https://github.com/RapidFireAI/rapidfireai/blob/main/README.md>
 - **Full documentation**: <https://oss-docs.rapidfire.ai>