Skip to content

Commit b6b9149

Browse files
docs: add Validation section to README with setup and configure instructions
1 parent 5b32b88 commit b6b9149

File tree

1 file changed

+44
-0
lines changed

1 file changed

+44
-0
lines changed

README.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,50 @@ Full docs at **[altimate.ai](https://altimate.ai)**.
157157
- [Agent Modes](https://altimate.ai/data-engineering/agent-modes/)
158158
- [Configuration](https://altimate.ai/configure/model-providers/)
159159
160+
## Validation
161+
162+
The `/validate` skill lets you audit past AI agent sessions against a set of quality criteria — checking whether the agent's reasoning, tool calls, and final response were correct, grounded, and complete. It pulls conversation traces from the backend, runs them through an evaluation pipeline, and reports per-criterion pass/fail results with details.
163+
164+
You can validate:
165+
- **A single trace**: `/validate <trace_id>`
166+
- **All traces in a session**: `/validate --session-id <session_id>`
167+
- **A date range for a user**: `/validate --from <datetime> --to <datetime> --user-id <user_id>`
168+
169+
### Setup
170+
171+
**1. Register your API key**
172+
173+
```bash
174+
altimate-code validate configure --api-key <your-key>
175+
```
176+
177+
This registers your key with the Altimate validation server and saves it to `~/.altimate-code/settings.json`. The `/validate` skill reads credentials from that file when running evaluations.
178+
179+
**2. That's it** — the skill files are installed automatically the next time you start `altimate-code`.
180+
181+
To verify the installation:
182+
183+
```bash
184+
altimate-code validate status
185+
```
186+
187+
To install manually without restarting:
188+
189+
```bash
190+
altimate-code validate install
191+
```
192+
193+
### What happens if you skip configuration
194+
195+
If you run `/validate` without configuring an API key first, the validation script will exit immediately with:
196+
197+
```
198+
ERROR: Altimate credentials not found.
199+
Run: altimate validate configure --api-key <key>
200+
```
201+
202+
No traces will be validated and nothing will be written. You must run `altimate-code validate configure` at least once before using the skill.
203+
160204
## Data Collection
161205
162206
Altimate Code logs conversation turns (prompt, tool calls, and assistant response) to improve validation quality and agent behavior. Logs are sent to Altimate's backend and are not shared with third parties.

0 commit comments

Comments
 (0)