feat: add validation framework with per-user API key auth

Adds the Altimate validation framework to altimate-code.

CLI commands:
- `altimate-code validate configure --api-key <key>` — registers API key
  with validation server, saves to ~/.altimate-code/settings.json
- `altimate-code validate install` — manually install /validate skill
- `altimate-code validate status` — check installation state

Auto-setup on every InstanceBootstrap (startup):
- Writes SKILL.md + batch_validate.py to ~/.claude/skills/validate/
- Writes logger hook to ~/.claude/hooks/altimate_logger_hook.py
- Registers hook in ~/.claude/settings.json under hooks.Stop
- Uses bare identifier references (declare const) for build-time define
  constants so Bun's bundler correctly inlines asset content into the binary

Conversation logger (conversation-logger.ts):
- Subscribes to SessionStatus idle events after each turn
- Posts user prompt, tool calls, and assistant response to /log-conversation
- Fire-and-forget — never blocks the session
- Disabled via ALTIMATE_LOGGER_DISABLED=true

/validate skill routing:
- session.ts resolvePrompt() intercepts prompts starting with /validate
  and dispatches as SessionPrompt.command instead of a normal chat turn

Server-side two-tier auth (validationframework):
- Static shared token for /log-conversation (write-only logging)
- Per-user registered API keys for /validate* endpoints
- POST /auth/register adds api_key to registered_keys.json flat list

Docs:
- README: Validation section covering what it does, setup steps,
  natural language usage examples, and what happens without configuration
- docs/docs/configure/logging.md: detailed data collection reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: Bharatram-altimate-ai

README.md

@@ -157,6 +157,64 @@ Full docs at **[altimate.ai](https://altimate.ai)**.
 - [Agent Modes](https://altimate.ai/data-engineering/agent-modes/)
 - [Configuration](https://altimate.ai/configure/model-providers/)
 
+## Validation
+
+The `/validate` skill lets you audit past AI agent sessions against a set of quality criteria — checking whether the agent's reasoning, tool calls, and final response were correct, grounded, and complete. It pulls conversation traces from the backend, runs them through an evaluation pipeline, and reports per-criterion pass/fail results with details.
+
+You can validate:
+- **A single trace**: `/validate <trace_id> or /validate the trace <trace-id>`
+- **All traces in a session**: `/validate --session-id <session_id> or /validate all the traces in session id <session-id>`
+- **A date range for a user**: `/validate --from <datetime> --to <datetime> --user-id <user_id> or /validate for user id <user id> for <relative duration>/ from <start date time> to <end date time>`
+
+### Setup
+
+**1. Register your API key**
+
+```bash
+altimate-code validate configure --api-key <your-key>
+```
+
+The api key is got from your altimate account API KEY.
+
+**2. That's it** — the skill files are installed automatically the next time you start `altimate-code`.
+
+To verify the installation:
+
+```bash
+altimate-code validate status
+```
+
+To install manually without restarting:
+
+```bash
+altimate-code validate install
+```
+
+### What happens if you skip configuration
+
+If you run `/validate` without configuring an API key first, the validation script will exit immediately with:
+
+```
+ERROR: Altimate credentials not found.
+Run: altimate validate configure --api-key <key>
+```
+
+No traces will be validated and nothing will be written. You must run `altimate-code validate configure` at least once before using the skill.
+
+## Data Collection
+
+Altimate Code logs conversation turns (prompt, tool calls, and assistant response) to improve validation quality and agent behavior. Logs are sent to Altimate's backend and are not shared with third parties.
+
+**To opt out:**
+
+```bash
+export ALTIMATE_LOGGER_DISABLED=true
+```
+
+Add it to your shell profile (`~/.zshrc`, `~/.bashrc`) to make it permanent.
+
+See [`docs/docs/configure/logging.md`](docs/docs/configure/logging.md) for details on what is collected.
+
 ## Community & Contributing
 
 - **Issues**: [GitHub Issues](https://github.com/AltimateAI/altimate-code/issues)

bun.lock

@@ -0,0 +1,60 @@
+# Conversation Logging
+
+Altimate Code automatically logs each conversation turn to the Altimate backend. This powers validation, audit, and quality analysis features. Logging is **enabled by default** — no configuration is required to activate it.
+
+## What Is Logged
+
+Each turn (one user prompt + all assistant responses) sends the following to the Altimate backend:
+
+| Field | Description |
+|-------|-------------|
+| `session_id` | The current session identifier |
+| `conversation_id` | The assistant message ID for this turn |
+| `user_id` | Your email or username (from your Altimate account) |
+| `user_prompt` | The text of your message |
+| `parts` | All reasoning, text, and tool call/response parts from the assistant |
+| `final_response` | The last text response from the assistant |
+| `metadata` | Model ID, token counts, and cost for the turn |
+
+Logging fires after the session becomes idle (i.e., after the assistant finishes responding). Up to 500 messages are captured per turn to ensure complete coverage of multi-step agentic sessions.
+
+## Why We Log
+
+Conversation logs are used to:
+
+- **Validate AI responses** — power the `/validate` skill that audits factual claims against source data
+- **Quality analysis** — identify recurring failure patterns across sessions
+- **Audit trails** — provide a record of what the assistant did and why
+
+## Disabling Logging
+
+Logging is on by default. To disable it, set the following environment variable before starting Altimate Code:
+
+```bash
+export ALTIMATE_LOGGER_DISABLED=true
+```
+
+To make this permanent, add it to your shell profile (`~/.zshrc`, `~/.bashrc`, etc.):
+
+```bash
+echo 'export ALTIMATE_LOGGER_DISABLED=true' >> ~/.zshrc
+source ~/.zshrc
+```
+
+To re-enable logging, unset the variable:
+
+```bash
+unset ALTIMATE_LOGGER_DISABLED
+```
+
+Setting `ALTIMATE_LOGGER_DISABLED=false` is equivalent to not setting it — logging will be active.
+
+## Network
+
+Conversation logs are sent to:
+
+| Endpoint | Purpose |
+|----------|---------|
+| `apimi.tryaltimate.com` | Conversation log ingestion |
+
+Requests are fire-and-forget — a failed log request does not affect your session in any way.

docs/docs/configure/logging.md

@@ -71,6 +71,15 @@ const migrations = await Promise.all(
 )
 console.log(`Loaded ${migrations.length} migrations`)
 
+// Load validate skill assets for embedding
+const validateSkillMd = await Bun.file(path.join(dir, "src/skill/validate/SKILL.md")).text()
+const validateBatchPy = await Bun.file(path.join(dir, "src/skill/validate/batch_validate.py")).text()
+console.log("Loaded validate skill assets")
+
+// Load logger hook for embedding
+const loggerHookPy = await Bun.file(path.join(dir, "src/skill/validate/logger_hook.py")).text()
+console.log("Loaded logger hook")
+
 const singleFlag = process.argv.includes("--single")
 const baselineFlag = process.argv.includes("--baseline")
 const skipInstall = process.argv.includes("--skip-install")
@@ -224,6 +233,9 @@ for (const item of targets) {
       OPENCODE_LIBC: item.os === "linux" ? `'${item.abi ?? "glibc"}'` : "undefined",
       OPENCODE_MIGRATIONS: JSON.stringify(migrations),
       OPENCODE_CHANGELOG: JSON.stringify(changelog),
+      ALTIMATE_VALIDATE_SKILL_MD: JSON.stringify(validateSkillMd),
+      ALTIMATE_VALIDATE_BATCH_PY: JSON.stringify(validateBatchPy),
+      ALTIMATE_LOGGER_HOOK_PY: JSON.stringify(loggerHookPy),
       OPENCODE_WORKER_PATH: workerPath,
       OTUI_TREE_SITTER_WORKER_PATH: bunfsRoot + workerRelativePath,
     },

packages/opencode/script/build.ts

@@ -0,0 +1,163 @@
+import type { Argv } from "yargs"
+import { cmd } from "./cmd"
+import * as prompts from "@clack/prompts"
+import fs from "fs/promises"
+import path from "path"
+import os from "os"
+
+const BASE_URL = "https://apimi.tryaltimate.com"
+
+function getAltimateDotDir(): string {
+  return path.join(os.homedir(), ".altimate-code")
+}
+
+async function readSettings(): Promise<Record<string, unknown>> {
+  const settingsPath = path.join(getAltimateDotDir(), "settings.json")
+  try {
+    return JSON.parse(await fs.readFile(settingsPath, "utf-8"))
+  } catch {
+    return {}
+  }
+}
+
+async function writeSettings(settings: Record<string, unknown>): Promise<void> {
+  const dir = getAltimateDotDir()
+  await fs.mkdir(dir, { recursive: true })
+  await fs.writeFile(path.join(dir, "settings.json"), JSON.stringify(settings, null, 2))
+}
+
+// Injected at build time by build.ts (same pattern as ALTIMATE_CLI_MIGRATIONS).
+// In development these fall back to reading from disk via getAssets().
+declare const ALTIMATE_VALIDATE_SKILL_MD: string
+declare const ALTIMATE_VALIDATE_BATCH_PY: string
+
+interface ValidateAssets {
+  skillMd: string
+  batchPy: string
+}
+
+async function getAssets(): Promise<ValidateAssets> {
+  if (
+    typeof ALTIMATE_VALIDATE_SKILL_MD !== "undefined" &&
+    typeof ALTIMATE_VALIDATE_BATCH_PY !== "undefined"
+  ) {
+    return {
+      skillMd: ALTIMATE_VALIDATE_SKILL_MD,
+      batchPy: ALTIMATE_VALIDATE_BATCH_PY,
+    }
+  }
+  // Development fallback: read from disk relative to this source file
+  const skillsDir = path.join(import.meta.dir, "../../skill/validate")
+  const [skillMd, batchPy] = await Promise.all([
+    fs.readFile(path.join(skillsDir, "SKILL.md"), "utf-8"),
+    fs.readFile(path.join(skillsDir, "batch_validate.py"), "utf-8"),
+  ])
+  return { skillMd, batchPy }
+}
+
+
+
+const InstallSubcommand = cmd({
+  command: "install",
+  describe: "install the /validate skill into ~/.altimate-code",
+  handler: async () => {
+    prompts.intro("Altimate Validate — Installer")
+
+    const { skillMd, batchPy } = await getAssets()
+
+    const spinner = prompts.spinner()
+    spinner.start("Installing /validate skill...")
+    const skillTargetDir = path.join(os.homedir(), ".altimate-code", "skills", "validate")
+    await fs.mkdir(skillTargetDir, { recursive: true })
+    await fs.writeFile(path.join(skillTargetDir, "SKILL.md"), skillMd)
+    await fs.writeFile(path.join(skillTargetDir, "batch_validate.py"), batchPy)
+    spinner.stop(`Installed /validate skill → ${skillTargetDir}`)
+
+    prompts.outro("Altimate validation skill installed successfully!")
+  },
+})
+
+const StatusSubcommand = cmd({
+  command: "status",
+  describe: "check whether the /validate skill is installed",
+  handler: async () => {
+    const skillDir = path.join(os.homedir(), ".altimate-code", "skills", "validate")
+
+    prompts.intro("Altimate Validate — Installation Status")
+
+    const check = (exists: boolean, label: string, detail: string) =>
+      prompts.log.info(`${exists ? "✓" : "✗"} ${label}${exists ? "" : " (not found)"}: ${detail}`)
+
+    const skillMdExists = await fs.access(path.join(skillDir, "SKILL.md")).then(() => true).catch(() => false)
+    const batchPyExists = await fs.access(path.join(skillDir, "batch_validate.py")).then(() => true).catch(() => false)
+    check(skillMdExists && batchPyExists, "/validate skill", skillDir)
+
+    prompts.outro("Done")
+  },
+})
+
+const ConfigureSubcommand = cmd({
+  command: "configure",
+  describe: "register your Altimate API key to enable /validate",
+  builder: (yargs: Argv) =>
+    yargs.option("api-key", { type: "string", description: "Your Altimate API key" }),
+  handler: async (args) => {
+    prompts.intro("Altimate Validate — Configure")
+
+    const apiKey =
+      (args["api-key"] as string | undefined) ||
+      ((await prompts.text({
+        message: "Enter your Altimate API key:",
+        placeholder: "8a5b279d...",
+        validate: (v) => ((v ?? "").trim().length > 0 ? undefined : "API key is required"),
+      })) as string)
+
+    if (prompts.isCancel(apiKey)) {
+      prompts.cancel("Cancelled.")
+      process.exit(0)
+    }
+
+    const spinner = prompts.spinner()
+    spinner.start("Registering with validation server...")
+
+    try {
+      const res = await fetch(`${BASE_URL}/auth/register`, {
+        method: "POST",
+        headers: { "Content-Type": "application/json" },
+        body: JSON.stringify({ api_key: apiKey }),
+      })
+
+      if (!res.ok) {
+        const body = await res.text()
+        spinner.stop("Registration failed.")
+        prompts.log.error(`Server returned ${res.status}: ${body}`)
+        process.exit(1)
+      }
+
+      spinner.stop("Registered with validation server.")
+    } catch (err) {
+      spinner.stop("Could not reach validation server.")
+      prompts.log.warn(`Warning: ${err}. Credentials saved locally anyway.`)
+    }
+
+    // Save credentials to ~/.altimate-code/settings.json
+    const settings = await readSettings()
+    settings.altimate_api_key = apiKey
+    await writeSettings(settings)
+
+    prompts.log.success(`Credentials saved to ${path.join(getAltimateDotDir(), "settings.json")}`)
+    prompts.outro("Configuration complete. You can now run /validate.")
+  },
+})
+
+export const ValidateCommand = cmd({
+  command: "validate",
+  describe: "manage the Altimate validation framework (/validate skill)",
+  builder: (yargs: Argv) =>
+    yargs
+      .command(InstallSubcommand)
+      .command(StatusSubcommand)
+      .command(ConfigureSubcommand)
+      .demandCommand(),
+  handler: () => {},
+})

packages/opencode/src/cli/cmd/validate.ts

@@ -30,6 +30,7 @@ import { WebCommand } from "./cli/cmd/web"
 import { PrCommand } from "./cli/cmd/pr"
 import { SessionCommand } from "./cli/cmd/session"
 import { DbCommand } from "./cli/cmd/db"
+import { ValidateCommand } from "./cli/cmd/validate"
 import path from "path"
 import { Global } from "./global"
 import { JsonMigration } from "./storage/json-migration"
@@ -175,6 +176,7 @@ let cli = yargs(hideBin(process.argv))
   .command(PrCommand)
   .command(SessionCommand)
   .command(DbCommand)
+  .command(ValidateCommand)
 
 if (Installation.isLocal()) {
   cli = cli.command(WorkspaceServeCommand)