feat(eval): validate trace (behavioral) expectations (expect.trace)

[kunalkushwaha]

What

Implements the previously stubbed expect.trace assertions in the eval runner. Today the runner has a literal // TODO: Validate trace expectations and the fully-typed TraceExpectation struct goes unused. This wires it up so eval tests can check how an agent reached its answer — not just the output text.

This is feature B2 (behavioral assertions) from the FEATURES.md roadmap.

How it works

After the content match passes, the runner fetches the run's trace from the EvalServer (GET /traces/{id}) and evaluates the assertions. Tool calls also use the tools_called field from the /invoke response, so tool_calls is still checked even if the trace fetch fails.

tests:
  - name: "Answers about Paris using search, efficiently"
    input: "What's the weather in Paris?"
    expect:
      type: contains
      values: ["Paris"]
      trace:
        tool_calls: ["search"]                  # each listed tool must have been called
        llm_calls: 2                             # exact LLM-call count
        execution_path: ["research", "format"]   # ordered subsequence of span names
        min_steps: 2
        max_steps: 8

Field	Check
tool_calls	every listed tool must appear (subset)
llm_calls	exact match when > 0
execution_path	listed names appear in order (gaps allowed)
min_steps / max_steps	observed step count within bounds

Changes

• internal/eval/trace_validator.go — ValidateTrace (pure) + buildObservedTrace normalizer + minimal decode types.
• internal/eval/http_target.go — FetchTrace(traceID) against GET /traces/{id}.
• internal/eval/runner.go — wires validation in after the content match (replaces the TODO).
• docs/EVAL.md — new "Trace (Behavioral) Assertions" section documenting the real expect.trace schema.

Testing

• go build, go vet, go test ./..., gofmt all green.
• First unit tests for the eval package: ValidateTrace (11 cases), buildObservedTrace (with/without a trace), isOrderedSubsequence, and an httptest-backed FetchTrace (success + error paths) — so the behavior is verified without a live LLM/EvalServer.

Independent branch off main (alongside #13/#14/#15). The canonical expectation schema lives in internal/eval/types.go.

🤖 Generated with Claude Code