Summary

Adds the AIEVALS types — Runner, RunnerResult, Judge, and Evaluator — and wires Evaluator.noop() into all config types. Callers can now implement a Runner to wrap any model provider, construct a Judge to evaluate AI outputs against a judge prompt with structured {score, reasoning} output, and coordinate multiple judges through an Evaluator.

New types

public interface Runner {
  RunnerResult run(String input, Map<String, Object> outputType) throws Exception;
  default RunnerResult run(String input) throws Exception;
}

Wraps a model provider SDK. outputType carries a JSON-Schema-like map when structured output is needed. Single-arg overload delegates with outputType = null.

RunnerResult.builder(String content, AIMetrics metrics)
    .raw(Object raw)
    .parsed(Map<String, Object> parsed)
    .build();

Immutable result of a Runner invocation. parsed is defensively copied and returned as unmodifiable.

public Judge(AIJudgeConfig config, Runner runner, LDLogger logger);

JudgeResult evaluate(String input, String output);
JudgeResult evaluate(String input, String output, double samplingRate);
JudgeResult evaluateMessages(List<Message> messages, RunnerResult response);
JudgeResult evaluateMessages(List<Message> messages, RunnerResult response, double samplingRate);

Evaluates AI output by invoking a runner with a formatted evaluation prompt and parsing the structured response. Sampling gate runs first — below the rate, returns sampled=false immediately. Creates a fresh tracker per evaluation via config.createTracker(). Parses score (Number, [0.0, 1.0]) and reasoning (String, optional). Runner exceptions are caught and returned as JudgeResult(success=false) — judge failures are results, not exceptions. Does not call trackJudgeResult.

public static Evaluator noop();
public Evaluator(Map<String, Judge> judges, JudgeConfiguration judgeConfiguration, LDLogger logger);

CompletableFuture<List<JudgeResult>> evaluate(String input, String output);

Coordinates sequential execution of judges. Missing judges skipped with a warning. Evaluator.noop() returns a singleton whose evaluate immediately returns an empty list. For v1.0, all configs receive Evaluator.noop().

Config type changes

AIConfig base class gains an Evaluator field and getEvaluator() accessor. AICompletionConfig and AIAgentConfig constructors accept an Evaluator. AIJudgeConfig always wires Evaluator.noop() internally — judges do not evaluate themselves.

Test plan

• ./gradlew :lib:sdk:server-ai:test passes
• JudgeTest — successful evaluation, score boundary validation (0 and 1), reasoning optional, runner exception handling (caught not rethrown), null/missing parsed output, score out of range, sampling rates (0 always skips, 1 always runs), message formatting, getter accessors
• EvaluatorTest — noop returns empty list, noop singleton identity, single/multiple judge execution, missing judge skipped, evaluator does not call trackJudgeResult, returned future is already complete
• RunnerResultTest — builder field assignment, immutability, defensive copy of parsed map