feat(agentic): Phase 3 — local in-process inference (epic #1544)#1552
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
More reviews will be available in 1 hour, 33 minutes, and 1 second. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more credits in the billing tab to continue. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (19)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
Deployment failed with the following error: Learn More: https://vercel.com/franklins-projects-02a0b5a0?upgradeToPro=build-rate-limit |
072ed9e to
978191c
Compare
cea00fd to
dec8f40
Compare
…ion) (#1544) The raw material every self-improvement mechanism learns from: - AgentTrajectory: a captured run (transcript, final answer, iterations, usage, agent name, optional evaluator-assigned Reward, metadata). - ITrajectoryStore (Add/Get/GetAll/Query/Clear) + InMemoryTrajectoryStore. - TracingAgent<T> (IAgent<T>): transparently records each run of any wrapped agent (executor/supervisor/swarm/memory) to the store without changing behavior; supports per-experiment metadata. - 5 tests (green net10.0 + net471): records runs, accumulates + queryable, metadata, get-by-id + reward annotation, clear. net471 nullability fix on GetAsync (Task.FromResult<AgentTrajectory?>). No null-forgiving. Stacked on Phase 3 (#1552). Next: eval harness scoring trajectories with the existing reward models, then learned routing / prompt-opt / online LoRA. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ion) (#1544) The raw material every self-improvement mechanism learns from: - AgentTrajectory: a captured run (transcript, final answer, iterations, usage, agent name, optional evaluator-assigned Reward, metadata). - ITrajectoryStore (Add/Get/GetAll/Query/Clear) + InMemoryTrajectoryStore. - TracingAgent<T> (IAgent<T>): transparently records each run of any wrapped agent (executor/supervisor/swarm/memory) to the store without changing behavior; supports per-experiment metadata. - 5 tests (green net10.0 + net471): records runs, accumulates + queryable, metadata, get-by-id + reward annotation, clear. net471 nullability fix on GetAsync (Task.FromResult<AgentTrajectory?>). No null-forgiving. Stacked on Phase 3 (#1552). Next: eval harness scoring trajectories with the existing reward models, then learned routing / prompt-opt / online LoRA. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dec8f40 to
3cb7e09
Compare
…ion) (#1544) The raw material every self-improvement mechanism learns from: - AgentTrajectory: a captured run (transcript, final answer, iterations, usage, agent name, optional evaluator-assigned Reward, metadata). - ITrajectoryStore (Add/Get/GetAll/Query/Clear) + InMemoryTrajectoryStore. - TracingAgent<T> (IAgent<T>): transparently records each run of any wrapped agent (executor/supervisor/swarm/memory) to the store without changing behavior; supports per-experiment metadata. - 5 tests (green net10.0 + net471): records runs, accumulates + queryable, metadata, get-by-id + reward annotation, clear. net471 nullability fix on GetAsync (Task.FromResult<AgentTrajectory?>). No null-forgiving. Stacked on Phase 3 (#1552). Next: eval harness scoring trajectories with the existing reward models, then learned routing / prompt-opt / online LoRA. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@coderabbitai full review |
✅ Action performedFull review finished. |
…ion) (#1544) The raw material every self-improvement mechanism learns from: - AgentTrajectory: a captured run (transcript, final answer, iterations, usage, agent name, optional evaluator-assigned Reward, metadata). - ITrajectoryStore (Add/Get/GetAll/Query/Clear) + InMemoryTrajectoryStore. - TracingAgent<T> (IAgent<T>): transparently records each run of any wrapped agent (executor/supervisor/swarm/memory) to the store without changing behavior; supports per-experiment metadata. - 5 tests (green net10.0 + net471): records runs, accumulates + queryable, metadata, get-by-id + reward annotation, clear. net471 nullability fix on GetAsync (Task.FromResult<AgentTrajectory?>). No null-forgiving. Stacked on Phase 3 (#1552). Next: eval harness scoring trajectories with the existing reward models, then learned routing / prompt-opt / online LoRA. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
|
@coderabbitai full review |
✅ Action performedFull review finished. |
The flagship local-first differentiator: an IChatClient<T> that runs entirely in-process over AiDotNet's own model — no network, no API key. Same agent code drives it as drives OpenAI/Anthropic. - ICausalLanguageModel<T>: minimal next-token-logits seam the real Transformer (or any model) plugs into; implementations may keep an internal KV-cache. - IGenerationTokenizer (+TokenizerGenerationAdapter over the repo's ITokenizer): encode/decode/EOS seam. - TokenSampler<T>: greedy (temp 0 = argmax) + temperature + top-k + top-p (nucleus), seedable for reproducibility; reads logits via Convert.ToDouble so float/double share one path. - IChatPromptTemplate + ChatMlPromptTemplate (role-tagged, opens the assistant turn). - LocalEngineChatClient<T>: renders prompt -> encodes -> autoregressively samples until EOS or token limit -> decodes; non-streaming + streaming (incremental decode deltas); usage = prompt/generated token counts. Drop-in for agents/supervisor/swarm/memory. (Native tool-calling + structured-output constraints via constrained decoding are a follow-up; tools are ignored this slice.) - 9 tests (green net10.0 + net471) on a deterministic ScriptedCausalModel + WordTokenizer: greedy decode + EOS stop, token-limit -> Length, streaming deltas reconstruct full text, AgentExecutor drop-in, sampler argmax/seed-reproducibility/top-k/top-p, ChatML template. No null-forgiving. Stacked on Phase 2 (#1551) — the integration test shows the local engine driving an AgentExecutor. Part of epic #1544 (Phase 3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ence (#1544) - NeuralNetworkCausalLanguageModel<T>: adapts a trained NeuralNetworkBase<T> LM (Mamba/GLA/Transformer-LM-head) to ICausalLanguageModel<T>. Encodes context as the one-hot [1,seq,vocab] tensor these models expect, runs a forward pass (ResetState first so recurrent models start fresh), and extracts the final position's logits (handles rank-2 and rank-3 outputs). Full context re-fed each step; KV-cached fast path is a follow-up. - 3 integration tests over a real tiny untrained MambaLanguageModel<double> (greedy, deterministic): adapter logit width + no-NaN, end-to-end generation honoring the token budget, streaming termination. Pinned to CpuEngine + non-parallel collection (documented GPU-autodetect mitigation). - Fixed Engine_IsDropInForAgentExecutor: it relied on the engine's stochastic default sampling while asserting a fixed output; now configures greedy (Temperature 0) so the assertion is deterministic. Root-caused the flake to default-temperature sampling, not a library/GPU issue. - Local suite green + stable (verified repeated runs) on net10.0 + net471. No null-forgiving. Part of epic #1544 (Phase 3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Native LOCAL guaranteed-structure generation — enforced at the logits, not merely prompted (a capability cloud models can't guarantee): - ITokenConstraint seam: AllowedNextTokens(generated) returns null (free) / a set (restrict) / empty (terminal -> stop). TokenSampler.Sample gains an allowed-token mask honored by both greedy (argmax within allowed) and stochastic (candidates restricted before top-k/top-p). - AllowedTokenSetConstraint (fixed allow-list) + FiniteStateTokenConstraint (finite-state grammar over token ids: start set + per-token transitions; terminal states stop) — the general mechanism a JSON-schema/regular grammar compiles to. - LocalEngineOptions.Constraint wires it into LocalEngineChatClient generation (non-streaming + streaming). - Stop sequences: ChatOptions.StopSequences now halt generation and trim the output at the earliest match (both paths). - 5 tests (green, stable, net10.0 + net471): allowed-set greedy/stochastic masking, FSA forces an exact JSON-shaped sequence despite a model biased to 'garbage', allow-list restricts whole output, stop-sequence halts+trims. No null-forgiving. Part of epic #1544 (Phase 3). JSON-schema->token-grammar compiler (to drive structured output / tool-calling off this framework) is the next follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- LocalEngineOptions.BeamWidth (>1) switches non-streaming GetResponseAsync to beam search: explores N hypotheses in parallel, expands each by its top tokens (honoring the ITokenConstraint and stop sequences per beam), prunes to the top-N by length-normalized log-probability, and returns the best completion. Deterministic; streaming stays token-by-token.
- LogSoftmax (with allowed-token masking → -inf) + length-normalized scoring + Beam bookkeeping.
- 3 tests (green net10.0 + net471): greedy takes the locally-best token ("A"); beam width 2 discovers the globally better "BC" path the same model would miss greedily; beam respects a finite-state constraint ("AB"). No null-forgiving (removed an introduced allowed! via proper narrowing).
Part of epic #1544 (Phase 3).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e fast path (#1544) - IIncrementalCausalLanguageModel<T> : ICausalLanguageModel<T> adds ResetCache / StartSequence(prompt) / AppendToken(id) — the contract a KV-cached model implements to advance one token at a time instead of re-feeding the full O(n^2) context. - LocalEngineChatClient auto-detects the interface and takes the incremental fast path (prime once, then feed single tokens), falling back to full re-feed otherwise. Constraint + stop-sequence + sampler logic shared via a single PickToken helper across both paths. - 2 tests (green net10.0 + net471): the engine drives the incremental path correctly (ResetCache once, StartSequence with the prompt, AppendToken per generated token) and produces output IDENTICAL to the full-refeed path. The remaining half — a real K/V cache inside Mamba/GLA/attention forward — is a model-layer change (the network Predict API has no incremental entry point yet); this lands the engine-side support + verification it plugs into. Part of epic #1544 (Phase 3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ompression (#1544) Quantization for the local engine is achieved by quantizing the NeuralNetworkBase<T> with the existing ModelCompression stack before wrapping it — the adapter accepts any such network unchanged. No engine-side code (no duplication of ModelCompression). Documented on NeuralNetworkCausalLanguageModel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
801f973 to
69a8271
Compare
…ion) (#1544) The raw material every self-improvement mechanism learns from: - AgentTrajectory: a captured run (transcript, final answer, iterations, usage, agent name, optional evaluator-assigned Reward, metadata). - ITrajectoryStore (Add/Get/GetAll/Query/Clear) + InMemoryTrajectoryStore. - TracingAgent<T> (IAgent<T>): transparently records each run of any wrapped agent (executor/supervisor/swarm/memory) to the store without changing behavior; supports per-experiment metadata. - 5 tests (green net10.0 + net471): records runs, accumulates + queryable, metadata, get-by-id + reward annotation, clear. net471 nullability fix on GetAsync (Task.FromResult<AgentTrajectory?>). No null-forgiving. Stacked on Phase 3 (#1552). Next: eval harness scoring trajectories with the existing reward models, then learned routing / prompt-opt / online LoRA. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…1556) * feat(agentic): Phase 4 — trajectory capture (self-improvement foundation) (#1544) The raw material every self-improvement mechanism learns from: - AgentTrajectory: a captured run (transcript, final answer, iterations, usage, agent name, optional evaluator-assigned Reward, metadata). - ITrajectoryStore (Add/Get/GetAll/Query/Clear) + InMemoryTrajectoryStore. - TracingAgent<T> (IAgent<T>): transparently records each run of any wrapped agent (executor/supervisor/swarm/memory) to the store without changing behavior; supports per-experiment metadata. - 5 tests (green net10.0 + net471): records runs, accumulates + queryable, metadata, get-by-id + reward annotation, clear. net471 nullability fix on GetAsync (Task.FromResult<AgentTrajectory?>). No null-forgiving. Stacked on Phase 3 (#1552). Next: eval harness scoring trajectories with the existing reward models, then learned routing / prompt-opt / online LoRA. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(agentic): Phase 4 — evaluation harness (score trajectories) (#1544) Turns captured runs into a measurable quality signal — the continuous-eval step the learners optimize against: - ITrajectoryEvaluator (trajectory-native scoring seam; higher = better) + DelegateTrajectoryEvaluator (custom scoring functions: exact-match, JSON validity, cost penalties, LLM-as-judge, or an adapter over the reasoning reward models). - EvaluationReport (count + mean/min/max reward + pass-rate at a threshold) — the scoreboard a self-improvement loop watches. - TrajectoryEvaluationRunner: scores trajectories (writing AgentTrajectory.Reward) and aggregates the report; EvaluateAsync(list) + EvaluateStoreAsync(store). - 9 tests total (green net10.0 + net471): exact-match grading + reward annotation, pass-rate threshold, whole-store eval with write-back, empty-set report. No null-forgiving. Reward models (OutcomeRewardModel/etc.) are reasoning-chain-specific + internal; they plug in via an ITrajectoryEvaluator adapter (follow-up) rather than coupling the harness to reasoning types. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(agentic): Phase 4 — learned routing (contextual reward-weighted bandit) (#1544) Replaces a hand-tuned router with one that learns from graded history which agent performs best — the orchestration improves as rewarded runs accumulate: - LearnedAgentRouter<T> (an IAgent<T>): LearnFrom(trajectories) builds per-agent (optionally per-context) mean-reward stats; routing exploits the best agent, explores unseen candidates first (optimistic), and takes a random agent with probability explorationRate. Optional contextKey learns different best-agents per task type. - Composable: drops in anywhere an agent is expected and, wrapped by TracingAgent, its routed runs feed back into the store — closing the self-improvement loop. - 5 tests (green net10.0 + net471): routes to highest-reward agent after learning, explores unseen first, learns per-context (math vs prose), closed-loop capture under the chosen agent, empty/duplicate guards. No null-forgiving. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(agentic): Phase 4 — prompt auto-optimization (DSPy-like) (#1544) Evaluation-driven prompt search: pick the system prompt that scores best on a labeled eval set. - PromptOptimizer<T>.OptimizeAsync(candidatePrompts, agentFactory, cases): builds an agent per candidate prompt, runs the eval cases, scores each, returns the best prompt + ranked candidates. Default scorer = case-insensitive substring match vs expected; overridable (e.g., reuse an ITrajectoryEvaluator / reward model / length penalty). - PromptEvalCase (input + expected) + ScoredPrompt + PromptOptimizationResult. - The optimizer is the SELECTION half; candidate prompts (the SEARCH half) can come from a hand-written set, an LLM proposer, or AiDotNet's existing genetic/beam/annealing prompt optimizers feeding their population in. - 4 tests (green net10.0 + net471): selects best prompt (ranked), custom scorer, single candidate, empty-input guards. Tests use a real AgentExecutor whose scripted model keys off the system prompt (no manual result construction, no null-forgiving). Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(agentic): Phase 4 — reward-filtered fine-tuning dataset (online LoRA data prep) (#1544) The data-preparation half of online LoRA self-improvement (reward-filtered behavior cloning): - RewardFilteredDatasetBuilder.Build(trajectories): keeps graded runs at/above a reward threshold and distills each into a (prompt, completion) pair — prompt = conversation up to the final turn, completion = final answer; skips ungraded/degenerate runs. - FineTuningExample (prompt/completion/reward) + FineTuningDataset (examples + mean reward) — ready to hand to the repo's LoRA/FineTuning trainer to produce an adapter for the local model. - 3 tests (green net10.0 + net471): keeps only high-reward runs (prompt excludes completion), skips ungraded/too-short, threshold controls inclusion. No null-forgiving. The actual LoRA training (dataset -> adapter on the local model) is the model-layer step performed by the existing trainer; this is the agentic step that decides WHAT to learn from. Part of epic #1544 (Phase 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: franklinic <franklin@ivorycloud.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Draft — do not review yet. Epic #1544 (Phase 3). Stacked on Phase 2 (#1551), which is stacked on Phase 0 (#1545). Rebase down the stack as the lower PRs merge.
Phase 3 goal — the flagship differentiator
Run inference in-process over AiDotNet's own model: no Python, no API, no key. An
IChatClient<T>implementation means the exact same agent code (executor / supervisor / swarm / memory) runs against a local model.Landed (slice 1 — the generation core)
ICausalLanguageModel<T>— minimal next-token-logits seam the real Transformer plugs into (free to keep an internal KV-cache).IGenerationTokenizer+TokenizerGenerationAdapterover the repo'sITokenizer.TokenSampler<T>— greedy / temperature / top-k / top-p (nucleus), seedable for reproducibility.IChatPromptTemplate+ChatMlPromptTemplate.LocalEngineChatClient<T>— render → encode → autoregressive sample (until EOS / token cap) → decode; non-streaming and streaming (incremental deltas); token-count usage.AgentExecutordrop-in.Planned next in this PR
Transformer/LM behindICausalLanguageModel<T>(real logits + KV-cache).No null-forgiving operators; enums for closed sets; "For Beginners" docs throughout.
🤖 Generated with Claude Code