Skip to content

Commit 3cb7e09

Browse files
franklinicclaude
andcommitted
docs(agentic): Phase 3 — document quantization composition via ModelCompression (#1544)
Quantization for the local engine is achieved by quantizing the NeuralNetworkBase<T> with the existing ModelCompression stack before wrapping it — the adapter accepts any such network unchanged. No engine-side code (no duplication of ModelCompression). Documented on NeuralNetworkCausalLanguageModel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 09f94be commit 3cb7e09

1 file changed

Lines changed: 6 additions & 0 deletions

File tree

src/Agentic/Models/Local/NeuralNetworkCausalLanguageModel.cs

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,12 @@ namespace AiDotNet.Agentic.Models.Local;
1919
/// and calls <see cref="NeuralNetworkBase{T}.ResetState"/> before each pass so recurrent models (Mamba/GLA)
2020
/// start fresh — correct, if not yet optimal. A KV-cached fast path is a planned follow-up.
2121
/// </para>
22+
/// <para>
23+
/// <b>Quantization</b> composes through this adapter rather than being re-implemented here: quantize the
24+
/// network with the repository's <c>ModelCompression</c> stack first, then wrap the quantized
25+
/// <see cref="NeuralNetworkBase{T}"/> — the adapter accepts any such network unchanged, so a smaller/faster
26+
/// model needs no engine-side code.
27+
/// </para>
2228
/// <para><b>For Beginners:</b> This is the bridge that lets the local chat engine talk to a real AiDotNet
2329
/// network. It turns the running list of tokens into the exact tensor shape the network expects, asks the
2430
/// network for its prediction, and hands back the scores for the next token — which the engine then samples

0 commit comments

Comments
 (0)