Commit 65df1a1
Port LLM inference callbacks to the .NET SDK
Mirrors the TypeScript LLM inference callback feature in the .NET SDK so
consumers can observe/mutate the model-layer HTTP/WebSocket requests the
runtime issues (CAPI and BYOK), with the runtime session id threaded into
each callback.
- scripts/codegen/csharp.ts: emit the clientGlobal handler interface +
registration so Rpc.cs gains the llmInference handler surface.
- LlmInferenceProvider.cs: low-level ILlmInferenceProvider API + adapter
(request staging, response sink state machine) behind an internal
ILlmInferenceResponseChannel seam for unit testing.
- LlmRequestHandler.cs: idiomatic pass-through base class mapping to
HttpRequestMessage/HttpResponseMessage and ClientWebSocket, with virtual
transform/forward hooks for both transports.
- Types.cs/Client.cs: wire LlmInferenceConfig into the client and register
the provider on start.
- Tests: factored unit-test infra (recording channel/sink, inline provider,
frame builders) with adapter + handler tests, plus CAPI+BYOK e2e tests
asserting the session id reaches the callback. e2e provider emits raw JSON
(reflection-free STJ) and serves all model-layer traffic off-network.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent f812ac2 commit 65df1a1
13 files changed
Lines changed: 2419 additions & 41 deletions
File tree
- dotnet
- src
- Generated
- test
- E2E
- Unit/LlmInference
- scripts/codegen
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
85 | 85 | | |
86 | 86 | | |
87 | 87 | | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
88 | 95 | | |
89 | 96 | | |
90 | 97 | | |
| |||
165 | 172 | | |
166 | 173 | | |
167 | 174 | | |
| 175 | + | |
| 176 | + | |
168 | 177 | | |
169 | 178 | | |
170 | 179 | | |
| |||
276 | 285 | | |
277 | 286 | | |
278 | 287 | | |
| 288 | + | |
| 289 | + | |
279 | 290 | | |
280 | 291 | | |
281 | 292 | | |
| |||
1674 | 1685 | | |
1675 | 1686 | | |
1676 | 1687 | | |
| 1688 | + | |
| 1689 | + | |
| 1690 | + | |
| 1691 | + | |
| 1692 | + | |
| 1693 | + | |
| 1694 | + | |
| 1695 | + | |
| 1696 | + | |
| 1697 | + | |
| 1698 | + | |
| 1699 | + | |
| 1700 | + | |
| 1701 | + | |
| 1702 | + | |
| 1703 | + | |
| 1704 | + | |
| 1705 | + | |
| 1706 | + | |
| 1707 | + | |
| 1708 | + | |
| 1709 | + | |
| 1710 | + | |
| 1711 | + | |
| 1712 | + | |
| 1713 | + | |
| 1714 | + | |
| 1715 | + | |
| 1716 | + | |
| 1717 | + | |
| 1718 | + | |
| 1719 | + | |
| 1720 | + | |
| 1721 | + | |
| 1722 | + | |
| 1723 | + | |
1677 | 1724 | | |
1678 | 1725 | | |
1679 | 1726 | | |
| |||
2067 | 2114 | | |
2068 | 2115 | | |
2069 | 2116 | | |
| 2117 | + | |
| 2118 | + | |
| 2119 | + | |
| 2120 | + | |
2070 | 2121 | | |
2071 | 2122 | | |
2072 | 2123 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
30 | 34 | | |
31 | 35 | | |
32 | 36 | | |
| |||
0 commit comments