Commit c9dc62d
committed
Refactor AICI to use WebAssembly Component Model
This is a significant change to the AICI Runtime (host) and AICI Controller
(guest) to use WASI components. As part of this change, a significant amount of
unsafe code is removed and the protocol is simplified to remove the need for
"BLOB" types and side-channels.
The protocol is documented in `wit/controller.wit`, and a minimal WASI runtime
is provided to AI CI controllers.
Some notes:
* The AICI runtime now longer directly reads and writes to the guest's memory.
Instead, the guest provides a `Runner` resource (using WebAssembly Component
terminology), which exposes the low-level protocol to the host as a
constructor and trait with methods.
* The Blob protocols are removed entirely, replaced by the `Runner` resource.
This and other side-channels for communicating with the runtime, e.g. allowed
tokens (logit biases) outside of `MidProcessResult`, are removed.
* The (Variable) Storage and Tokenizer protocols are separate WebAssembly
Components, which can be versioned independently of the runtime.
* Types are changed to be consistent with the WebAssembly interface, e.g.:
`SeqId` is used in far more places to avoid casts.1 parent eec6e52 commit c9dc62d
74 files changed
Lines changed: 2208 additions & 2028 deletions
File tree
- .devcontainer
- .github/workflows
- aicirt
- src
- controllers
- aici_abi
- .cargo
- src
- aici_native/src
- aici_wasm_guest
- src
- aici_wasm_host
- src
- declctrl
- .cargo
- src
- guidance_ctrl/src
- jsctrl
- .cargo
- src
- llguidance_ctrl/.cargo
- pyctrl
- .cargo
- src
- uppercase
- .cargo
- src
- yesno
- .cargo
- src
- py
- pyaici
- tests
- rllm
- llama-cpp-low
- rllm-base/src
- iface
- server
- rllm-cuda/src/llm
- paged
- rllm-llamacpp/src/llamacpp
- scripts
- wit
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
0 commit comments