Skip to content

Commit c9dc62d

Browse files
committed
Refactor AICI to use WebAssembly Component Model
This is a significant change to the AICI Runtime (host) and AICI Controller (guest) to use WASI components. As part of this change, a significant amount of unsafe code is removed and the protocol is simplified to remove the need for "BLOB" types and side-channels. The protocol is documented in `wit/controller.wit`, and a minimal WASI runtime is provided to AI CI controllers. Some notes: * The AICI runtime now longer directly reads and writes to the guest's memory. Instead, the guest provides a `Runner` resource (using WebAssembly Component terminology), which exposes the low-level protocol to the host as a constructor and trait with methods. * The Blob protocols are removed entirely, replaced by the `Runner` resource. This and other side-channels for communicating with the runtime, e.g. allowed tokens (logit biases) outside of `MidProcessResult`, are removed. * The (Variable) Storage and Tokenizer protocols are separate WebAssembly Components, which can be versioned independently of the runtime. * Types are changed to be consistent with the WebAssembly interface, e.g.: `SeqId` is used in far more places to avoid casts.
1 parent eec6e52 commit c9dc62d

74 files changed

Lines changed: 2208 additions & 2028 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.devcontainer/Dockerfile-prod-vllm

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ FROM rust:1.75.0-bookworm AS aicirt
22

33
WORKDIR /workspace
44

5-
RUN rustup target add wasm32-wasi
5+
RUN rustup target add wasm32-wasip2
66
RUN curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
77
RUN apt-get install -y nodejs
88

.devcontainer/common.dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ ENV RUSTUP_HOME=/usr/local/rustup \
3131

3232
RUN curl https://sh.rustup.rs -sSf | sh -s -- \
3333
-y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION
34-
RUN rustup target add wasm32-wasi
34+
RUN rustup target add wasm32-wasip2
3535
RUN rustup component add rustfmt
3636

3737
# run as root please; note that settings in devcontainer.json are also needed...

.github/workflows/aicirt-release.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
- uses: actions/checkout@v3
2020
with:
2121
submodules: true
22-
- run: rustup target add wasm32-wasi
22+
- run: rustup target add wasm32-wasip2
2323
- uses: hendrikmuhs/ccache-action@v1.2
2424
- uses: Swatinem/rust-cache@v2
2525
with:

.github/workflows/aicirt.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
- uses: actions/checkout@v3
1919
with:
2020
submodules: true
21-
- run: rustup target add wasm32-wasi
21+
- run: rustup target add wasm32-wasip2
2222
- uses: hendrikmuhs/ccache-action@v1.2
2323
- uses: Swatinem/rust-cache@v2
2424
with:

0 commit comments

Comments
 (0)