Support bulk lowering for list<flat T> in dynamic component calls

[jeffparsons]

I have a use case where I want to store collections of canonical-ABI-shaped component values in host memory, and then pass them in bulk to guest component functions.

Crucially, the host has no static knowledge of the component model types involved. The relevant WIT/component types are discovered at runtime. The host also never directly constructs the individual elements in a type-aware way: they are produced by guests that do have static knowledge of those component model types, then stored by the host as opaque bytes associated with reflected component-model type information.

Right now, as far as I can tell, there is no component-oriented way to dynamically call a guest function with an argument like “a list<T> whose elements are already available as contiguous canonical-ABI-layout bytes,” without recursively constructing something like Val::List(Vec<Val>) and paying per-element dynamic lowering overhead.

What I’m imagining is a Wasmtime API, either as a new kind of dynamic call function or as a special dynamic value representation, that would support bulk lowering for a restricted case:

• the argument is a list<T>;
• T is known through Wasmtime’s reflected component type information;
• Wasmtime verifies that T is safe to bulk-copy in this way, for example because it has a fixed canonical ABI layout and contains no transitive strings, lists, resources, handles, or other pointer/ownership-bearing values;
• Wasmtime computes the element size/stride/alignment from its own canonical ABI metadata;
• the source byte slice length is checked against the computed element stride; and
• Wasmtime allocates/copies the data into the callee’s linear memory using the normal component/canonical-ABI machinery.

The goal is not to expose an unchecked “raw bytes can be any component value” escape hatch. I’m looking for a checked fast path for the flat/fixed-size case, where the current dynamic Val API is semantically appropriate but forces avoidable per-element boxing/lowering work for large lists.

Would the Wasmtime project be open to supporting this use case in some form?

This seems related to broader requests for lower-overhead dynamic component calls, but the intended scope here is narrower than a fully unchecked component-call API: only bulk lowering/lifting for list<T> where T is a reflected, fixed-size, canonical-ABI-flat element type.

Edit: My specific use case is about trying to make a Wasm Component native archetypal ECS, but I wonder if there might be other, less "fringe" use cases for such a mechanism.

[alexcrichton]

This is definitely something I'd be interested in supporting in Wasmtime! I've long lamented how the dynamic path in Wasmtime is basically too inefficient for a lot of use cases like yours. Historically though I've not had any good ideas about how to expose something that's a mid-point between "you do everything yourself" and the current API. I agree that it'd be quite desirable to have a middle-ground of checked-with-a-fast-path or something similar.

If you've got ideas of what this might look like to add to Wasmtime though I'd be happy to discuss it here and/or review a PR!

[jeffparsons]

I've done some brainstorming over here (warning/disclosure: heavy Claude use inside) and came up with a design for an embedder calling into a guest component instance that I think offers a decent amount of flexibility in how arguments are provided and return values are received. The full rationale if you're exceedingly curious — a cost breakdown, borrow-safety "proofs", and the API sketch — is in call-builder-sim/DESIGN.md.

First, component::Func would grow a new prepare_call method that accepts a specification of how each argument will be provided and (assuming success) returns a PreparedCall object that can be used for multiple calls. I'm proposing that this be a separate first step so that you can front-load validation that the user's nominated kind of source for each argument is actually compatible with that guest function parameter. (More on validation below.)

When you're ready to bind concrete arguments for a single invocation of the guest function, you would then call PreparedCall::bind to get a BoundCall object. You'd then repeatedly call BoundCall::arg (or a typed convenience like arg_flat) to bind a specific source for each argument, and finally call BoundCall::invoke to perform the call and take ownership of the results. There's also BoundCall::invoke_scoped, which instead hands a closure a borrowed view into the results (and both have _async counterparts for async calls).

Results are read by naming a sink type — get::<&[u8]> for a zero-copy view, get::<Vec<u8>> for an owned copy, or get::<Val> for a dynamic value — the mirror of naming a source when providing; view/copy/val are sugar for those.

This proposal covers only the host-calls-guest direction. I also have ideas for the mirror — functions the embedder exports for the guest to call — but I'm leaving those for later to focus on this direction first.

Here's what the new API surface might look like for an embedder calling into a guest:

impl component::Type {
    pub fn is_cabi_inline(&self) -> bool;
}

pub trait Producer {
    fn next_chunk(&mut self) -> Option<&[u8]>;
}

#[non_exhaustive]
pub enum ArgSpec {
    Val,
    Flat,
    Stream,
    TypedBuffer,
    Lowerable,
}

#[non_exhaustive]
pub enum ArgSource<'a> {
    Val(Val),
    Flat(&'a [u8]),
    Stream(&'a mut dyn Producer),
    TypedBuffer(...),
    Lowerable(...),
}

impl component::Func {
    pub fn prepare_call(&self, store: impl AsContext, args: &[ArgSpec]) -> Result<PreparedCall>;
}

impl PreparedCall {
    pub fn bind(&self) -> BoundCall<'_>;
}

impl<'a> BoundCall<'a> {
    pub fn arg(self, source: ArgSource<'a>) -> Self;
    pub fn arg_val(self, value: Val) -> Self;
    pub fn arg_flat(self, bytes: &'a [u8]) -> Self;
    pub fn arg_stream(self, producer: &'a mut dyn Producer) -> Self;

    pub fn invoke(self, store: impl AsContextMut) -> Result<Results<'static>>;
    pub fn invoke_scoped<R>(
        self,
        store: impl AsContextMut,
        f: impl FnOnce(&Results<'_>) -> Result<R>,
    ) -> Result<R>;
}

pub trait FromResult<'a>: Sized {
    fn from_result(results: &'a Results<'_>, index: usize) -> Result<Self>;
}

impl<'a> Results<'a> {
    pub fn len(&self) -> usize;
    pub fn get<'r, T: FromResult<'r>>(&'r self, index: usize) -> Result<T>;
    pub fn view(&self, index: usize) -> Result<&[u8]>;
    pub fn copy(&self, index: usize) -> Result<Vec<u8>>;
    pub fn val(&self, index: usize) -> Result<Val>;
}

Val, Flat, and Stream (which models the CM stream<T>) are the worked-out sources; Buffer (a slice of a host-side typed buffer, for collecting results out of one instance and replaying them into others) and Lowerable (any T: Lower, i.e. direct bindgen interop) are elided with ... here just to show that the enum is meant to grow — it's #[non_exhaustive] for exactly that reason.

On validation: prepare_call checks, once, that the number of ArgSpecs matches the function's arity and that each nominated source is compatible with its parameter — e.g. a Flat (pre-encoded as cABI) source is accepted only when the parameter's element type has a fixed cABI layout with no out-of-line storage or ownership (Type::is_cabi_inline), so the fast path can be a plain memcpy. Structural mismatches like these fail at prepare_call, so the hot invoke path doesn't re-check them; only data-dependent problems (for example, a byte buffer whose length isn't a whole number of elements) surface at invoke.

With this an embedder can even, for example, receive a large list of values of some specific component model type (of which the embedder has no static knowledge) returned from one component instance, and hand it straight to another — copying the list's bytes just once, directly from one instance's linear memory into the other's:

let make_items = source.prepare_call(&store_a, &[])?;
let consume_items = sink.prepare_call(&store_b, &[ArgSpec::Flat])?;

make_items.bind().invoke_scoped(&mut store_a, |produced| {
    consume_items
        .bind()
        .arg_flat(produced.view(0)?)
        .invoke(&mut store_b)?;
    Ok(())
})?;

How's that sound? Is there anything that jumps out to you as obviously bad or incompatible with the way Wasmtime actually works vs. my mental model?

[jeffparsons]

I just realised that the inline-representable check is inadequate because not all bit patterns are valid for all types, e.g. char. So that concept needs a bit more nuance.

I also need to have a bit more of a think about the consequences of lazy value lowering, assuming that comes along eventually; it could be accommodated by the API I've proposed, but I wonder if there's a more elegant API that could somehow do something more for non-inline-representable types...

[jeffparsons]

UPDATE: I'm working on a "v2" of this proposal, which switches from deciding lowering strategy at the argument level to a "value source" tree that allows freely mixing different kinds of value sources for a single argument. E.g. an argument struct might contain both an array where the elements are all inline-representable and where all bit patterns are valid values, and also something like a resource handle that can't be blindly memcpy'd. It's basically a fancy version of the existing "fully dynamic" Val tree, where you can choose for individual nodes where the data comes from.

I have a few kinks to work out before I propose a concrete API surface, but it feels promising so far in that it manages to give the embedder heaps of flexibility in how values are lowered, lets you copy already-validated bytes from a guest into owned buffers for later use or forward them directly to another component instance (as long as it's in a different Store), and IINM should play really nicely with lazy value lowering if/when that comes along.

[alexcrichton]

This overall looks pretty reasonable to me, but I've found the devil's often in the details for changes like this so it's hard to provide a super-solid opinion before reviewing a PR (which is fine by me, but want to say so as well). One thing I'd personally like to take on as a constraint though is to avoid adding another side method of invoking component functions compared to what we already have. Lowering and such is pretty critical to both correctness and safety and so I want to make sure the critical surface area in Wasmtime stays minimized.

To that extent my ideal for a system like this would be to build the currently-existing Func::call on top of it. Another example could be that Val fits in as "one way to use the system" but it could be used in any number of other ways if so desired. Basically I'm happy keeping Val, but I'd prefer to not have 3 ways to call functions (Val-full-dynamic, this way, and TypedFunc-full-static). How exactly to achieve this, if it is even achievable, I'm not entirely sure, but it's a personal goal of mine at least.