This repository was archived by the owner on Sep 8, 2025. It is now read-only.
Merge with upstream#241
Merged
alexcrichton merged 77 commits intobytecodealliance:mainfrom Aug 1, 2025
Merged
Conversation
Both our current domains failed in https://github.com/bytecodealliance/wasmtime/actions/runs/16349123310/job/46190913918 so add a few more.
* More table safety improvements This is some more progress on #11179 aimed at improving the safety of management of tables internally within Wasmtime: * `Instance::table_index` is removed as it can be replaced with data stored directly in the `VMTableImport` now. * `Instance::get_table` now returns `&mut Table` * `Instance::get_defined_table_with_lazy_init` now returns `&mut Table` * `Instance::with_defined_table_index_and_instance` now directly returns `DefinedTableIndex` plus `Pin<&mut Instance>`, codifying the ability to "laterally move" between instances. * `Instance::table_init_segment` was refactored to "take" the tables during initialization and replace them afterwards, resolving the split borrow issue and removing an `unsafe` block in the function. cc #11179 * Improve safety of `Table::copy` This commit fixes an issue in the previous commit with respect to Miri and Stacked Borrows. This does so by improving the safety of the `Table::copy`-related functions to all work mostly on safe code rather than unsafe references. Some minor amount of unsafety is still present but it is now clearly documented and easier to verify. * Fix tests
…268) Per the Component Model async ABI spec, this option need not be present for payload-less streams and futures. Fixes #11251 Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Update it to use slices in a similar manner to `Func::call_async`. Closes #11218
Turn the lint on and add some safety comments where appropriate. cc #11180
* Stratification of call graphs for parallel bottom-up inlining
This commit takes a call graph and constructs a strata, which is essentially a
parallel execution plan. A strata consists of an ordered sequence of layers, and
a layer of an unordered set of functions. The `i`th layer must be processed
before the `i + 1`th layer, but functions within the same layer may be processed
in any order (and in parallel).
For example, when given the following tree-like call graph:
+---+ +---+ +---+
| a |-->| b |-->| c |
+---+ +---+ +---+
| |
| | +---+
| '---->| d |
| +---+
|
| +---+ +---+
'---->| e |-->| f |
+---+ +---+
|
| +---+
'---->| g |
+---+
then stratification will produce these layers:
[
{c, d, f, g},
{b, e},
{a},
]
Our goal in constructing the layers is to maximize potential parallelism at each
layer. Logically, we do this by finding the strongly-connected components of the
input call graph and peeling off all of the leaves of SCCs'
condensation (i.e. the DAG that the SCCs form; see the documentation for the
`StronglyConnectedComponents::evaporation` method for details). These leaves
become the strata's first layer. The layer's components are removed from the
condensation graph, and we repeat the process, so that the condensation's new
leaves become the strata's second layer, and etc... until the condensation graph
is empty and all components have been processed. In practice we don't actually
mutate the condensation graph or remove its nodes but instead count how many
unprocessed dependencies each component has, and a component is ready for
inclusion in a layer once its unprocessed-dependencies count reaches zero.
This commit also renames the entity type for strongly-connected components from
`Component` to `Scc`, as I felt the former was a bit ambiguous given Wasm
components.
The next PR will extend Wasmtime's compilation driver code to actually make use
of this new infrastructure.
* Address review feedback
* x64: Add EVEX shifts to the new assembler
This commit adds bindings for the EVEX encodings of `vps{ll,ra,rl}{d,q}`
to the new assembler. Currently the 16-bit shifts `vps{ll,ra,rl}w` are
omitted due to the `avx512bw` feature not yet being bound in Cranelift.
In implementing these instructions a few refactorings/fixes were
necessary:
* Primarily all EVEX instructions now need to be defined not only with
their vector length but also their "tuple type" found in encoding
tables. This is required to correctly handle the 8-bit displacement
scaling that happens with EVEX instructions.
* Some small helpers to the `Evex` structure were added such as
`Evex::digit` and `Evex::ib`.
* The `evex_scaling` factor is now calculated in `generate_evex_prefix`
according to the instruction format itself.
* The VEX and EVEX `generate_*_prefix` functions now delegate to a
shared function to handle the same operand formats across both of
them.
* Fuzz generation of `AmodeOffset` is now updated to bias to some
"interesting" offsets that exercise the cases where EVEX scaling is
necessary.
* The ISLE `XmmUnaryRmRImmEvex` instruction format was removed as it's
no longer necessary.
* Fix emit tests
* x64: Use 8-bit jumps in pseudo-insts Cranelift does not currently implement any form of "relaxation" of instructions where, for example, a 32-bit jump is shrunk to an 8-bit jump if the destination actually fits. In lieu of this Cranelift pessimistically emits 32-bit jumps on x64, for example, for all jumps between basic blocks. This is a difficult problem to solve in general but for pseudo-instructions it's a much more targeted problem which should be easier to solve. This commit updates all pseudo-instructions in the x64 backend to use 8-bit jumps instead of full 32-bit jumps within their code bodies. It's statically known that the instructions bodies being generate are all small enough to fit in 8 bits. This helps shrink the generated code for a number of instructions whenever a pseudo-inst is used instead of basic blocks. Optimizing jumps between basic blocks is left as a future optimization as it's likely to be much more difficult to implement than this. * Fix emit tests
* feat(wasi): introduce common CLI context Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * chore(wasi): implement traits for boxed values Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * chore(wasi): implement `Default` for common WASI builder Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * feat(wasip3): implement `wasi:cli` Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor: require streams to be `Send` Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor: avoid typing `WasiCli` in task Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor: remove `Unpin` bound from stream I/O Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor: remove `ResourceView` Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * chore: update `serve` to new WASI `isatty` API Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * chore: adapt to stream API changes Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor: avoid `**` syntax Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor(wasip3): remove `Impl` wrappers Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor(wasip3): use shorthand closure syntax Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * chore: account for different env on different targets prtest:full Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> --------- Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* fix(wasip1): prevent duplicate FD usage The implementation assumed that only the runtime could ever issue FDs, however that's not the case in p1, where guests can choose arbitrary FDs to use (e.g. via `fd_renumber`). Due to incorrect accounting, guests could "mark" arbitrary FDs as "free" and trigger a panic in the host by requesting a new FD. Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * test(wasip1): expand `fd_renumber` test Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * doc: add release notes Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * test(wasip1): ignore `fd_renumber` tests when using adapter Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * refactor(wasip1): do not modify descriptors on `fd_renumber(n, n)` Since `remove` is now only used once, remove it. As a sideffect, this makes the implementation more explicit . Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * doc: reference the CVE prtest:full Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> --------- Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* Revert "test(wasip1): ignore `fd_renumber` tests when using adapter" This reverts commit 00325ff36d44a3cf666085d56a7c80e907636d8b. * fix(wasip1-adapter): prevent `unreachable` panic on `fd_renumber` Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * doc: document adapter panic fix Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> * doc: add PR reference prtest:full Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net> --------- Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: pingshuijie <pingshuijie@outlook.com>
* Fix cargo vet on main Account for newly published crates. * More vets
…282) We copy *all* callee blocks into the caller's layout, but were then only copying the callee instructions in *reachable* callee blocks into the caller. Therefore, any *unreachable* blocks would remain empty in the caller, which is invalid CLIF because all blocks must end in a terminator, so this commit adds a quick pass over the inlined blocks to remove any empty blocks from the caller's layout.
This reverts commit 2fd1abf.
Despite #11265 this test [still failed][failure] during a backport to the 33.0.0 branch. I don't know what all the domains are returning instead of "200 OK" so this is an attempt to add some debugging to that effect and see if we can catch it in the future. [failure]: https://github.com/bytecodealliance/wasmtime/actions/runs/16447909572/job/46484874418#step:19:687
* x64: Migrate vpabsq to the new assembler * x64: Migrate vpopcntb to new assembler * x64: Migrate vpmullq to the new assembler * x64: Migrate vcvtudq2ps to the new assembler * x64: Migrate vpermi2b to new assembler * Fix test warnings
This commit relaxes the requirement that all `*.cwasm` images are aligned for their ELF header by enabling the `unaligned` feature of the `object` crate. This is useful in scenarios where paging isn't in use and shouldn't, in theory, impact preexisting scenarios where everything happens to already be aligned. Closes #11300
* Fix build on macOS * Set up MACOSX_DEPLOYMENT_TARGET correctly * Update CMakeLists.txt
When the HTTP library is not used as a proxy, removing of the forbidden headers by default may not make sense. This change makes it configurable via the `WasiHttpView` by overriding the is_forbidden_header method, so that the DEFAULT_FORBIDDEN_HEADERS will not be always included. The behavior is now close to p3.
This includes a pruning of older proposals which removes a number of duplicated tests.
Keeping it up-to-date.
* Update documentation of wasm proposal support * Indicate that stack-switching is implemented for x86_64 Linux. * Indicate that exception-handling is a work-in-progress. * Change emojis to something that more editors/viewers seem to agree is "two things wide" as opposed to some thinking it's "one thing wide". * Fix copy/paste
* Deny `unsafe_op_in_unsafe_fn` in `wasmtime::runtime::vm` Slowly expanding this lint to more of the crate. prtest:full * Fix lints in custom module * Fix some lints with miri * Fix non-VM build * Fix arm windows
There's a bug on OSS-Fuzz which I can't reproduce and I believe it's due to the CPU features on OSS-Fuzz being different than what I have locally. Try to work around this by consuming a constant amount of data from the fuzz input regardless of what features the host CPU has.
These variants were related to the runtime borrow checker, and are no longer constructed anywhere since #8277 (for `BorrowCheckerOutOfHandles`) and #8702 (for `PtrBorrowed`).
* Update CMakeLists.txt * Update CMakeLists.txt
When adding crates over time (or renaming) we've often forgotten to add either the `wasmtime-publish` user or the `wasmtime-publish` team. Add a check that looks at the `owners` endpoint on crates.io and verifies that the two are present.
The `match` makes the cases a lot more clear IMO, and additionally the conditions asserted in each branch become trivially obvious (they are exactly what was matched upon for their associated branch) so I removed them. I also made some minor copy-edit tweaks to some comments while I was here. This would have just been a nitpick review comment on bytecodealliance/wasmtime#11255 but I was traveling and didn't get a chance to leave review comments in a reasonable amount of time, so instead I am just fixing them up myself in this follow up.
…tx_mut` (#11318) I initially thought that the documentation was missing a safety invariant that callers must uphold, and then realized that it was a little subtler than I originally thought, so I updated the documentation to clarify this for future readers.
This fixes a minor merge conflict between #11325 and #11328 which isn't currently exercised by in-repo WASI bindings but will be soon once wasip3-prototyping is finished merging.
These maps are all keyed on entities, and should all be fairly dense and/or small, so we can use `SecondaryMap`s to get fast lookups and avoid hashing.
* Account for concurrent resource destructors This fixes a minor merge conflict between #11325 and #11328 which isn't currently exercised by in-repo WASI bindings but will be soon once wasip3-prototyping is finished merging. * Update expanded test expectations
* ensure component value lowerings are run on a worker fiber This is necessary because lowering a component value may require calling realloc, which may involve epoch interruption, which may require suspending the current fiber. Which obviously doesn't work if we're not running in a fiber. Also, we need to make sure there are no host frames on the stack. Note the use of `Mutex` for `WorkItem::WorkerFunction` and `WorkerItem::Function`. We never lock the mutex -- it's only used to plumb through the inner, non-`Sync` value while satisfying the compiler that `Store` is `Sync`. Signed-off-by: Joel Dice <joel.dice@fermyon.com> * add "we're on a fiber" assertion to `LowerContext::new` Signed-off-by: Joel Dice <joel.dice@fermyon.com> --------- Signed-off-by: Joel Dice <joel.dice@fermyon.com>
…355) These methods get-or-create various entities given some kind of Wasm index. Those Wasm indices were previously passed as `u32`s, but are now typed indices (e.g. `MemoryIndex` instead of `u32`). This makes it more difficult to pass the wrong index to these methods. I also removed some unused arguments from a few `FuncEnvironment` methods that the results of `FuncTranslationState::get_*` calls were feeding into. These were left over from when `FuncEnvironment` was a trait and not a single type.
* Don't support fallible drop in futures_and_streams This commit is a refinement of #11325 to use `.unwrap()` internally instead of ignoring errors from dropping futures and streams. Fallible drop isn't supported in Rust and these shouldn't panic assuming the host is properly matching handles to stores. * Fix build of wasmtime-wasi * Don't empty the table during store destruction Leave it around while futures/fibers are being manually dropped so any destructors associated there get access to the table (as required by streams/futures/etc). * Remove unused import
This is a follow-up to #11325 with a number of cosmetic changes about
the shape of the API and structure of the internals:
* `{Stream,Future}{Reader,Writer}::guard` is now an alternative
constructor to `Guard*::new` (import fewer types).
* Internally `WithAccessor` and `DropWithStore` are removed in favor of
direct `Drop for Guard*` impls.
* An `Option` is used to replace `ManuallyDrop` and `unsafe` code.
* `{Stream,Future}{Reader,Writer}::close{,_with}` now take `&mut self`
instead of `self` to be more composable with `&mut self` arguments
during `Drop` for other structures (e.g. build-your-own
drop-with-store).
* The type parameters on `Guard*` are simplified to just `T`, the future
or stream payload, and `A: AsAccessor`. This helps cut down on the
complexity of signatures.
* `Guard*` types now have `into_{stream,future}` as an alternative to
`.into()` which doesn't require type annotations.
…356) Not on a per `TypeIndex` basis. The former is deduplicated, the latter is not. This way we don't end up creating duplicate `ir::Signature`s in the CLIF.
prtest:full
There are a few places in `concurrent.rs` where we use `GuestTask::sync_call_set` to wait on waitables during synchronous calls. However, they may have been members of another set before we joined them to `sync_call_set`, in which case we need to move them back when we're done (or at least remove them from `sync_call_set`). Prior to this fix, we would panic when dropping a task which had subtasks which had been synchronously cancelled. I've updated `async_cancel_callee.rs` to cover that case. Signed-off-by: Joel Dice <joel.dice@fermyon.com>
This is a follow-up to #11325 with a number of cosmetic changes about
the shape of the API and structure of the internals:
* `{Stream,Future}{Reader,Writer}::guard` is now an alternative
constructor to `Guard*::new` (import fewer types).
* Internally `WithAccessor` and `DropWithStore` are removed in favor of
direct `Drop for Guard*` impls.
* An `Option` is used to replace `ManuallyDrop` and `unsafe` code.
* `{Stream,Future}{Reader,Writer}::close{,_with}` now take `&mut self`
instead of `self` to be more composable with `&mut self` arguments
during `Drop` for other structures (e.g. build-your-own
drop-with-store).
* The type parameters on `Guard*` are simplified to just `T`, the future
or stream payload, and `A: AsAccessor`. This helps cut down on the
complexity of signatures.
* `Guard*` types now have `into_{stream,future}` as an alternative to
`.into()` which doesn't require type annotations.
Collaborator
|
bytecodealliance/wasmtime#11361 fixes the test failures. |
This commit removes blanket "forwarding" impls for `&mut T` and `Box<T>` in favor of having forwarding impls of `WasiView` implying specific interfaces such as `WasiRandomView` for example. This is intended to make future integration with wasi-http easier where wasi-http will take a few specific proposals (e.g. not `WasiFilesystemView`) but implementing `WasiView` for a contexet will still be sufficient (as opposed to requiring multiple implementations of separate traits). The original use case of forwarding impls has more-or-less been refactored away at this point so I don't think it's as critical to preserve them.
Eventually we're going to want significantly more sharing than we have today but while things are still gated in Wasmtime by default this for now starts out by only changing the structure of the implementations such that the impl of `Host` traits are the same for both p2 and p3. Additionally the `WasiCtx` values are brought closer together to ideally unify them eventually too.
* optimize host stream/future writes for flat payloads When we know statically that a payload will not require any guest realloc calls to lower, there's no need to defer the lowering to a fiber -- we can just do it immediately. Signed-off-by: Joel Dice <joel.dice@fermyon.com> * replace `ComponentType::IS_FLAT_TYPE` with `MAY_REQUIRE_REALLOC` Per review comments, this more clearly conveys the purpose of the constant. I've also added a doc comment about what it means and how it is used. Signed-off-by: Joel Dice <joel.dice@fermyon.com> --------- Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* fix sending default value when closing host-owned future writer My earlier commit didn't handle all the cases, which caused a regression for the wasi-http tests in the `wasip3-prototyping` repo. Signed-off-by: Joel Dice <joel.dice@fermyon.com> * add test for specific close-future-writer-with-default-value scenario This covers the case where we're closing the future write end from the host without having written a value and while the guest has already started a read (which I fixed in the previous commit). Signed-off-by: Joel Dice <joel.dice@fermyon.com> --------- Signed-off-by: Joel Dice <joel.dice@fermyon.com>
…11366) We already have easy access to the function types themselves, and asking for their parameter length is cheap once we have a reference to them.
…d into `FuncEnvironment` (#11367) It more naturally lives in the environment, where we already keep track of CLIF entities. That these entities were in `FuncTranslationState` is a throwback to back when `FuncEnvironment` was a trait, and not a concrete type. Also rename `FuncTranslationState` to `FuncTranslationStacks`, since all it does is manage the value and control stacks now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
18 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bringing things up-to-date to get a baseline to send filesystem bits over to wasmtime. Currently tests are failing as discussed with @dicej, so that'll get figured out tomorrow.