Only allow async ABI options for async-typed functions

Author: lukewagner

design/mvp/CanonicalABI.md

@@ -777,15 +777,22 @@ class Task(Supertask):
     self.threads = []
 ```
 
-The `Task.needs_exclusive` predicate returns whether the Canonical ABI options
-indicate that the core wasm being executed does not expect to be reentered
-(e.g., because the code is using a single global linear memory shadow stack).
-Concretely, this is assumed to be the case when core wasm is lifted
-synchronously or with `async callback`. This predicate is used by the other
-`Task` methods to determine whether to acquire/release the component instance's
-`exclusive` lock.
+The `Task.needs_exclusive` method returns whether an `async`-typed function's
+ABI options indicate that the Core WebAssembly code requires serialized
+execution (with the common reason being that there is a single, global linear
+memory shadow stack). This serialized execution is implemented by
+acquiring/releasing the component-instance-wide `exclusive` lock before/after
+executing Core WebAssembly code executing on the task's *implicit thread*
+(explicit threads created by `thread.new-indirect` ignore the `exclusive` lock).
+Specifically, sync- and stackless-async-lifted (`async callback`) functions
+require the `exclusive` lock and stackful-async-lifted (`async`) functions
+ignore the `exclusive` lock (just like explicit threads). Note that
+non-`async`-typed functions' implicit threads also ignore the `exclusive` lock
+since they must complete synchronously without blocking and thus don't have to
+worry about non-LIFO stack interleaving.
 ```python
   def needs_exclusive(self):
+    assert(self.ft.async_)
     return not self.opts.async_ or self.opts.callback
 ```
 
@@ -3452,7 +3459,9 @@ present, is validated as such:
 * if `realloc` is present then `memory` must be present
 * `post-return` - only allowed on [`canon lift`](#canon-lift), which has rules
   for validation
-* 🔀 `async` - cannot be present with `post-return`
+* 🔀 `async` - is only allowed when used with an `async` function type in
+  [`canon lift`](#canon-lift) or [`canon lower`](#canon-lower) and cannot be
+  present with `post-return`
 * 🔀,not(🚟) `async` - `callback` must also be present. Note that with the 🚟
   feature (the "stackful" ABI), this restriction is lifted.
 * 🔀 `callback` - the function has type `(func (param i32 i32 i32) (result i32))`
@@ -3580,7 +3589,7 @@ function (specified as a `funcidx` immediate in `canon lift`) until the
     [packed] = call_and_trap_on_throw(callee, flat_args)
     code,si = unpack_callback_result(packed)
     while code != CallbackCode.EXIT:
-      assert(inst.exclusive is task)
+      assert(task.needs_exclusive() and inst.exclusive is task)
       inst.exclusive = None
       match code:
         case CallbackCode.YIELD:

design/mvp/Concurrency.md

@@ -74,16 +74,6 @@ the same way that they already bind to various OS's concurrent I/O APIs (such
 as `select`, `epoll`, `io_uring`, `kqueue` and Overlapped I/O) making the
 Component Model "just another OS" from the language toolchain's perspective.
 
-The new async ABI can be used alongside or instead of the existing Preview 2
-"sync ABI" to call or implement *any* WIT function type. When *calling* an
-imported function via the async ABI, if the callee [blocks](#blocking), control
-flow is returned immediately to the caller, and the callee continues executing
-concurrently. When *implementing* an exported function via the async ABI,
-multiple concurrent export calls are allowed to be made by the caller.
-Critically, both sync-ABI-calls-async-ABI and async-ABI-calls-sync-ABI pairings
-have well-defined, composable behavior for both inter-component and
-intra-component calls.
-
 In addition to adding a new async *ABI* for use by the language's compiler and
 runtime, the Component Model also adds a new `async` [effect type] that can be
 added to function types (in both WIT and raw component function type
@@ -103,6 +93,16 @@ invariant is necessary to allow non-`async` component exports to be called in
 synchronous contexts (like event listeners, callbacks, getters, setters and
 constructors).
 
+The new async ABI can be used alongside or instead of the existing Preview 2
+"sync ABI" to call or implement any `async`-typed functions. When *calling* an
+imported function via the async ABI, if the `async` callee [blocks](#blocking),
+control flow is returned immediately to the caller, and the callee continues
+executing concurrently. When *implementing* an `async` function via the async
+ABI, multiple concurrent export calls are allowed to be made by the caller.
+Critically, both sync-ABI-calls-async-ABI and async-ABI-calls-sync-ABI pairings
+have well-defined, composable behavior for both inter-component and
+intra-component calls.
+
 Because `async` function exports may be implemented with the *sync* ABI and
 then call `async` function imports using the *sync* ABI, traditional sync code
 can compile directly to components exporting `async` functions without having
@@ -873,19 +873,12 @@ JS [top-level `await`] or I/O in C++ constructors executing during `start`.
 
 ## Async ABI
 
-At an ABI level, native async in the Component Model defines for every WIT
-function an async-oriented core function signature that can be used instead of
-or in addition to the existing (Preview-2-defined) synchronous core function
-signature. This async-oriented core function signature is intended to be called
-or implemented by generated bindings which then map the low-level core async
-protocol to the languages' higher-level native concurrency features.
-
-Note that *every* WIT-level function type can be lifted and lowered using the
-async (or sync) ABI. While calling a non-`async`-typed function import using
-the async ABI will never returned that the call "blocked" (as guaranteed by the
-Component Model trapping if the callee would have blocked), the async ABI is
-still allowed to be used (for the benefit of code generators that only want
-to think about one ABI).
+At an ABI level, native async in the Component Model defines for every
+`async`-typed function a non-blocking core function signature that can be
+used instead of or in addition to the existing (Preview-2-defined) synchronous
+core function signature. This non-blocking core function signature is intended
+to be called or implemented by generated bindings which then map the low-level
+core async protocol to the languages' higher-level native concurrency features.
 
 ### Async Import ABI
 

design/mvp/Explainer.md

@@ -1328,13 +1328,10 @@ be deallocated and destructors called. This immediate is always optional but,
 if present, is validated to have parameters matching the callee's return type
 and empty results.
 
-🔀 The `async` option specifies that the component wants to make (for imports)
-or support (for exports) multiple concurrent (asynchronous) calls. This option
-can be applied to any component-level function type and changes the derived
-Canonical ABI significantly. See the [concurrency explainer] for more details.
-When a function signature contains a `future` or `stream`, validation of `canon
-lower` requires the `async` option to be set (since a synchronous call to a
-function using these types is highly likely to deadlock).
+🔀 The `async` option may only be used with `async` function types and specifies
+that the component wants to make (for imports) or support (for exports) multiple
+concurrent (asynchronous) calls. This option changes the derived Canonical ABI
+significantly; see the [concurrency explainer] for more details.
 
 🔀 The `(callback ...)` option may only be present in `canon lift` when the
 `async` option has also been set and specifies a core function that is

design/mvp/canonical-abi/definitions.py

@@ -449,6 +449,7 @@ def __init__(self, ft, opts, inst, supertask, on_start, on_resolve):
     self.threads = []
 
   def needs_exclusive(self):
+    assert(self.ft.async_)
     return not self.opts.async_ or self.opts.callback
 
   def may_block(self):
@@ -2104,7 +2105,7 @@ def thread_func():
     [packed] = call_and_trap_on_throw(callee, flat_args)
     code,si = unpack_callback_result(packed)
     while code != CallbackCode.EXIT:
-      assert(inst.exclusive is task)
+      assert(task.needs_exclusive() and inst.exclusive is task)
       inst.exclusive = None
       match code:
         case CallbackCode.YIELD: