Rebase CABI onto explicit stack-switching interface (no behavior change)

Author: lukewagner

design/mvp/CanonicalABI.md

@@ -16,6 +16,7 @@ specified here.
     * [Component Instance State](#component-instance-state)
     * [Table State](#table-state)
     * [Resource State](#resource-state)
+    * [Stack Switching](#stack-switching)
     * [Thread State](#thread-state)
     * [Waitable State](#waitable-state)
     * [Task State](#task-state)
@@ -587,69 +588,200 @@ class ResourceType(Type):
 ```
 
 
-#### Thread State
+#### Stack Switching
 
-As described in the [concurrency explainer], threads are created both
-*implicitly*, when calling a component export (in `canon_lift` below), and
-*explicitly*, when core wasm code calls the `thread.new-indirect` built-in (in
-`canon_thread_new_indirect` below). Threads are represented here by the
-`Thread` class and the [current thread] is represented by explicitly threading
-a reference to a `Thread` through all Core WebAssembly calls so that the
-`thread` parameter always points to "the current thread". The `Thread` class
-provides a set of primitive control-flow operations that are used by the rest
-of the Canonical ABI definitions.
-
-While `Thread`s are semantically created for each component export call by the
-Python `canon_lift` code, an optimizing runtime should be able to allocate
-`Thread`s lazily, only when needed for actual thread switching operations,
-thereby avoiding cross-component call overhead for simple, short-running
-cross-component calls. To assist in this optimization, `Thread`s are put into
-their own per-component-instance `threads` table so that thread table indices
-and elements can be more-readily reused between calls without interference from
-the other kinds of handles.
-
-`Thread` is implemented using the Python standard library's [`threading`]
-module. While a Python [`threading.Thread`] is a preemptively-scheduled [kernel
-thread], it is coerced to behave like a cooperatively-scheduled [fiber] by
-careful use of [`threading.Lock`]. If Python had built-in fibers (or algebraic
-effects), those could have been used instead since all that's needed is the
-ability to switch stacks. In any case, the use of `threading.Thread` is
-encapsulated by the `Thread` class so that the rest of the Canonical ABI can
-simply use `suspend`, `resume`, etc.
-
-When a `Thread` is suspended and then resumed, it receives a `Cancelled`
-value indicating whether the caller has cooperatively requested that the thread
-cancel itself which is communicated to Core WebAssembly with the following
-integer values:
+Component Model concurrency support is defined in terms of the Core WebAssembly
+[stack-switching] proposal's `cont.new`, `resume` and `suspend` instructions.
+However, the Component Model only needs a limited subset of the full
+[stack-switching] proposal:
+
+First, there are only two global [control tags] used with `suspend`:
+```wat
+(tag $block (param $switch-to (ref null $Thread)) (result $cancelled bool))
+(tag $current-thread (result (ref $Thread)))
+```
+The `$block` tag is used to suspend a [thread] until some future event. The
+parameters and results will be described in the next section, where they are
+used to define `Thread`. The `$current-thread` tag is used to retrieve the
+current `Thread`, which is semantically stored in the `resume` handler's local
+state (although an optimizing implementation would instead maintain the current
+thread in the VM's execution context so that it could be retrieved with a single
+load and/or kept in register state).
+
+Second, there is only a single continuation type used with `resume`:
+```wat
+(type $ct (cont (func (param $cancelled bool) (result (ref null $Thread)))))
+```
+Thus, continuations are only produced for the `$block` event; the continuation
+produced for `$current-thread` is immediately resumed and so never "escapes".
+
+Third, *every* `resume` performed by the Canonical ABI always handles *both*
+`$block` and `$current-thread` and thus every Canonical ABI `suspend` is always
+handled by the innermost Canonical ABI `resume` without a dynamic handler/tag
+search.
+
+Given these restrictions, specialized versions of `cont.new`, `resume` and
+`suspend` that are "monomorphized" to the above types and tags can be easily
+implemented in terms of Python's standard preemptive threading primitives, using
+[`threading.Thread`] to provide a native stack, [`threading.Lock`] to only allow
+a single `threading.Thread` to execute at a time, and [`threading.local`] to
+maintain the dynamic handler scope using thread-local storage. This could have
+been implemented more directly and efficiently using [fibers], but the Python
+standard library doesn't have fibers. However, a realistic implementation is
+expected to use (a pool of) fibers.
+
+Starting with `cont.new`, the monomorphized version takes a function type
+matching `$ct` above:
 ```python
 class Cancelled(IntEnum):
   FALSE = 0
   TRUE = 1
-```
+
+class Continuation:
+  lock: threading.Lock
+  handler: Handler
+  cancelled: Cancelled
+
+class Handler:
+  tls = threading.local()
+  lock: threading.Lock
+  current: Thread
+  cont: Optional[Continuation]
+  switch_to: Optional[Thread]
+
+def cont_new(f: Callable[[Cancelled], Optional[Thread]]) -> Continuation:
+  cont = Continuation()
+  cont.lock = threading.Lock()
+  cont.lock.acquire()
+  def wrapper():
+    cont.lock.acquire()
+    Handler.tls.value = cont.handler
+    f(cont.cancelled)
+    handler = Handler.tls.value
+    handler.cont = None
+    handler.switch_to = switch_to
+    handler.lock.release()
+  threading.Thread(target = wrapper).start()
+  return cont
+```
+Here, `Continuation` is used to pass parameters from `resume` to the
+continuation's thread. These parameters are set on `Continuation` right before
+`resume` calls `Continuation.lock.release()` to transfer control flow to the
+continuation. The `Handler` object is created by `resume` with the expectation
+that `Handler.lock.release()` will be called to transfer control flow and
+results back to `resume` handler. The `Handler` is stored in the thread-local
+storage of the internal `threading.Thread` to implement the dynamic scoping
+required by stack-switching. Because a single `threading.Thread` can be
+suspended and resumed many times (each time with a new `Continuation` /
+`Handler`), the `Handler` must be re-loaded from TLS after `f` returns since it
+may have changed.
+
+Next, `resume` is monomorphized to take a continuation of type `$ct`, the
+`cancelled` argument passed to `$ct` and, lastly, the `current` `Thread` which
+is to be immediately returned by the `(on $current-thread)` handler. The
+`(on $block)` and "returned without suspending" cases are merged into a single
+return value, where the latter "returned without suspended" case produces
+`None` for the returned `Optional[Continuation]`.
+```python
+def resume(cont: Continuation, cancelled: Cancelled, current: Thread) -> \
+           tuple[Optional[Continuation], Optional[Thread]]:
+  handler = Handler()
+  handler.lock = threading.Lock()
+  handler.lock.acquire()
+  handler.current = current
+  cont.handler = handler
+  cont.cancelled = cancelled
+  cont.lock.release()
+  handler.lock.acquire()
+  return (handler.cont, handler.switch_to)
+```
+
+Lastly, `suspend` is monomorphized into 2 functions for the `$block` and
+`$current-thread` tags shown above, so that their signatures and implementations
+can be specialized. Since `$current-thread` has a trivial handler that simply
+returns the `current` `Thread` passed to `resume`, it can simply return
+`Handler.current` directly without any stack switching.
+```python
+def block(switch_to: Optional[Thread]) -> Cancelled:
+  cont = Continuation()
+  cont.lock = threading.Lock()
+  cont.lock.acquire()
+  handler = Handler.tls.value
+  handler.cont = cont
+  handler.switch_to = switch_to
+  handler.lock.release()
+  cont.lock.acquire()
+  Handler.tls.value = cont.handler
+  return cont.cancelled
+
+def current_thread() -> Thread:
+  return Handler.tls.value.current
+```
+
+In the future, when Core WebAssembly gets [stack-switching], the Component Model
+`$block` and `$current-thread` tags would not be exposed to Core WebAssembly.
+Thus, an optimizing implementation would continue to be able to implement
+`block()` as a direct control flow transfer to the innermost `resume()` and
+`current_thread()` via implicit context, both without an O(n) handler-stack tag
+search. In particular, this avoids the pathological O(N<sup>2</sup>) behavior
+which would otherwise arise if Component Model cooperative threads were used in
+conjunction with deeply-nested Core WebAssembly handlers.
+
+Additionally, once Core WebAssembly has stack switching, any unhandled events
+that originate in Core WebAssembly would turn into traps if they reach a
+component boundary (just like unhandled exceptions do now; see
+`call_and_trap_on_throw` below). Thus, all cross-component/cross-language stack
+switching would continue to be mediated by the Component Model's types and
+Canonical ABI, with Core WebAssembly stack-switching used to implement
+intra-component concurrency according to the language's own internal ABI, which
+can be different inside each component.
+
+
+#### Thread State
+
+As described in the [concurrency explainer], threads are created both
+*implicitly*, when calling a component export (in `canon_lift` below), and
+*explicitly*, when core wasm code calls the `thread.new-indirect` built-in (in
+`canon_thread_new_indirect` below). While threads are *logically* created for
+each component export call, an optimizing runtime should be able to allocate
+threads lazily when needed for actual thread switching operations, thereby
+avoiding cross-component call overhead for simple, short-running cross-component
+calls. To assist in this optimization, threads are put into their own
+`ComponentInstance.threads` table to reduce interference from the other kinds of
+handles.
+
+Threads are represented in the Canonical ABI by the `Thread` class defined in
+this section. The `Thread` class is implemented in terms of the `cont_new`,
+`resume`, `block` and `current_thread` stack-switching primitives defined in the
+previous section. `Thread` defines a set of higher-level concurrency operations
+that are used by all the other Canonical ABI definitions. In particular, a
+"thread" adds the higher-level concepts of:
+* [waiting on external I/O]
+* [async call stack]
+* [thread index]
+* [thread-local storage]
+* [cancellation]
 
 Introducing the `Thread` class in chunks, a `Thread` has the following fields
 and can be in one of the following 3 states based on these fields:
-* `running`: actively executing with a "parent" thread that is waiting
-  to run once the `running` thread suspends or returns
-* `suspended`: waiting to be `resume`d by another thread
-* `waiting`: waiting to be `resume`d by `Store.tick` once `ready`
+* `running`: actively executing on the stack
+* `suspended`: waiting to be resumed by some other thread `running` in
+  the same component instance (via its `index`)
+* `pending`: waiting to to be resumed by the host (in `Store.tick` once `ready`
 
 ```python
 class Thread:
-  task: Task
-  fiber: threading.Thread
-  fiber_lock: threading.Lock
-  parent_lock: Optional[threading.Lock]
+  cont: Optional[Continuation]
   ready_func: Optional[Callable[[], bool]]
-  cancellable: bool
-  cancelled: Cancelled
+  task: Task
   index: Optional[int]
   context: list[int]
+  cancellable: bool
 
   CONTEXT_LENGTH = 2
 
   def running(self):
-    return self.parent_lock is not None
+    return self.cont is None
 
   def suspended(self):
     return not self.running() and self.ready_func is None
@@ -3494,10 +3626,11 @@ optimization to avoid allocating stacks for async languages that have avoided
 the need for stackful coroutines by design (e.g., `async`/`await` in JS,
 Python, C# and Rust).
 
-Uncaught Core WebAssembly [exceptions] result in a trap at component
-boundaries. Thus, if a component wishes to signal an error, it must use some
-sort of explicit type such as `result` (whose `error` case particular language
-bindings may choose to map to and from exceptions):
+Uncaught Core WebAssembly [exceptions] or, in a future with [stack-switching],
+unhandled events, result in a trap at component boundaries. Thus, if a component
+wishes to signal an error, it must use some sort of explicit type such as
+`result` (whose `error` case particular language bindings may choose to map to
+and from exceptions):
 ```python
 def call_and_trap_on_throw(callee, thread, args):
   try:
@@ -4981,16 +5114,22 @@ def canon_thread_available_parallelism():
 [Shared-Everything Dynamic Linking]: examples/SharedEverythingDynamicLinking.md
 [Concurrency Explainer]: Concurrency.md
 [Suspended]: Concurrency#thread-built-ins
+[Thread Index]: Concurrency#thread-built-ins
+[Async Call Stack]: Concurrency.md#subtasks-and-supertasks
 [Structured Concurrency]: Concurrency.md#subtasks-and-supertasks
 [Recursive Reentrance]: Concurrency.md#subtasks-and-supertasks
 [Backpressure]: Concurrency.md#backpressure
+[Thread]: Concurrency.md#threads-and-tasks
+[Threads]: Concurrency.md#threads-and-tasks
 [Current Thread]: Concurrency.md#current-thread-and-task
 [Current Task]: Concurrency.md#current-thread-and-task
 [Block]: Concurrency.md#blocking
+[Waiting on External I/O]: Concurrency.md#blocking
 [Subtasks]: Concurrency.md#subtasks-and-supertasks
 [Readable and Writable Ends]: Concurrency.md#streams-and-futures
 [Readable or Writable End]: Concurrency.md#streams-and-futures
 [Thread-Local Storage]: Concurrency.md#thread-local-storage
+[Cancellation]: Concurrency.md#cancellation
 [Subtask State Machine]: Concurrency.md#cancellation
 [Stream Readiness]: Concurrency.md#stream-readiness
 
@@ -5013,6 +5152,7 @@ def canon_thread_available_parallelism():
 [WASI]: https://github.com/webassembly/wasi
 [Deterministic Profile]: https://github.com/WebAssembly/profiles/blob/main/proposals/profiles/Overview.md
 [stack-switching]: https://github.com/WebAssembly/stack-switching
+[Control Tags]: https://github.com/WebAssembly/stack-switching/blob/main/proposals/stack-switching/Explainer.md#declaring-control-tags
 [`memaddr`]: https://webassembly.github.io/spec/core/exec/runtime.html#syntax-memaddr
 [`memaddrs` table]: https://webassembly.github.io/spec/core/exec/runtime.html#syntax-moduleinst
 [`memidx`]: https://webassembly.github.io/spec/core/syntax/modules.html#syntax-memidx
@@ -5028,8 +5168,7 @@ def canon_thread_available_parallelism():
 [Code Units]: https://www.unicode.org/glossary/#code_unit
 [Surrogate]: https://unicode.org/faq/utf_bom.html#utf16-2
 [Name Mangling]: https://en.wikipedia.org/wiki/Name_mangling
-[Kernel Thread]: https://en.wikipedia.org/wiki/Thread_(computing)#kernel_thread
-[Fiber]: https://en.wikipedia.org/wiki/Fiber_(computer_science)
+[Fibers]: https://en.wikipedia.org/wiki/Fiber_(computer_science)
 [Asyncify]: https://emscripten.org/docs/porting/asyncify.html
 
 [`import_name`]: https://clang.llvm.org/docs/AttributeReference.html#import-name
@@ -5040,7 +5179,8 @@ def canon_thread_available_parallelism():
 
 [`threading`]: https://docs.python.org/3/library/threading.html
 [`threading.Thread`]: https://docs.python.org/3/library/threading.html#thread-objects
-[`threading.Lock`]:  https://docs.python.org/3/library/threading.html#lock-objects
+[`threading.Lock`]: https://docs.python.org/3/library/threading.html#lock-objects
+[`threading.local`]: https://docs.python.org/3/library/threading.html#thread-local-data
 
 [OIO]: https://en.wikipedia.org/wiki/Overlapped_I/O
 [io_uring]: https://en.wikipedia.org/wiki/Io_uring

design/mvp/Concurrency.md

@@ -348,6 +348,7 @@ feature is necessary in any case (due to iloops and traps).
 
 ### Current Thread and Task
 
+TODO
 At any point in time while executing Core WebAssembly code or a [canonical
 built-in] called by Core WebAssembly code, there is a well-defined **current
 thread** whose containing task is the **current task**. The "current thread" is