refactor: cut-over to new wsdb package - comms over uds ipc. (#23036)

## Summary

Cuts TypeScript world-state and the in-process C++ AVM (NAPI) over to
the standalone generated `@aztec/wsdb` package and the `aztec-wsdb` IPC
service. After this PR, both the TS world-state path and the C++ AVM
talk to world state **out-of-process over IPC** (UDS by default, SHM
optional); the in-process NAPI WSDB module is deleted.

The concurrent wsdb server this relies on (reactor + async handlers +
per-fork ordering, #24198) is **now merged into `next`**, so the IPC
world-state path introduced here serves parallel reads concurrently
rather than serializing them through a single server loop.

## IPC contract / schema

The wsdb wire API — the request/response messages between the TS & C++
clients and the `aztec-wsdb` server — is defined by a single schema,
from which `ipc-codegen` generates the C++ server dispatch, the C++/TS
clients, and the shared type definitions:

-
[`barretenberg/cpp/src/barretenberg/wsdb/wsdb_schema.jsonc`](https://github.com/AztecProtocol/aztec-packages/blob/cl/ipc-3-avm-wsdb-cutover/barretenberg/cpp/src/barretenberg/wsdb/wsdb_schema.jsonc)

## Stack

`next` already contains the merged foundation (#23610), wsdb migration
(#23611), and the concurrent wsdb server (#24198).

1. #23036 `cl/ipc-3-avm-wsdb-cutover` — **this PR**
2. #23084 `cl/ipc-4-avm-binary`
3. #23697 `cl/ipc-5-avm-cutover`

## What changes

### C++ / NAPI AVM path

- `AvmSimAPI::simulate` takes a `LowLevelMerkleDBInterface&` instead of
an in-process `WorldState&`.
- NAPI AVM accepts a WSDB socket path and constructs a generated wsdb
IPC client plus VM2 wsdb adapter per simulation.
- NAPI world-state module is deleted; the NAPI module no longer exposes
in-process `WorldState`.
- NAPI module links the current `ipc_runtime` path from the lower stack.

### TypeScript world-state path

- World-state startup is encapsulated behind the world-state
service/facade instead of leaking binary discovery into callers.
- `NativeWorldStateService` spawns `aztec-wsdb` through the generated
`@aztec/wsdb` package.
- The generated wsdb wrapper resolves its service binary from the
installed/local arch package optional dependency.
- The old `WorldStateRevisionWithHandle` shape is removed; callers use
ordinary revisions and, where needed, the wsdb socket path.
- Cleanup paths close spawned wsdb processes explicitly in tests and
service lifecycles.
- The client-side `WorldStateOpsQueue` is removed: ordering is now
enforced server-side per fork (in #24198), so the client just sends
requests directly.

### Test / runtime adjustments for the IPC backend

These adapt existing tests to the fact that world state is now a spawned
process rather than an in-process call:

- `NativeWorldStateService.tmp()` LMDB map sizes reduced 10 GB → 256 MB
per tree. With the IPC backend each `tmp()` spawns a separate
`aztec-wsdb` that maps these per tree, and oversized maps added real
cold-start I/O under parallel CI load. Production map sizes are
unaffected (`tmp()` is the test/dev helper).
- `validator.integration.test.ts` re-anchors its controlled clock
*after* world-state setup. Reexecution's deadline is the slot end; the
real wall-clock spent spawning `aztec-wsdb` during setup could otherwise
push past it before the test body runs.
- Bumped the reexecution test timeout to accommodate IPC setup latency
under load.

## End state after this PR

The TS world-state path and the C++ AVM path both talk to world state
over IPC. The in-process NAPI WSDB module is gone.

## Performance / concurrency

This is a cut-over, not a performance change. The work that keeps
parallel reads concurrent — the reactor + async-handler server with
per-fork ordering — lives in **#24198** and is already in `next`; this
PR just routes TS world-state and the C++ AVM through it. (For
reference: a naive IPC cut-over with a synchronous server loop would
serialize every read through one thread, flat at ~13k reads/s regardless
of concurrency — the reactor in #24198 is what avoids that. See that PR
for the server design and the full throughput analysis.)

To guard the cut-over against accidentally introducing a serialization
bottleneck, this PR keeps `parallel_read.bench.test.ts` (single
connection, pipelined) and adds `multi_connection_read.bench.test.ts` (N
separate connections — the real AVM-pool topology). On this branch,
single-connection pipelined reads reach ~30k/s over SHM at c16 (~80–86%
of the in-process baseline; the residual is per-op TS-facade overhead),
and multi-connection SHM aggregates ~40–47k/s at 7 connections — i.e.
the cut-over preserves read concurrency rather than serializing it.

## Validation

- `wsdb/bootstrap.sh` and root `./bootstrap.sh` pass on this branch.
- `native_world_state.test.ts` (50 tests incl. concurrent
mutating/non-mutating ordering) green, including at the reduced 256 MB
map sizes.
- `validator.integration.test.ts` green over the IPC backend after the
clock re-anchor.

Author: fcarreiro

aztec-up/bootstrap.sh

@@ -1,11 +1,18 @@
 #!/usr/bin/env bash
 source $(git rev-parse --show-toplevel)/ci3/source_bootstrap
 
-hash=$(hash_str $(cache_content_hash ^aztec-up/) $(../yarn-project/bootstrap.sh hash))
+hash=$(hash_str $(cache_content_hash ^aztec-up/) $(../ipc-runtime/bootstrap.sh hash) $(../wsdb/bootstrap.sh hash) $(../yarn-project/bootstrap.sh hash))
 
 # Bare aliases ("nightly", "latest") resolve to this major version.
 DEFAULT_MAJOR_VERSION=${AZTEC_TOOLCHAIN_DEFAULT_MAJOR_VERSION:-4}
 
+function wsdb_package_dirs {
+  for package_dir in "$root"/wsdb/ts/packages/*; do
+    [ -d "$package_dir" ] && echo "$package_dir"
+  done
+  echo "$root/wsdb/ts"
+}
+
 function build {
   # Noop if user doesn't have docker.
   if ! command -v docker &>/dev/null; then
@@ -103,7 +110,9 @@ EOF
     # TODO(AD): we have kludged a retry here. a local NPM install ought to be robust enough not to.
     echo "Deploying packages to local npm registry (version: $version)..."
     {
+      echo $root/ipc-runtime/ts
       echo $root/barretenberg/ts
+      wsdb_package_dirs
       $root/noir/bootstrap.sh get_projects
       $root/yarn-project/bootstrap.sh get_projects
     } | DRY_RUN= parallel --tag --line-buffer --halt now,fail=1 "retry 'cd {} && dump_fail \"deploy_npm $version\" >/dev/null'"

barretenberg/cpp/src/barretenberg/nodejs_module/CMakeLists.txt

@@ -27,7 +27,7 @@ string(REGEX REPLACE "[\r\n\"]" "" NODE_API_HEADERS_DIR ${NODE_API_HEADERS_DIR})
 add_library(nodejs_module SHARED ${SOURCE_FILES})
 set_target_properties(nodejs_module PROPERTIES PREFIX "" SUFFIX ".node")
 target_include_directories(nodejs_module PRIVATE ${NODE_API_HEADERS_DIR} ${NODE_ADDON_API_DIR})
-target_link_libraries(nodejs_module PRIVATE world_state ipc vm2_sim)
+target_link_libraries(nodejs_module PRIVATE ipc ipc_runtime vm2_sim wsdb_ipc_merkle_db)
 
 # On macOS, Node.js N-API symbols are provided by the runtime, not at link time
 if(CMAKE_SYSTEM_NAME STREQUAL "Darwin")

barretenberg/cpp/src/barretenberg/nodejs_module/avm_simulate/avm_simulate_napi.cpp

@@ -12,6 +12,8 @@
 #include "barretenberg/vm2/avm_sim_api.hpp"
 #include "barretenberg/vm2/common/avm_io.hpp"
 #include "barretenberg/vm2/simulation/lib/cancellation_token.hpp"
+#include "barretenberg/vm2_wsdb/wsdb_ipc_merkle_db.hpp"
+#include "barretenberg/wsdb/generated/wsdb_ipc_client.hpp"
 
 namespace bb::nodejs {
 namespace {
@@ -227,15 +229,13 @@ Napi::Value AvmSimulateNapi::simulate(const Napi::CallbackInfo& cb_info)
             env, ContractCallbacks::get(contract_provider, CALLBACK_REVERT_CHECKPOINT), CALLBACK_REVERT_CHECKPOINT),
     };
 
-    /*****************************
-     *** WorldState (required) ***
-     *****************************/
-    if (!cb_info[2].IsExternal()) {
-        throw Napi::TypeError::New(env, "Third argument must be a WorldState handle (External)");
+    /**********************************
+     *** WSDB IPC path (required) ***
+     **********************************/
+    if (!cb_info[2].IsString()) {
+        throw Napi::TypeError::New(env, "Third argument must be a WSDB IPC path (string)");
     }
-    // Extract WorldState handle (3rd argument)
-    auto external = cb_info[2].As<Napi::External<world_state::WorldState>>();
-    world_state::WorldState* ws_ptr = external.Data();
+    std::string wsdb_ipc_path = cb_info[2].As<Napi::String>().Utf8Value();
 
     /***************************
      *** LogLevel (optional) ***
@@ -281,10 +281,8 @@ Napi::Value AvmSimulateNapi::simulate(const Napi::CallbackInfo& cb_info)
      **********************************************************/
 
     auto deferred = std::make_shared<Napi::Promise::Deferred>(env);
-    // Run on a dedicated std::thread (not libuv pool) to prevent libuv thread pool
-    // exhaustion when callbacks need libuv threads for I/O.
     ThreadedAsyncOperation::Run(
-        env, deferred, [data, tsfns, logger_tsfn, ws_ptr, cancellation_token](msgpack::sbuffer& result_buffer) {
+        env, deferred, [data, tsfns, logger_tsfn, wsdb_ipc_path, cancellation_token](msgpack::sbuffer& result_buffer) {
             // Collect all thread-safe functions including logger for cleanup
             auto all_tsfns = tsfns.to_vector();
             all_tsfns.push_back(logger_tsfn);
@@ -309,10 +307,14 @@ Napi::Value AvmSimulateNapi::simulate(const Napi::CallbackInfo& cb_info)
                                                  *tsfns.commit_checkpoint,
                                                  *tsfns.revert_checkpoint);
 
-                // Create AVM API and run simulation with the callback-based contracts DB,
-                // WorldState reference, and optional cancellation token
+                // Connect to aztec-wsdb and wrap in a WsdbIpcMerkleDB that implements
+                // LowLevelMerkleDBInterface. The connection is per-simulation; aztec-wsdb is a
+                // long-running server that the TS layer spawned and owns.
+                bb::wsdb::WsdbIpcClient wsdb_client(wsdb_ipc_path);
+                bb::avm2::simulation::WsdbIpcMerkleDB merkle_db(wsdb_client, inputs.ws_revision);
+
                 avm2::AvmSimAPI avm;
-                avm2::TxSimulationResult result = avm.simulate(inputs, contract_db, *ws_ptr, cancellation_token);
+                avm2::TxSimulationResult result = avm.simulate(inputs, contract_db, merkle_db, cancellation_token);
 
                 // Serialize the simulation result with msgpack into the return buffer to TS.
                 msgpack::pack(result_buffer, result);

barretenberg/cpp/src/barretenberg/nodejs_module/avm_simulate/avm_simulate_napi.hpp

@@ -25,7 +25,7 @@ class AvmSimulateNapi {
      * - info[1]: Object with contract provider callbacks:
      *   - getContractInstance(address: string): Promise<Buffer | undefined>
      *   - getContractClass(classId: string): Promise<Buffer | undefined>
-     * - info[2]: External WorldState handle (pointer to world_state::WorldState)
+     * - info[2]: WSDB IPC path (string) — TS layer spawned aztec-wsdb at this path
      * - info[3]: Log level number (0-7)
      * - info[4]: External CancellationToken handle (optional)
      *

barretenberg/cpp/src/barretenberg/nodejs_module/init_module.cpp

@@ -2,12 +2,10 @@
 #include "barretenberg/nodejs_module/lmdb_store/lmdb_store_wrapper.hpp"
 #include "barretenberg/nodejs_module/msgpack_client/msgpack_client_async.hpp"
 #include "barretenberg/nodejs_module/msgpack_client/msgpack_client_wrapper.hpp"
-#include "barretenberg/nodejs_module/world_state/world_state.hpp"
 #include "napi.h"
 
 Napi::Object Init(Napi::Env env, Napi::Object exports)
 {
-    exports.Set(Napi::String::New(env, "WorldState"), bb::nodejs::WorldStateWrapper::get_class(env));
     exports.Set(Napi::String::New(env, "LMDBStore"), bb::nodejs::lmdb_store::LMDBStoreWrapper::get_class(env));
     exports.Set(Napi::String::New(env, "MsgpackClient"),
                 bb::nodejs::msgpack_client::MsgpackClientWrapper::get_class(env));