Qwen3.5-MoE CUDA V2 foundation: one model, many isolated sessions

Proves the V2 core independently of any serving change: one physical AOTI-CUDA
Qwen model with weights loaded once can host multiple logical sessions, each with
its own KV/conv/recurrent state, with no cross-session bleed. The mechanism is
the AOTI user-managed-constant rebind, kept CUDA-backend-private; the public
surface stays LLMEngine/LLMSession. No multi-session serving wiring here (that is
a follow-up PR) — the worker/runner still create a single session.

Review order:
1. examples/models/qwen3_5_moe/export.py - the model-facing change: emits
   get_mutable_buffer_metadata ({"version":1,"mutable_buffers":[fqn...]}) naming
   the model's per-session state buffers (FQN list only; the backend owns the
   tensor descriptors). The FQNs come from the model's explicit module contract.
2. backends/cuda/runtime/cuda_mutable_state.h - CUDA-private API, keyed by a
   per-engine context (not a generic Module/Method/BackendInterface API, and not
   process-global).
3. backends/cuda/runtime/cuda_mutable_state.cpp - context-keyed manager:
   descriptor table + initial-template capture at load, per-session GPU buffers
   cloned from the template, rebind via update_user_managed_constant_buffer_pairs,
   and declared-vs-discovered FQN coverage validation.
4. backends/cuda/runtime/cuda_backend.cpp - two hooks: note_handle in init(),
   rebind_for_execute in execute().
5. examples/models/qwen3_5_moe/qwen35_moe_engine.{h,cpp} - the engine owns one
   shared Module and its context; sessions rebind their own state under the
   engine lock; adds serving_capacity, capacity enforcement, the coverage check,
   and context teardown. Also removes the now-incompatible cuda_graph flag (a
   captured graph's baked pointers would ignore per-session rebinds), dropping it
   from the engine config, the qwen3_5_moe_worker/runner (main.cpp) CLIs, and the
   README/model.md docs.
6. backends/cuda/{CMakeLists.txt,runtime/TARGETS} and the qwen CMakeLists - build
   wiring (CMake + Buck).

Validated on the real 35B-A3B model (HQQ-INT4) with two interleaved sessions:

  after engine load: 17983 MB | capacity max_sessions=4, est 117309440 B/session
  A solo : 348 10 4838 1665 15 16 17 18 19 20 21 22
  A inter: 348 10 4838 1665 15 16 17 18 19 20 21 22   (identical -> no bleed)
  GPU: engine=17983MB, +2 sessions=+108MB   (weights once; ~112 MiB state/session)

Falls closed to single-session capacity if the AOTI constant-management symbols
are absent or the declared mutable FQNs do not fully match the loaded methods.

MLX V2 (shared constants + per-session MutableBufferData) is the next backend and
is not addressed here.


ghstack-source-id: 6bd59c4
ghstack-comment-id: 4652591759
Pull-Request: #20117

Author: mergennachin

backends/cuda/CMakeLists.txt

@@ -179,7 +179,9 @@ install(
 )
 
 # CUDA backend implementation
-set(_aoti_cuda_backend_sources runtime/cuda_backend.cpp)
+set(_aoti_cuda_backend_sources runtime/cuda_backend.cpp
+                               runtime/cuda_mutable_state.cpp
+)
 if(_cuda_is_msvc_toolchain)
   # MSVC links aoti_cuda_backend into portable_lib without relying on C++
   # symbols exported from aoti_cuda_shims.dll.

backends/cuda/runtime/TARGETS

@@ -105,9 +105,11 @@ runtime.cxx_library(
     name = "cuda_backend",
     srcs = [
         "cuda_backend.cpp",
+        "cuda_mutable_state.cpp",
     ],
     headers = [
         "cuda_delegate_handle.h",
+        "cuda_mutable_state.h",
     ],
     # @lint-ignore BUCKLINT: Avoid `link_whole=True` (https://fburl.com/avoid-link-whole)
     link_whole = True,

backends/cuda/runtime/cuda_backend.cpp

@@ -42,6 +42,7 @@
 #include <executorch/backends/aoti/utils.h>
 #include <executorch/backends/cuda/runtime/cuda_allocator.h>
 #include <executorch/backends/cuda/runtime/cuda_delegate_handle.h>
+#include <executorch/backends/cuda/runtime/cuda_mutable_state.h>
 #include <executorch/backends/cuda/runtime/platform/platform.h>
 #include <executorch/backends/cuda/runtime/shims/memory.h>
 #include <executorch/backends/cuda/runtime/utils.h>
@@ -466,6 +467,10 @@ class ET_EXPERIMENTAL CudaBackend final
           kCudaGraphWarmupSteps);
     }
 
+    // Record whether this AOTI build exposes the constant-management symbols
+    // needed for per-session mutable-buffer rebinding (CUDA V2 multi-session).
+    mutable_state_note_handle(handle);
+
     return (DelegateHandle*)handle; // Return the handle post-processing
   }
 
@@ -514,6 +519,12 @@ class ET_EXPERIMENTAL CudaBackend final
           static_cast<int>(device_type));
     }
 
+    // CUDA V2 multi-session: if a logical session is active on this thread,
+    // rebind this container's mutable constants (KV/conv/recurrent) to the
+    // session's own GPU buffers before running. No-op for
+    // single-session/legacy.
+    ET_CHECK_OK_OR_RETURN_ERROR(mutable_state_rebind_for_execute(handle));
+
     // ---------------------------------------------------------------
     // CUDA graph REPLAY path — skip all tensor setup and just replay
     // ---------------------------------------------------------------