### Description
Fixes split heuristic edge cases in Flash Attention and Lean Attention
when the SM count is reported as zero or when there are no key tiles.
The change clamps the SM count used by the heuristics, prevents
divide-by-zero paths, and returns stable buffer sizing for empty tile
cases.

### Changes
- Guard Flash Attention split heuristic against zero SM count and
degenerate split counts.
- Guard Lean Attention split/buffer sizing against zero SM count, zero
key tiles, and single-tile division cases.
- Add CUDA provider regression tests for zero SM count and zero key-tile
inputs.

### Testing
- Added `attention_split_heuristic_test.cc` coverage for Flash Attention
and Lean Attention split heuristics.

Fixes microsoft#29550

… an empty graph (microsoft#29457)

### Description
<!-- Describe your changes. -->
InferenceSession::Initialize() validates, for any EP that has graph
capture enabled and uses the ALLOW_CPU_FOR_SHAPES node-assignment policy
(DirectML, WebGPU), that the partitioned graph is eligible for capture
via AreAllComputeNodesAssignedToEpOrCpu(). That helper currently
requires at least one node to be assigned to the EP:

return has_node_on_provider && !HasMemcpyNodes(graph);

For a graph with no nodes, like a model that is fully consumed by the
EP's runtime graph fusion, or that trivially folds away (like a lone
Constant  node), has_node_on_provider is false, so the session throws:

This session cannot use the graph capture feature as requested by the
user as all compute graph nodes have not been partitioned to the
DmlExecutionProvider

An empty graph contains nothing that violates the "all compute on the EP
or CPU, no Memcpy" requirement, so it should be allowed. This change
permits the empty-graph case:

return (has_node_on_provider || graph.NumberOfNodes() == 0) &&
!HasMemcpyNodes(graph);

Behavior is unchanged for all non-empty graphs (the expression collapses
to the original  has_node_on_provider).

Adds  CApiTest.DmlGraphCaptureEmptyGraph : creates a DML session with
 ep.dml.enable_graph_capture=1 on a single Constant model (folds to an
empty graph) and asserts session creation does not throw. The test fails
without this fix and passes with it.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
microsoft#27958 refactored the graph-capture selection logic and replaced
AreAllComputeNodesAssignedToCudaOrJsOrDmlEpWebGpuEp() (which returned
true for an empty graph) with AreAllComputeNodesAssignedToEpOrCpu() and
its new  has_node_on_provider  requirement. The behavior verified with a
DLL-swap: onnxruntime-directml 1.24.x/1.25.2 create the session
successfully; 1.27 throws.

The failure surfaces through ONNX Runtime GenAI on DirectML: GenAI
creates a per-device "allocator-initialization" session on a
Constant-only model with DML graph capture enabled to obtain a device
allocator (model.cpp). Under ORT 1.27 that session throws at
construction, which breaks every GroupQueryAttention-based GenAI LLM on
the DirectML EP (observed as WinML EP certification failures across
DeepSeek/Llama/Qwen/Phi models

---------

Co-authored-by: Aditya Rastogi <adityar@ntdev.microsoft.com>

…#29444)

This pull request strengthens the safety of arithmetic operations in the
convolution implementation by using `SafeInt` to prevent integer
overflows, especially in cases where tensor shapes may be
attacker-controlled. It also removes some compiler-specific warning
pragmas that are no longer needed due to these changes.

**Enhanced integer safety in convolution calculations:**

* Replaced raw `size_t` multiplications with `SafeInt<size_t>` for
calculating input/output sizes and kernel dimensions, ensuring that any
arithmetic overflow is caught and handled appropriately. This is
particularly important for preventing security vulnerabilities from
attacker-controlled tensor shapes.
[[1]](diffhunk://#diff-3f8429fb413f2d0edd2e6b08b2a516744293e5ba092ab9c9da7b34ed0b28948dL1563-R1566)
[[2]](diffhunk://#diff-3f8429fb413f2d0edd2e6b08b2a516744293e5ba092ab9c9da7b34ed0b28948dL1581-R1594)
* Used `SafeInt<size_t>` for all working buffer size calculations,
including those involving thread counts and batch/group products, to
prevent buffer overflows and ensure correct memory allocation.
[[1]](diffhunk://#diff-3f8429fb413f2d0edd2e6b08b2a516744293e5ba092ab9c9da7b34ed0b28948dL1678-R1686)
[[2]](diffhunk://#diff-3f8429fb413f2d0edd2e6b08b2a516744293e5ba092ab9c9da7b34ed0b28948dL1777-L1791)

**Code cleanup and maintenance:**

* Removed MSVC-specific warning suppression pragmas since `SafeInt` now
guards against arithmetic overflow, making these warnings obsolete.
[[1]](diffhunk://#diff-3f8429fb413f2d0edd2e6b08b2a516744293e5ba092ab9c9da7b34ed0b28948dL1331-L1335)
[[2]](diffhunk://#diff-3f8429fb413f2d0edd2e6b08b2a516744293e5ba092ab9c9da7b34ed0b28948dL1777-L1791)
* Added the `core/common/safeint.h` include to provide access to the
`SafeInt` functionality.

…yml. (microsoft#29575)

### Description
<!-- Describe your changes. -->

Don't echo command when setting VSO variable in
mac-cpu-packing-jobs.yml.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

If the command is echoed again with `set -x`, the VSO variable may end
up with a trailing `'`, which is invalid.
Whether this actually happens is intermittent and probably dependent on
the output ordering.

## Description

Adds a CUDA-focused `CudaQuantizer` helper to ONNX Runtime's Python
quantization tools and exports it from `onnxruntime.quantization`. The
helper centralizes the CUDA quantization paths needed by ONNX Runtime
tests and `onnxruntime-genai` model building, covering both per-channel
QMoE weight packing and MatMulNBits blockwise formats without
duplicating that logic in test code.

This also updates the CUDA MoE/QMoE transformer tests to consume the
shared helper, including full-range symmetric offset storage and CUTLASS
mixed-GEMM prepacked layouts.

## Summary of Changes

### Quantization API

| File | Change |
|------|--------|
| `onnxruntime/python/tools/quantization/cuda_quantizer.py` | Adds
`CudaQuantizer` with per-channel INT4/INT8 quantization, CUDA mixed-GEMM
prepacking, MatMulNBits blockwise quantization, CUTLASS prepacked
blockwise quantization, and a pure-PyTorch symmetric blockwise reference
path. |
| `onnxruntime/python/tools/quantization/__init__.py` | Exports
`CudaQuantizer` from `onnxruntime.quantization`. |
| `onnxruntime/python/tools/quantization/qmoe_quantizer.py` | Replaced
by the more general CUDA quantizer implementation. |

### Test Updates

| File | Change |
|------|--------|
| `onnxruntime/test/python/transformers/test_moe_cuda.py` | Replaces
duplicated blockwise quantization/prepacking logic with `CudaQuantizer`
calls. |
| `onnxruntime/test/python/transformers/test_qmoe_cuda.py` | Uses
`CudaQuantizer` for per-channel and blockwise QMoE test setup, including
full-range unsigned offset storage coverage and default prepacked layout
smoke tests. |

## Testing

- Built the CUDA Release configuration.
- `pytest transformers/test_qmoe_cuda.py::TestQMoEIntPrePackSmoke` - 5
passed, 2 subtests passed.
- `pytest transformers/test_qmoe_cuda.py` - 97 passed, 12 skipped, 2
subtests passed.
- Validated without CUDA that importing the quantization module does not
require `torch`, and that torch-backed methods raise a clear error only
when used.
- `python -m py_compile
onnxruntime/python/tools/quantization/cuda_quantizer.py`

## Checklist

- [x] Tests added/updated
- [x] No breaking changes to released APIs
- [ ] CI passes

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

### Description
<!-- Describe your changes. -->
Gather is missing from the transpose optimizer's handler map. This PR
adds HandleGather, which pushes a Transpose past a Gather so the
  transpose can cancel or fuse with neighboring transposes.

  The rewrite is:

  `data -> Transpose(perm) -> Gather(axis=k, indices=scalar_const)`

  becomes

`data -> Gather(axis=perm[k], indices=scalar_const) ->
Transpose(SqueezePerm({perm[k]}, perm))`

Scope is intentionally narrow — only the scalar (0-D) constant-indices
case, which is structurally identical to a Squeeze along the gathered
axis and reuses SqueezePerm for the post-rewrite output perm. Non-scalar
or non-constant indices are left untouched. The framework's
DefaultCostCheck still gates the rewrite on profitability, so
unprofitable cases are rejected before the handler runs.

Tests added in transpose_optimizer_test.cc cover the positive
scalar-indices case, negative-axis normalization, rank-1-indices
fall-through, and dynamic-indices fall-through.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

We've observed that the Transpose → Gather → Transpose pattern produces
sub-optimal inference performance on the QNN EP. With this
handler in place, the surrounding transposes can fold or cancel,
eliminating layout-conversion overhead around Gather.

---------

Signed-off-by: Mu-Chein Hsu <quic_muchhsu@quicinc.com>

…9499)

## Summary

This PR adds CUDA `MatMulNBits` support for weights that have already
been packed into the fpA_intB SM80 weight-only layout. It lets offline
tooling provide prepacked int4/int8 weights while preserving the
existing runtime-prepack path for standard MatMulNBits initializers.

## Key Changes

- Adds the `weight_prepacked` attribute to `com.microsoft::MatMulNBits`:
- `0`: standard MatMulNBits packed layout, runtime-prepacked by CUDA
fpA_intB when enabled.
  - `1`: already prepacked in the CUDA SM80 fpA_intB layout.
  - `2`: reserved SM90 layout, rejected for now.
- Routes SM90 mixed FP16/BF16 activation + int4/int8 weight fpA_intB
dispatch through the SM80 CUTLASS layout/kernel path for this operator.
- Enforces strict gating for prepacked weights:
  - requires `onnxruntime_USE_FPA_INTB_GEMM=ON` at build time,
  - requires `ORT_FPA_INTB_GEMM` at runtime,
  - requires FP16/BF16 input `A`,
  - rejects unsupported/reserved prepacked formats.
- Reduces GPU memory usage for device-resident prepacked weights by
keeping the original CUDA `B` tensor alive and passing it directly to
fpA_intB instead of copying it into a second GPU buffer.
- Documents the CUDA prepacked-weight contract and adds focused
positive/error tests.

## Testing

- `PATH=/home/tianlei/git/onnxruntime/.venv/bin:$PATH lintrunner -a`
- `cmake --build build/cu130_fp4_bench/Release --target
onnxruntime_providers_cuda --parallel 16`
- `cd /tmp &&
PYTHONPATH=/home/tianlei/git/onnxruntime/build/cu130_fp4_bench/Release:/home/tianlei/git/onnxruntime/onnxruntime/test/python/quantization
LD_LIBRARY_PATH=/home/tianlei/git/onnxruntime/build/cu130_fp4_bench/Release:/home/tianlei/git/onnxruntime/build/cu130_fp4_bench/Release/onnxruntime/capi:/home/tianlei/cuda13.0/lib64:/home/tianlei/cudnn_9.23_cuda13/lib64:/home/tianlei/cudnn_9.23_cuda13/lib:${LD_LIBRARY_PATH:-}
/home/tianlei/git/onnxruntime/.venv/bin/python
/home/tianlei/git/onnxruntime/onnxruntime/test/python/quantization/test_op_matmulnbits_prepacked_cuda.py
-v`

C++ unit tests were not run because the local build tree was configured
with `onnxruntime_BUILD_UNIT_TESTS=OFF`.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

### Description

Fixes the CUDA packaging pipeline build error (with CUDA arch `52-real`)
that was introduced by microsoft#29451.

### Root cause

The small-M batched GEMV path added in microsoft#29451 defines `half` helpers for
the batched kernel — the `Acc2<half>` accumulator trait and the
`PackNatural` / `DotAccum` / `HorizontalAdd` overloads — inside a single
`#if (!defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 530)` guard that
compiles the *entire* declaration out for archs below `sm_53`.

However, `MatMulFloat4BatchedKernel<half>` is launched from host code
and is therefore instantiated for **every** target architecture,
including `sm_52`. During the `sm_52` device pass those `half` helpers
no longer exist, so `nvcc` reported (100 errors, e.g.):

- `incomplete type "Acc2<half>" is not allowed`
- `no suitable user-defined conversion from "WPack<half>" to "const
WPack<nv_bfloat16>" exists` (the compiler fell back to the `nv_bfloat16`
overloads)

### Fix

Mirror the existing `nv_bfloat16` convention in the same file: always
define the type and function **signatures**, and gate only the
arch-specific (half2-intrinsic) **bodies**. Now `Acc2<half>` is always a
complete type and the `half` overloads always exist, so
`MatMulFloat4BatchedKernel<half>` compiles for `sm_52`. On archs below
`sm_53` the bodies compile to no-ops (returning zeros), which matches
the already-shipped `nv_bfloat16` behavior for archs below `sm_80`.

### Motivation and Context

The CUDA packaging pipeline builds with
`CMAKE_CUDA_ARCHITECTURES=52-real;61-real;75-real;86-real;89-real;90-virtual`,
so the `sm_52` device pass is required and the pipeline was broken by
microsoft#29451.

### Description
Move the `requests` imports in `convert_tf_models_to_pytorch.py` into
the two download paths that actually use them
(`download_compressed_file` and the non-archive path in
`download_tf_checkpoint`). This lets the archive-extraction tests import
the module without requiring `requests` to be installed.

### Motivation
The transformer conversion tests import the converter module at pytest
**collection** time (`from convert_tf_models_to_pytorch import
safe_extract_archive`) purely to exercise `safe_extract_archive`.
However, `requests` is **not** a declared dependency of the transformers
tests — `onnxruntime/python/tools/transformers/requirements.txt` lists
`onnx`, `numpy`, `transformers`, `torch`, etc., but not `requests` (it
is normally present only transitively via `transformers`).

When `requests` is absent, the module-level `import requests` raises
`ModuleNotFoundError` at collection time, which fails the
`safe_extract_archive` traversal/symlink tests even though those tests
never touch the download paths that use `requests`. Deferring the import
into the download functions decouples the archive-safety tests from this
undeclared optional dependency.

Note: this is about dependency *availability*, not import speed —
importing `requests` is cheap (its own module code is ~1 ms; the full
transitive tree is a one-time ~0.25 s cold, mostly stdlib). Adding
`requests` to the transformers test requirements would be an equivalent
fix; the lazy import is just the smaller, more localized change.

### Testing
- Reproduced the original failure in a venv without `requests`:
importing the module raises `ModuleNotFoundError: No module named
'requests'`. After moving the imports, `safe_extract_archive` imports
and runs without `requests`.
- Direct `safe_extract_archive` tar/zip traversal checks passed with
`.\.venv\Scripts\python.exe`.
- `.\.venv\Scripts\python.exe -m pytest
onnxruntime\test\python\transformers\test_convert_tf_models_to_pytorch.py
-q` was attempted locally but collection requires `torch`, which is not
installed in this venv.

Co-authored-by: Gopalakrishnan Nallasamy <gnallasamy@microsoft.com>

… flag to support Gemma4 (microsoft#29392)

## Summary

These four ops were forcing CPU fallback when inputs were INT64,
preventing WebGPU graph capture. Converted from static macro
registration to the factory function pattern (same as
Cast/Unsqueeze/Expand/Range) so INT64 type constraints are included when
`enable_int64=true`.

`enable_int64` is always `true` when `enable_graph_capture=true` —
`GetKernelRegistry` calls `RegisterKernels(true, true)` unconditionally
in that path — so this fix has no effect on the default
(non-graph-capture) execution path.

## Changed ops

| Op | Versions |
|---|---|
| `Equal` | 7–10, 11–12, 13–18, 19+ |
| `Sub` | 7–12, 13, 14+ |
| `Where` | 9–15, 16+ |
| `ReduceSum` | 1–10, 11–12, 13+ |

## Approach

Each op follows the established factory function pattern:

- `CreateXVersionedKernelInfo<Start, End>(bool enable_int64)` /
`CreateXKernelInfo<Since>(bool enable_int64)`
- Removed from static `build_kernel_create_info_function_table[]`
- Registered in `RegisterKernels()` alongside
Cast/Unsqueeze/Expand/Range

`ReduceSum` version 13+ retains `InputMemoryType(OrtMemTypeCPUInput, 1)`
(axes input must be on CPU) — unchanged from the original registration.

`Where` factory functions delegate to
`GetOpTypeConstraints(enable_int64, /*enable_bool=*/true)` rather than a
local duplicate type list. All three files (`where.cc`,
`binary_elementwise_ops.cc`, `reduction_ops.cc`) use stack-allocated
`KernelDefBuilder()` consistent with other WebGPU factory functions, and
drop the unused `webgpu_execution_provider.h` include.

## Runtime fixes for INT64 inference

`BinaryElementwise` (`Sub`, `Equal`) and `Where` required additional
shader-level fixes to correctly handle INT64 tensors at inference time:

- **Vectorization disabled for INT64**: INT64 is stored as `vec2<u32>`
(8 bytes/element) and has no vec4 representation. Vectorization is now
disabled when either input is INT64, and `component=1` is used for
correct buffer binding.
- **Correct dispatch size**: `vec_size` is computed as element count
(not rounded-up vec4 count) for INT64 outputs.
- **Scalar input path**: The scalar broadcast path no longer applies an
extra `.x` dereference on top of `GetByOffset`, which already extracts
the `i32` element for INT64.
- **Where non-broadcast path**: Non-broadcast INT64 `Where` now uses the
element-per-thread shader path instead of the vec4 fast path that would
truncate values to i32.
- **Cache key**: `is_int64` is included in the `Where` cache hint,
preventing float and INT64 shaders from incorrectly sharing a cached
entry.
- **Shape indices for INT64**: Shape indices are registered for
non-broadcast INT64 `Where` so `BroadcastedIndicesToOffset` has the
required helpers available.
- **Equal INT64 with bool output**: The element-wise path now explicitly
reads 4 consecutive INT64 elements per thread and packs them into a
`vec4<bool>` before writing to the bool output buffer. Previously, only
element 0 was read and broadcast across all 4 positions of the
comparison, giving wrong results.

## Testing

- All 1174 nodes of a Gemma4 WebGPU decoder model assigned to
`WebGpuExecutionProvider` (previously ~20 nodes fell back to CPU due to
INT64 Equal/Sub/Where/ReduceSum)
- End-to-end graph capture inference: ~87 tok/s on Gemma4-E2B-it INT4
- `USE_WEBGPU`-guarded unit tests added for all four ops with
`kEnableInt64=1`: `Sub_int64_webgpu`, `Equal_int64_webgpu`,
`Where_int64_webgpu`, `ReduceSum_int64_webgpu`

## Related

- Indirect dispatch fix (prerequisite for graph capture): microsoft#29236
- onnxruntime/mobius#380

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

…icrosoft#29545)

### Description

Official Windows DLLs embed the wrong version in their VersionInfo
resource, for example File version 1.27.26.615 for the 1.27.0 release.
The cmake fix from microsoft#24606 is intact but only covers local builds;
official packages are produced by the ADO pipelines, which take a
different path in tools/ci_build/build.py. When Build_BuildNumber and
Build_SourceVersion are set, that branch stamps VERSION_BUILD_PART from
the last two digits of the build year and drops the patch version from
the version string entirely. I checked the shipped 1.22.0, 1.23.0,
1.26.0, and 1.27.0 Windows packages and all four carry the wrong
version, so this path was never fixed.

This change parses major.minor.patch from VERSION_NUMBER, stamps the
real ORT version in the numeric FILEVERSION (keeping MMDD as the fourth
part for build traceability; it fits a 16-bit WORD), and includes the
patch in the version string, which becomes for example
1.27.0.20260615.4.8f0278c. Local builds are unaffected. One open
question for reviewers: whether you would rather have the fourth numeric
part be the pipeline revision or zero instead of MMDD.

### Motivation and Context

Fixes microsoft#29536. Anyone inspecting DLL properties, or tooling that reads
VersionInfo to identify the installed ONNX Runtime version, currently
sees a build-date artifact instead of the actual release version.

Verified by executing the pristine and patched stamping logic with the
real 1.27.0 release inputs (the pristine output reproduces the issue
exactly), and by compiling onnxruntime.rc with the Windows SDK rc.exe
under both define sets and decoding VS_FIXEDFILEINFO from the .res:
1.27.26.615 before, 1.27.0.615 after. A full pipeline build was not run;
confirming end to end means building with Build_BuildNumber and
Build_SourceVersion set and checking the DLL's FileVersionRaw.

### Description
<!-- Describe your changes. -->

Enables `GetModel()` in extended minimal builds.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Fixes microsoft#29386. Extended
minimal builds with the Core ML Execution Provider fail because getting
the model metadata in coreml_execution_provider.cc on L63 requires
Graph::GetModel(), which is only available in full (non-minimal) builds.

…wn (microsoft#29149)

Recover Conv/ConvTranspose rank from weight when input shape is unknown,
enabling layout transformation to NHWC for more nodes.

### Description
The layout transformer skips converting a node to NHWC when input[0] has
no inferred shape.

For Conv and ConvTranspose operators, the data input (input[0]) and the
weight (input[1]) always share the same rank. When the input rank is
unknown, recover it from the weight.

### Performance Impact
Measured on Kokoro-82M-v1.0-ONNX text-to-speech model
([onnx-community/Kokoro-82M-v1.0-ONNX](https://huggingface.co/onnx-community/Kokoro-82M-v1.0-ONNX))
with WebGPU ep,

|      Platform      | Latency reduction | Speedup |
|:------------------:|:-----------------:|:-------:|
| Intel Wildcat Lake | −32.0%            | 1.47×   |
| Intel Panther Lake | −20.0%            | 1.25×   |

This change yields a 1.2–1.5× speedup on the Kokoro-82M text-to-speech
model.

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…Bits bias (microsoft#29596)

### Description

Fixes a build break in the internal CUDA-EP unit-test module
(`onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS=ON`).

`TryMatMulNBits` in `matmul_nbits.cuh` is a function template:

```cpp
template <class T>
bool TryMatMulNBits(int bits, T* output, const T* a_data, const uint8_t* b_data_quant,
                    const T* scales_data, const uint8_t* zero_points, const T* bias_data,
                    int m, int n, int k, int block_size, size_t shared_mem_per_block, cudaStream_t stream);
```

The MoE router bias fusion change (microsoft#29170) added the `const T*
bias_data` parameter. The internal CUDA-EP test
`fpA_intB_gemm_kernel_test.cc` calls this with a bare `nullptr` for
`bias_data`. Since `nullptr` has type `std::nullptr_t` (not a pointer),
it fails to match `const T*` during template argument deduction, even
though `T` is deducible from the other arguments. GCC reports:

```
error: no matching function for call to 'TryMatMulNBits(...)'
note:   mismatched types 'const T*' and 'std::nullptr_t'
```

This is compiled only when
`onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS=ON`, so the internal CUDA-EP
unit-test build leg is currently broken on main.

### Key Changes

- Cast the `bias_data` argument at the call site to `static_cast<const
AType*>(nullptr)` so `T` deduces correctly and the existing no-bias path
is exercised unchanged.

### Motivation and Context

Restores the ability to build and run the internal CUDA-EP unit tests
(e.g. `./onnxruntime_provider_test --gtest_filter=CUDA_EP_Unittest.*`).

### Validation

- Reproduced the exact "no matching function / mismatched types 'const
T*' and 'std::nullptr_t'" deduction failure with a minimal template
repro; confirmed the typed-nullptr cast compiles cleanly.

…rosoft#29593)

## Summary

- Fix an out-of-bounds/data-race in the WebGPU GQA decode split-reduce
shader (`flash_attention_decode_qkv.wgsl.template`) that corrupts the
upper half of each head when `v_head_size_vec < tile_size_k_vec`.
- Correct 16 stale `seqlens_k` values in the WebGPU
`GroupQueryAttention` op tests (`group-query-attention.jsonc`).

## Motivation

The experimental "Run ort-web tests - WebGPU EP" CI stage had 9 failing
`GroupQueryAttention` op tests (tensor mismatches). Root cause is two
independent bugs:

**1. Kernel out-of-bounds race.** In the fused QKV decode shader, the
V-multiply reduction writes `tile_output[m][k + local_idx]` (where
`tile_output` is sized `v_head_size_vec`) but is guarded only by `if
(local_idx < tile_size_k_vec)` (`tile_size_k_vec = 8`). When
`v_head_size_vec < tile_size_k_vec` (e.g. `head_size = 8` →
`v_head_size_vec = 2`), threads with `local_idx` in `2..7` compute an
out-of-bounds index. WGSL clamps OOB array indices to the last valid
element, so those threads all race on `tile_output[m][1]` with a `+=`,
corrupting the second head_size vec4. The manifestation is GPU-dependent
(zeros on some GPUs, off-by-8 stale values on the CI GPU); the first
vec4 stays correct.

Fix: add `&& k + local_idx < v_head_size_vec` to the reduction guard in
both the TurboQuant and non-TurboQuant branches.

**2. Stale test data.** 16 cases in `group-query-attention.jsonc` set
`seqlens_k = total_sequence_length`, but the canonical GQA convention is
`seqlens_k = total_sequence_length - 1` (0-based past length), as used
by the C++ GQA tests and enforced by the CPU kernel bounds check. This
produced an off-by-one KV read on WebGPU and `seqlens_k out of range`
errors on the CPU/wasm backend. The stored expected outputs were already
correct.

## Test plan

- [x] Built wasm + WebGPU EP (`--build_wasm --use_webgpu --use_webnn`)
and deployed to `js/web/dist`.
- [x] Ran `npm test --webgpu-ep -- op group-query-attention -b=webgpu`:
all `[webgpu]` and `[wasm]` GQA cases pass (previously 9 `[webgpu]`
tensor mismatches + CPU `seqlens_k` errors).
- [x] Confirmed the kernel fix alone reduced `[webgpu]` failures 16 → 9,
and the test-data fix cleared the remaining off-by-one cases on both
backends.

…icrosoft#29576)

### Description

`Java_ai_onnxruntime_OrtTrainingSession_evalStep` acquires the output
handle array with `GetLongArrayElements(jniEnv, outputHandlesArr, NULL)`
but never calls the matching `ReleaseLongArrayElements`, so the pinned
array (or its native copy, depending on the JVM) leaks on every call and
on all return paths: the `EvalStep` error `goto`, the output-conversion
failure, and the normal success return.

This adds the release with `JNI_ABORT` immediately after the handle
pointers are copied into `outputValues`, before `EvalStep` is invoked,
so it covers every subsequent path. `JNI_ABORT` is correct because the
handles are only read. The change mirrors the input handling a few lines
above in the same function, and the sibling `trainStep()` and
`OrtSession.run()` bindings, which already release their output handles
this way. Inference results are unaffected, which is why functional
tests do not catch the leak.

### Motivation and Context

Fixes microsoft#29573. `evalStep()` was the only one of the three equivalent
bindings missing the release, so repeated evaluation (for example a
validation loop) accumulated native memory and GC pressure.

… attention (microsoft#28963)

## Description

The quantized KV-cache flash-attention path for the CPU
`GroupQueryAttention` contrib op carried two latent batch-stride bugs
that produced incorrect results for `batch_size > 1`. This PR ports the
fixes already landed for the FP32 path (PR microsoft#28962) to the quantized
path, and adds a regression test that exercises the previously-uncovered
scenario.

## Summary of Changes

### Bug fixes

| File | Change |
|------|--------|
| `onnxruntime/core/mlas/inc/mlas_qkv_quant.h` | Add `q_batch_stride`
field to `MlasFlashAttentionQuantizedKVArgs` so the kernel uses a
caller-supplied Q batch stride instead of assuming the unpacked
`num_heads*S*H` layout. |
| `onnxruntime/core/mlas/lib/flashattn_qkv.cpp` | Use
`args->q_batch_stride` for the Q pointer in both the main tiled kernel
and the flash-decoding kernel; compute the attention-bias batch stride
from `bias_head_extent = broadcast_head ? 1 : num_heads` (two sites)
instead of always using `num_heads`. |
| `onnxruntime/contrib_ops/cpu/bert/gqa_attention_base.h` | Set
`q_batch_stride` to `(num_heads + 2*kv_num_heads)*S*H` for packed QKV
(else `num_heads*S*H`) in the unified dispatch; correct the per-batch Q
offset and bias slice to use the packed stride and head extent. |

### Tests

- Add `test_int8_bias_broadcast_head_multi_batch` to
`test_gqa_cpu_quantized.py` covering `[B, 1, S, T]` bias with
`batch_size > 1`. The existing `test_int8_bias_broadcast_head` used
`batch_size == 1`, which masked the head-broadcast batch-stride bug.

## Testing

- `cd onnxruntime/test/python/transformers && python -m pytest
test_gqa_cpu_quantized.py -q` → 21 passed, 2 skipped.
- Verified the new test catches the bug: temporarily reverting the
bias-stride fix makes `test_int8_bias_broadcast_head_multi_batch` fail
(mismatches starting at batch index 1) while the `batch_size == 1`
variant still passes; restoring the fix makes all tests pass.
- `lintrunner -a` → no lint issues.

## Motivation and Context

Follow-up to PR microsoft#28962, which fixed the identical two bugs (packed-QKV Q
batch stride and attention-bias head-broadcast batch stride) in the
non-quantized FP32 CPU GQA path. The quantized path was left untouched
there as out of scope; existing quantized parity tests did not cover
`batch > 1` with `[B, 1, S, T]` bias, so the bugs went undetected.

…acy directory load path (microsoft#29501)

### Description

Two changes to the model package flow, enabled by the file-path external
initializers support:

**1. Fold `external_data` into `session_options` with path resolution.**
- Path-valued session options (an allowlist:
`session.model_external_initializers_file_folder_path` and
`ep.context_file_path`) are resolved against the package
(`sha256:<hex>`, relative, or absolute) at variant-parse time, using the
same rules as `model_file`.
- The dedicated `external_data` variant field and its special session
injection are removed; the model is always loaded from the selected
variant path.
- On the advanced path (caller supplies their own `OrtSessionOptions`),
path-valued options are carried over from the variant for keys the
caller did not set, so a model that needs its external-initializers
folder still loads.

**2. Remove the legacy directory-based `CreateSession(package_dir)`
path.**
- Removes the `is_directory(package_root)` branch in
`CreateSessionAndLoadModelImpl`. Model packages are loaded through the
experimental `OrtModelPackageApi` (`CreateModelPackageContext` ->
`SelectComponent` -> `CreateSession`).
- The three end-to-end tests that exercised the directory path are
migrated to the experimental API via a `CreateSessionFromModelPackage`
test helper, preserving coverage for factory-based selection,
`PREFER_CPU` policy selection, and compiled-model compatibility scoring.

### Motivation and Context

The file-path external initializers folder support (microsoft#29459) lets the
model package flow drop its workaround of memory-mapping the model and
forcing a buffer load: it sets the folder option and loads from the
selected variant path directly. Building on that, the dedicated
`external_data` field folds into the general session-options mechanism.
A variant declares
`session.model_external_initializers_file_folder_path` (or other
path-valued options) in its `session_options`, and ORT resolves those
values against the package at parse time. This removes special-case code
and lets other path-valued options such as the EPContext file path be
resolved the same way. An allowlist is used rather than value-syntax
sniffing because session-option values are arbitrary strings; only known
path-valued keys are resolved, so ordinary values pass through
untouched.

The legacy directory-based `CreateSession(package_dir)` path was meant
to be removed when the experimental `OrtModelPackageApi` landed. It only
did variant/EP selection and never merged the variant's
`session_options`/`provider_options`, so a package loaded via
`Ort::Session(env, dir, so)` silently ignored its manifest session
options, inconsistent with the experimental API. Removing it leaves a
single, consistent way to load a model package.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# PR: Clamp 1D attention mask_index to valid bounds

## Description

The CPU attention helper translates a 1D `mask_index` (right-side end
position, and
optionally a left-side start position) into per-token mask-fill loop
bounds. The
existing `std::max(0, std::min(...))` clamp was correct but verbose;
this PR replaces it
with `std::clamp` and adds a regression test that exercises out-of-range
mask values.
This hardens the kernel against out-of-bounds writes when callers supply
mask positions
outside `[0, all_sequence_length]`.

## Summary of Changes

### Bounds clamping

| File | Change |
|------|--------|
| `onnxruntime/contrib_ops/cpu/bert/attention_helper.h` | Use
`std::clamp(static_cast<int>(mask_index[...]), 0, all_sequence_length)`
for both `end_position` and `start_position` in the 1D `mask_index`
branch of `PrepareMask`. |

### Test coverage

| File | Change |
|------|--------|
| `onnxruntime/test/contrib_ops/attention_op_test.cc` | Add
`AttentionMaskIndex1DClampOOB`, covering an over-large `end_position`
(999) and a negative `start_position` (-5). Both clamp cleanly and
produce the fully-unmasked output. |

## Testing

- Build and run the contrib attention tests:
`onnxruntime_test_all
--gtest_filter='ContribOpAttentionTest.AttentionMaskIndex1DClampOOB'`
- The new test asserts that an out-of-range end position (above
`all_sequence_length`)
applies no right-side masking, and a negative start position applies no
left-side
  masking — matching a fully-unmasked sequence.
- Existing `ContribOpAttentionTest.*` cases continue to pass; behavior
for in-range
  mask values is unchanged.

## Motivation and Context

`end_position`/`start_position` drive raw index loops into the mask
buffer. An
unclamped value past `all_sequence_length` (or a negative start) could
otherwise step
outside the intended range. `std::clamp` guarantees both stay within
`[0, all_sequence_length]`, preventing out-of-bounds writes with no
behavior change for
valid inputs.

## Checklist

- [x] Tests added/updated
- [ ] Documentation updated (if applicable)
- [x] No breaking changes (or documented in description)
- [ ] CI passes

…-overflow (microsoft#29443)

This pull request improves the safety of buffer size calculations in the
`SamplingState` initialization logic by ensuring that all
multiplications involving `batch_size` and `vocab_size` are safely
performed using `SafeInt<size_t>`. This prevents potential integer
overflow bugs that could lead to under-allocated buffers and memory
errors.

**Buffer allocation safety improvements:**

* All buffer size calculations that multiply `batch_size` and
`vocab_size` now use `SafeInt<size_t>` to ensure checked arithmetic,
preventing silent integer overflows that could cause
heap-buffer-overflow issues. This includes allocations for both CPU and
CUDA buffers in `SamplingState`.
[[1]](diffhunk://#diff-ad3815054e84321b726b1e4c36d32cf2ab224301f699094f33b6ffd81b91eb64L25-R48)
[[2]](diffhunk://#diff-ad3815054e84321b726b1e4c36d32cf2ab224301f699094f33b6ffd81b91eb64L52-R58)
* The calculation for the buffer size of `h_sampled_all` now also safely
casts `max_iter` to `size_t` before multiplication, further protecting
against overflow.

These changes make the code more robust and secure, especially when
handling large or model-controlled input sizes.

…t#29445)

MatMulNBits::PrePack ran at session initialization and called the MLAS
pack routines using byte counts derived from the node attributes (N, K,
bits, block_size) without ever comparing those attributes to the actual
tensor Shape(). A crafted .onnx whose attributes overstate the real B
(or scales / zero_points) extent triggered a heap-buffer-overflow READ
inside MlasQNBitGemmPackQuantBData / MlasLutGemmPack during
OrtApis::CreateSession (no Run() required).

The canonical shape check already lives in
matmul_nbits_helper::CheckInputs, but is invoked only from Compute() --
after PrePack has already done the OOB read, and by then the original B
tensor is replaced with nullptr in the kernel context so the
Compute-time check never re-validates it.

Fix: at the top of PrePack, after the existing early-return guards and
before any tensor.DataRaw() read, validate the incoming initializer's
Shape() against the attribute-derived shape:

  - B           -> (N, k_blocks, blob_size)
  - scales      -> (N * k_blocks) or (N, k_blocks)
  - zero_points -> uint8: (N * zp_blob) or (N, zp_blob); else
                   (N * k_blocks) or (N, k_blocks)

A mismatch returns INVALID_ARGUMENT so the session fails to load rather
than reading past the buffer.

This pull request improves the validation logic for the RotaryEmbedding
operator to prevent out-of-bounds reads when the rotary embedding
dimension derived from `cos_cache` exceeds the input tensor's
`hidden_size`. It also adds dedicated unit tests to verify that this
validation triggers as expected.

**Validation improvements:**

* Added a check in `rotary_embedding_helper.h` to ensure that the
effective rotary embedding dimension (`cos_cache` width × 2) does not
exceed `hidden_size` when `rotary_embedding_dim` is 0, returning an
error if the condition is violated.

**Unit test additions:**

* Added `ContribRotaryEmbedding_RejectsCosCacheExceedsHiddenSize` test
in `rotary_embedding_op_test.cc` to verify that an invalid configuration
is correctly rejected in the contrib op.
* Added `RotaryEmbedding_RejectsCosCacheExceedsHiddenSize` test in
`rotary_embedding_op_test.cc` (providers/cpu/llm) to verify the same
validation in the mainline op.

### Description
<!-- Describe your changes. -->

Prevents buffer overflow for conv node when input images are too large 

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

Fixes microsoft#29130

The ReduceMean hypothesis in the issue was wrong and this addresses the
actual root cause

---------

Co-authored-by: Prathik Rao <prathikrao@microsoft.com>

…icrosoft#29574)

### Description
Remove the blanket version check in CustomOpKernel that prevented custom
ops compiled against a newer ORT from loading on an older runtime.
Instead, cap the version passed to GetApi() at ORT_API_VERSION.

This aligns custom ops with the EP plugin ABI pattern where forward
compatibility is supported via runtime version detection. Individual
newer functions in OrtCustomOp (CreateKernelV2, InferOutputShapeFn,
GetMayInplace, etc.) are already gated by per-function version checks
throughout custom_ops.cc, making the blanket reject both redundant and
harmful.

The EP is responsible for not calling API functions newer than the
runtime version -- the same contract as the EP plugin interface.



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…ety (microsoft#29584)

### Description

Three related fixes in the QMoE (`contrib_ops/cuda/moe`) weight-only
GEMM profiling path that can cause a sticky `CUDA 700` (illegal memory
access) surfacing at a later MoE kernel launch:

- **`moe_kernels.cu`** — `GemmProfilerBackend::runProfiler` now keys its
workspace layout on `mSM >= 90`, the same key used by `getWorkspaceSize`
/ `prepareRouting` / `prepareQuantParams` / `prepareTmaWsInputs`.
Previously it keyed on the per-tactic `is_tma_warp_specialized`, which
diverges on SM90 when a non-TMA (Ampere-fallback) tactic is profiled
(e.g. INT4 mixed-input GEMM tiles). That divergence drops the
`tma_ws_input` region and shifts every subsequent sub-buffer offset, so
the profiler reads `expert_first_token_offset` and the GEMM inputs from
the wrong (random-filled) locations → OOB read in the grouped GEMM.

- **`moe_gemm_profiler`** — `profileTactics`/`runProfiling` accept an
optional `timing_stream`; the profiler runs on the caller-supplied
compute stream when provided, so its kernels are strictly ordered with
surrounding compute-stream work and share the temp allocator's stream
context. Profiling on a private side stream races with the stream-aware
temp arena, which can hand the same scratch block to a later
compute-stream allocation (e.g. the real MoE workspace) while the
profiler's grouped-GEMM kernels are still in flight.

- **`moe_quantization.cc`** — Skip profiling while the compute stream is
being captured into a CUDA graph (profiling launches kernels,
records/synchronizes events, and allocates/frees scratch — all illegal
during capture) and fall back to a config cached from an earlier
non-capturing run, or the runner's default tactic.

### Motivation and Context

These are latent correctness/stability bugs in the QMoE tactic profiler
that manifest on SM90 (H200) and under CUDA graph capture. They are
independent of the fpA_intB MatMulNBits work and are split out as a
standalone fix.

### Testing

- Built with `USE_FPA_INTB_GEMM=ON` (arch 80;90) on H200.
- `test_qmoe_cuda.py` passes (SwiGLU int4/int8, FP16/BF16, multiple
batch/seq shapes).

… MatMulNBits (microsoft#29585)

### Description

Extends the CUDA fpA_intB weight-only `MatMulNBits` path (FP16/BF16
int4/int8) in three ways. All are gated behind the existing
`ORT_FPA_INTB_GEMM` opt-in; default behavior is unchanged.

**1. Native SM90 (Hopper) mixed-GEMM kernel**
- `cutlass_extensions` `GemmFpAIntB::operator()`: reuse the SM80
(Ampere) mixed-GEMM kernel for `__CUDA_ARCH__ >= 900` (Hopper/Blackwell)
instead of stubbing it out, so the SM80-compat dispatch produces a real
kernel on Hopper (mirrors `MoeFCGemm::operator()`).
- `fpA_intB_launcher_sm90.inl`: gate the host-callable launcher on
`COMPILE_HOPPER_TMA_GEMMS` only (not `__CUDA_ARCH__` /
`__NV_SASS_VERSION__`, which are unset in the host pass), so the native
SM90 TMA/WGMMA launcher symbol is emitted instead of the "recompile with
90a" stub.
- `CutlassFpAIntBGemmRunnerInterface`: add `setArch(int)` and
`setUseSm90Native(bool)`; the runner routes SM90 to `sm90_dispatch` when
native, else the SM80 compat path.
- `matmul_nbits`: `FpAIntBPackingSmForKernel()` returns 90 for
`weight_prepacked=2` on an SM90 device (else 80); `InitGemmProfiler`
opts into the native kernel (sm==90) or forces the runner to SM80
(compat on Hopper) so tactic enumeration and workspace sizing match the
dispatched kernel; `GemmIdCore` gains an `sm` field so SM80-compat and
SM90-native tactics for the same shape are not confused; the GEMV is
launched with the packing arch (not the raw device SM) to match the
packed interleave layout.
- `weight_prepacked=2` (SM90 layout) is now accepted (was
reserved/rejected): requires an SM90 device and `block_size ∈ {64,
128}`. Contrib-op docs updated accordingly.

**2. `block_size=32`**
- Relax the group-size gates in the GEMV dispatcher, the CUTLASS
fine-grained `can_implement` / kernel launcher, and the `MatMulNBits`
eligibility check. The SM80 fine-grained kernel supports group size 32
natively.

**3. Fused bias**
- Drop the `!has_bias_` gate: a fused bias (input 5) is already
supported by the GEMV, the SM80/SM90 CUTLASS epilogue, and the tactic
profiler, so bias-bearing nodes (e.g. gpt-oss `qkv_proj`/`o_proj`)
become eligible.

### Motivation and Context

Enables the fpA_intB weight-only path for more shapes and for
Hopper-native execution, unblocking int4/int8 models with fused bias and
`block_size=32` on H200. Builds on microsoft#29499 (prepacked fpA_intB weights).

### Testing

- Built with `USE_FPA_INTB_GEMM=ON` (arch 80;90) on H200.
- C++ `onnxruntime_provider_test
--gtest_filter='*MatMulNBits*:*FpAIntB*'`: 65 passed, 1 skipped.
- `test_op_matmulnbits_prepacked_cuda.py`: 7 passed (int4/int8,
SM80/SM90, bias parity vs runtime prepack).

…osoft#29606)

Update pinned dependencies in the llama and phi2 transformer model
requirements to their patched versions, clearing the flagged 1ES
Component Governance alerts.

Package changes:
- onnx: 1.18.0 -> 1.22.0 (llama, phi2)
- protobuf: 4.25.8 -> 6.33.5 (llama)

onnx 1.18.0 -> 1.22.0 (patched in 1.21.0) resolves:
- CVE-2026-27489: path traversal via symlink (arbitrary file read)
- CVE-2026-34445: unsafe setattr in ExternalDataInfo (object property
overwrite via crafted model)
- CVE-2026-28500: onnx.hub.load() trust-check bypass via silent=True
- GHSA-q56x-g2fj-4rj6: TOCTOU arbitrary file read/write in
save_external_data

protobuf 4.25.8 -> 6.33.5 resolves:
- CVE-2026-0994: recursion-depth bypass in json_format.ParseDict()
causing DoS (no fix in the 4.25.x line; patched in 5.29.6 / 6.33.5)

Versions align with the onnx==1.22.0 / protobuf==6.33.5 pairing already
used across the repo's CI and transformers-test requirements.

Files:
- onnxruntime/python/tools/transformers/models/llama/requirements.txt
- onnxruntime/python/tools/transformers/models/phi2/requirements.txt

### Summary

Two small, independent CI fixes that unblock currently-failing required
pipelines. The NPM packaging pipeline's web e2e consuming test broke
when a floating `vite` range pulled the just-released 7.3.x line, and
the Python DML pipeline started failing on a MeanVarianceNormalization
precision mismatch. Neither change affects runtime code.

### Key Changes

| Pipeline | File | Change | Why |
|---|---|---|---|
| NPM packaging (`web-ci` → e2e) | `js/web/test/e2e/package.json` | Cap
`vite` from `^7.1.12` to `>=7.1.12 <7.3.0` | The e2e runner uses a
non-deterministic `npm install`, so `^7.1.12` floated onto vite 7.3.6
(the tarball the install aborted on). vite 7.3.0 also bumped esbuild
`^0.25.0 → ^0.27.0`, pulling platform binaries not reliably mirrored in
the internal feed. Capping below 7.3.0 keeps the bundler smoke test on
the known-good line. |
| Python DML |
`onnxruntime/test/testdata/onnx_backend_test_series_filters.jsonc` |
Exclude `^test_mvn_cpu` from the DML EP backend test list | `test_mvn`
(MeanVarianceNormalization) fails on DML with 27/27 mismatched elements
(max rel diff ~25) — a precision issue, not a functional regression.
Filtering it matches how other DML precision/known-issue cases are
already handled in this list. |

### Testing Notes

- **NPM packaging**: re-run the web CI e2e step (`npm run test:e2e --
--browser=Chrome_default`); `npm install` in `build/js/e2e` now resolves
vite to a 7.1/7.2 release instead of 7.3.6, so the install no longer
aborts (and the Windows `npm warn cleanup ... EPERM` rollback noise
disappears).
- **Python DML**: re-run the DML python backend test job; `test_mvn_*`
is now skipped for the DML EP alongside the existing excluded cases.

… on Linux (microsoft#29591)

On Linux with the WebGPU EP, if `SessionOptionsAppendExecutionProvider`
is called when no Vulkan adapter is available, a subsequent
`OrtReleaseEnv` on the same thread hangs forever in a futex wait.

### Root causes

**1. C++ exceptions thrown inside Dawn `WaitAny` callbacks (primary)**

`ORT_ENFORCE` was called directly inside the
`RequestAdapter`/`RequestDevice` callback lambdas. Dawn's `WaitAny` does
not release its internal `EventManager` mutex on exception, so throwing
through it leaves that mutex permanently locked. The deadlock fires
later when `Cleanup()` calls `wgpuInstanceRelease` →
`EventManager::ShutDown()` → tries to re-acquire the same mutex on the
same thread.

**2. Zombie `WebGpuContext` left in the factory map (secondary)**

When `Initialize()` threw, the `WebGpuContext` entry remained in
`contexts_` with `ref_count=1` and no owner — a resource leak that would
also re-trigger the deadlock path at `Cleanup()`.

### Changes (`webgpu_context.cc`)

- **Adapter callback**: replace the throwing lambda with a non-throwing
one that writes into a local `RequestAdapterResult` struct;
`ORT_ENFORCE` is moved to *after* `WaitAny` returns:

  ```cpp
  // Before — throws inside Dawn's callback dispatch:
  [](wgpu::RequestAdapterStatus status, wgpu::Adapter adapter,
     wgpu::StringView message, wgpu::Adapter* ptr) {
      ORT_ENFORCE(status == wgpu::RequestAdapterStatus::Success, ...);
      *ptr = std::move(adapter);
  }, &adapter

  // After — captures result, throws outside Dawn:
  [](wgpu::RequestAdapterStatus status, wgpu::Adapter adapter,
     wgpu::StringView message, RequestAdapterResult* result) {
      result->status = status;
      if (status == wgpu::RequestAdapterStatus::Success)
          result->adapter = std::move(adapter);
      else
          result->message = std::string{message};
  }, &adapter_result
// ORT_ENFORCE(adapter_result.status == ...) called here, after WaitAny
  ```

- **Device callback**: same pattern applied to `RequestDevice`.

- **`CreateContext` cleanup**: wrap `Initialize()` in a `try/catch`; on
failure decrement `ref_count` and erase the entry from `contexts_` if it
reaches zero.

### Motivation and Context

Fixes the hang reported when registering the WebGPU EP with no usable
Vulkan adapter and then calling `OrtReleaseEnv`. The process would park
on a futex with `__owner` set to its own TID — a non-recursive mutex
locked twice on the same thread — and never exit.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

…osoft#29271)

1. Double-buffer the B tile in workgroup memory when type is float16.
2. Load A as vec4 when K % 4 == 0.

### Description
<!-- Describe your changes. -->
Optimize the Intel subgroup GEMM/MatMul kernels in the WebGPU EP with
two changes, both gated on the Xe-3LPG architecture:

vec4 A loads: when K is a multiple of 4, A is loaded from global memory
as vec4 (4 consecutive K elements per load) via cooperative subgroup
loading, instead of scalar loads.
Double-buffered B tile: for fp16 B inputs, the B tile in workgroup
memory is double-buffered, prefetching the next tile while computing on
the current one. This overlaps global-memory load latency with compute
and reduces from 2 workgroupBarriers per K-loop iteration to 1.

Both `a_vec4` and `b_is_fp16` are added to the program CacheHint so
distinct pipelines are cached per configuration. On other architectures
the kernels fall back to scalar A loads and a single B buffer,
preserving previous behavior for shared memory limitation and registers
pressure.

Measured speedups on Xe-3LPG (avg ~12.7%):
Model	Speedup
jina-clip-v1-version-fp16	13.4%
sd-v1.5-text-encoder-demo	15.9%
florence-2-base-decoder-fp16	13.3%
moondream2-vision-encoder-fp16	11.4%
sd-turbo-text-encoder-fp16-demo-layernorm	9.6%

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
The Intel subgroup matmul kernels were memory-bound on A loads and
incurred two workgroup barriers per K tile. Loading A as vec4 cuts the
number of global-memory accesses, and double-buffering the B tile hides
load latency behind compute, improving throughput on the fp16
vision/text-encoder workloads above without regressing other
architectures.

…path (microsoft#29525)

### Description

`com.microsoft.GroupQueryAttention`'s optional `attention_bias` input
(input #10) is implemented by the CPU EP (microsoft#23944) and the WebGPU EP
(microsoft#25285, microsoft#26769), but the CUDA EP rejects it at runtime. This PR wires
it through.

No new kernel is needed: the unfused GQA fallback added for microsoft#28195 calls
`LaunchUnfusedAttention`, whose kernel already implements an additive
bias with dim-0/dim-1 broadcast, per-batch `seqlens_k`,
causal/sliding-window masking and softcap — the op just passed
`attn_bias=nullptr`. The change is dispatch plumbing:

- **`group_query_attention.cc`** — remove the blanket rejection;
validate the bias element type; set `broadcast_attn_bias_dim_0/1` from
the bias shape (the fields already exist on `AttentionParameters`); add
`!has_attention_bias` to the XQA / cuDNN SDPA / flash /
flash-fast-decode / MEA eligibility so bias-carrying nodes dispatch to
the unfused fallback; set `data.attention_bias`.
- **`attention_data.h`** — add the `attention_bias` pointer to
`GroupQueryAttentionData`.
- **`group_query_attention_impl.cu`** — pass the real pointer and
broadcast flags in `UnfusedGqaAttention` (previously hardcoded
`nullptr`/`false`).

Why each fused path stays disqualified with a bias:
- flash / flash fast-decode: `flash_api.h` has no bias parameter (same
exclusion as MHA and the ONNX `Attention`-op CUDA kernel).
- XQA: no bias parameter.
- cuDNN SDPA: GQA's cuDNN path is bottom-right causal, which cuDNN
documents as incompatible with a bias (`multihead_attention.cc` has the
same restriction).
- cutlass MEA: the wrapper computes the bias row stride from
`kv_sequence_length`, which GQA sets to the KV-cache capacity
(`seqlen_present_kv_cache`) rather than `total_sequence_length`, so rows
would be misaligned under past/present buffer sharing. Left for a
follow-up (needs an explicit `attn_bias_strideM` in
`MemoryEfficientAttentionParams`).

Kept `NOT_IMPLEMENTED` (explicit, clear errors instead of the previous
blanket rejection): bias × quantized KV cache (unfused requires `T ==
U`), bias × smooth-softmax/head_sink.

### Tests

New `TestGQAAttentionBias` in `test_gqa.py`: prompt and past/decode
parity across packed/unpacked QKV, shared/separate KV buffer, rotary,
odd head sizes (40/80), and a subsequent multi-token prompt. The bias is
**non-zero random** so a kernel that silently ignores the input fails
parity (the harness previously modeled a zeros bias). Also fixes the
test graph builder declaring the bias input's last dim as the KV-cache
capacity instead of `total_sequence_length` (the shape the op
validates).

Full `test_gqa.py` suite: 476 tests pass, no regressions (SM89, CUDA
12.8).

### Motivation and Context

Fixes microsoft#29506.

Transformers.js-exported speech models carry non-causal attention
patterns as `attention_bias` and currently cannot run on the CUDA EP at
all, while running fine on WebGPU and CPU:
-
[`onnx-community/Voxtral-Mini-4B-Realtime-2602-ONNX`](https://huggingface.co/onnx-community/Voxtral-Mini-4B-Realtime-2602-ONNX)
(streaming ASR)
-
[`onnx-community/cohere-transcribe-03-2026-ONNX`](https://huggingface.co/onnx-community/cohere-transcribe-03-2026-ONNX)

End-to-end validation with this patch on an RTX 4070 SUPER: the full
Voxtral-Mini-4B-Realtime streaming pipeline (q4f16) transcribes
correctly at **RTF 0.23** (31 s clip, 25.6 tok/s sustained decode, 6.4
GB VRAM) — on this workload the unfused-attention path outperforms the
same model on the WebGPU EP (RTF 0.26).

(While validating, an unrelated pre-existing issue surfaced:
`GroupQueryAttentionFusion` breaks graphs whose GQA nodes carry >9
inputs — filed as microsoft#29524.)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>