[ET-VK][quantized] Store dq8ca per-token zero-point as fp32

[SS-JIA]

Stack from ghstack (oldest at bottom):

• [ET-VK][ops] Add eq.Scalar operator #20383
• [ET-VK][ops] Add bitwise_or / logical_or operators #20382
• [ET-VK][patterns] Fuse torchao 4-bit quantized embedding to embedding_q4gsw #20381
• -> [ET-VK][quantized] Store dq8ca per-token zero-point as fp32 #20491

The per-token dynamic-activation-quant (dq8ca) zero-point was corrupted by a tensor-allocation vs shader-access dtype mismatch. The per-token zero-point tensor is created with a float dtype -- fp32, or fp16 under USE_VULKAN_FP16_INFERENCE -- so its backing image uses a float texel format (rgba32f / rgba16f). But the shader declared and accessed that image with an integer dtype (int8, an integer image format rgba8i). Reading a float-format image through an integer-format binding is the bug. On ARM Mali (Valhall) GPUs this mismatch corrupted the per-token zero-points: negative zero-points came back as garbage (-k read as -2^23 - k), driving the quantized activation to the int8 floor, the per-group sums to -4096, and the GEMM output to garbage, producing garbled, runaway generation for 8da4w models (e.g. the Llama4-mini TISO TTS backbone on Mali-G715/G710). Adreno happened to tolerate the same mismatch and read correct values, so the corruption was Mali-specific even though the mismatch itself is general.

The per-token zero-point is serialized as fp32 by torchao design: Int8DynamicActivationIntxWeightConfig (8da4w) uses asymmetric per-token activation quant with an explicit fp32 zero_point_dtype. Decoding the serialized .pte confirms the zero-point tensor is FLOAT32, and (like the scale) it is stored in a texture as an rgba32f texel -- never rgba8i. The float allocation is the truth; the int8 shader access was the mismatched side.

The fix is to declare, store, and read the per-token zero-point as fp32 across the dq8ca qparams shaders, so the shader access dtype matches the tensor's allocation dtype and the texture is read as the rgba32f image it actually is. The zero-point value is integer-valued (nudged to [-128, 127]), so fp32 represents it exactly and the consumer's int(zp) conversion for the integer dequant-correction is lossless. This touches the dq8ca qparams shaders -- choose_qparams_per_row, quantize_and_pack_4h4w_with_group_sums, linear_dq8ca_q4gsw_tiled, the shared linear_int8_input_scales_zps_load helper, and the linear_q4gsw_coop variant (whose zero-point binding only matches the descriptor-set layout and is never read) -- plus a documentation comment in ChooseQParams.cpp.

Because the per-token qparams remain in texture storage (unchanged from before) and only the zero-point dtype changes, this is a pure runtime shader fix: existing texture-qparams 8da4w .pte files are corrected without re-export, since the texture already bakes the zero-point as rgba32f and the shader now reads it as such.

Authored with Claude Code.

Differential Revision: D109595977

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20491

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [ROCm] MI350 CI runner label rename: rebase PRs using old linux.rocm.gpu.gfx950.* labels

❌ 4 New Failures, 4 Unrelated Failures

As of commit e7160e7 with merge base 1621fa2 ():

NEW FAILURES - The following jobs have failed:

• pull / test-arm-backend-no-driver (test_pytest_ops_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t a2f42551944eb60cfb6fb74e1186c944be01675016893defb5a2872ff5ba1162 /exec failed with exit code 1
• pull / test-quantized-aot-lib-linux / linux-job (gh)
  RuntimeError: Command docker exec -t bcbfa086a82396832a72f65a6c6b02d2776a6489ac3db1138571540633ebe68f /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5b66cedf02541fac4d329ea5e3e0ff047e0703f9b7ae0dfe959c160c6a5d6dcf /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t bba436b4fadfcce494bac76597c3b6a46a49f1e7002968e4f63bfc09ab3f8d65 /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (trunk failure)
• pull / unittest / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[linux-foundation-easycla]

• ❌ The email address for the commit (e7160e7) is not linked to the GitHub account, preventing the EasyCLA check. Consult this Help Article and GitHub Help to resolve. (To view the commit's email address, add .patch at the end of this PR page's URL.) For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.