Fused quant linear kernel (#19490)

[DrJessop]

Summary:

Fused quant linear kernel (out = inp @ weight^T + bias) with optional dequantize/quantize. Supports 4 sets of qparams (inp, weight, bias, out), optional bias, and per-tensor/per-channel quantization.

Reviewed By: mvartani-meta

Differential Revision: D103754853

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19490

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode

❌ 39 New Failures, 2 Unrelated Failures

As of commit 5bf8d39 with merge base 7cd209d ():

NEW FAILURES - The following jobs have failed:

• Build Presets / apple (ios-simulator) / build (gh)
• Build Presets / apple (ios) / build (gh)
• Build Presets / apple (llm) / build (gh)
• Build Presets / apple (macos) / build (gh)
• Build Presets / apple (profiling) / build (gh)
• Build Presets / apple (pybind) / build (gh)
• Build Presets / windows (pybind) / build (gh)
• Build Presets / windows (windows) / build (gh)
• Lint / lintrunner (gh)
• pull / test-coreml-bc-macos (macos-m2-stable) / macos-job (gh)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/Users/ec2-user/runner/_work/executorch/executorch/test-infra/.github/actions/check-disk-space'. Did you forget to run actions/checkout before running your local action?
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-llama_runner_eager-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-lora-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-lora-multimethod-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-mcu-cortex-m-backend / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-mediatek-models-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-parakeet-xnnpack-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-phi-3-mini-runner-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / test-voxtral-realtime-xnnpack-linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / unittest / linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / unittest-buck / linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• pull / unittest-editable / linux / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test ARM Backend / test-arm / test-backend-linux (arm_tosa_fp, models) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test ARM Backend / test-arm / test-backend-linux (arm_tosa_fp, operators) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test ARM Backend / test-arm / test-backend-linux (arm_vgf_fp, models) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test ARM Backend / test-arm / test-backend-linux (arm_vgf_fp, operators) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test QNN Backend / test-qnn / test-backend-linux (qnn, models) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test QNN Backend / test-qnn / test-backend-linux (qnn, operators) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test Vulkan Backend / test-vulkan / test-backend-linux (vulkan, models) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test Vulkan Backend / test-vulkan / test-backend-linux (vulkan, operators) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test XNNPACK Backend / test-xnnpack / test-backend-linux (xnnpack, models) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory
• Test XNNPACK Backend / test-xnnpack / test-backend-linux (xnnpack, operators) / linux-job (gh)
  An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/executorch/executorch/pytorch/executorch'. No such file or directory

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / unittest / windows / windows-job (gh) (matched win rule in flaky-rules.json)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under 'C:\actions-runner\_work\executorch\executorch\test-infra\.github\actions\teardown-windows'. Did you forget to run actions/checkout before running your local action?
• pull / unittest-editable / windows / windows-job (gh) (matched win rule in flaky-rules.json)
  Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under 'C:\actions-runner\_work\executorch\executorch\test-infra\.github\actions\teardown-windows'. Did you forget to run actions/checkout before running your local action?

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@DrJessop has exported this pull request. If you are a Meta employee, you can view the originating Diff in D103754853.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.