Arm backend: Lower grid_sampler_2d to VGF TOSA CUSTOM by bdemirb · Pull Request #19547 · pytorch/executorch

bdemirb · 2026-05-13T12:28:07Z

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

pytorch-bot · 2026-05-13T12:28:12Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19547

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Unrelated Failures, 2 Unclassified Failures

As of commit be760f4 with merge base c5da8fb ():

NEW FAILURES - The following jobs have failed:

pull / test-multimodal-linux (gemma3-4b) / linux-job (gh)
RuntimeError: Command docker exec -t e85bf770cb24363f8818bbf6086da58e84defcae407970457e4fc70cefe605a1 /exec failed with exit code 139
trunk / test-models-macos-coreml (resnet50) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
trunk / test-models-macos-cpu (ic4, xnnpack-quantization-delegation) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

Build macOS Wheels / pytorch/executorch / build-wheel-py3_10-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Process completed with exit code 1.
Build macOS Wheels / pytorch/executorch / upload / upload-wheel-py3_10-cpu (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

trunk / test-arm-backend-ethos-u (test_smaller_stories_llama) / linux-job (gh) (trunk failure)
##[error]The operation was canceled.
trunk / test-coreml-delegate / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 65

This comment was automatically generated by Dr. CI and updates every 15 minutes.

bdemirb · 2026-05-13T12:32:23Z

@pytorchbot label "partner: arm"

bdemirb · 2026-05-13T12:32:34Z

@pytorchbot label "release notes: arm"

robell

Reviewed internally,

zingo · 2026-05-13T13:18:37Z

Hi @digantdesai this one is probably good to tests internally at Meta :)

meta-codesync · 2026-05-13T15:18:35Z

@digantdesai has imported this pull request. If you are a Meta employee, you can view this in D105009413.

zingo · 2026-05-18T18:57:21Z

Hi @digantdesai did the internal testing work? This also need a re-import after a small update.

digantdesai · 2026-06-01T16:19:15Z

Can you please rebase?

linux-foundation-easycla · 2026-06-01T17:48:02Z

The committers listed above are authorized under a signed CLA.

✅ login: bdemirb / name: Baris Demir (52830fe, 69b7418, 76e6e22, bb7c4f0)

digantdesai · 2026-06-01T22:37:52Z

There were some changes in the CLA, I guess from the Foundation move, can you try it again?

This patch builds on the preceding Arm TOSA custom-op infrastructure changes in Gerrit changes 1222279, 1222280, and 1222278. On top of that base, it adds the first concrete use case for grid_sampler_2d by preserving aten.grid_sampler_2d through partitioning and rewriting it in the Arm lowering pipeline to tosa.CUSTOM instead of decomposing it into a large standard TOSA subgraph. The lowering attaches CustomAttribute metadata carrying a placeholder shader payload in implementation_attrs, adjusts the fake tosa.CUSTOM op to behave as a single-output tensor op so it composes correctly with the existing Arm TOSA pipeline, and adds a focused regression test showing that a torch.nn.functional.grid_sample model lowers successfully and produces exactly one CUSTOM op in the serialized TOSA graph. Signed-off-by: Baris Demir <baris.demir@arm.com> Change-Id: I183afd344715e38f613db89ad8945060df7f2f1f

Refactor the grid_sampler_2d TOSA CUSTOM payload into a helper module so the custom shader contract, constants, and validation logic live in one place. Emit the ML SDK Vulkan custom-shader attribute format, including the converter-recognized domain, entry point, workgroup sizes, shader language/code, tensor storage-buffer bindings, descriptor sets, and Vulkan format metadata. Add focused tests for payload round-trip behavior and for the grid_sampler_2d rewrite pass. Signed-off-by: Baris Demir <baris.demir@arm.com> Change-Id: I8f13f91b97e86b3b32bf2c25479684bc09faeddc

Add a VGF export/integration test for grid_sampler using the existing VgfPipeline. The test is gated on model-converter availability and runs with quantization disabled and Vulkan runtime execution disabled, so it validates the converter/export handoff without depending on runtime shader dispatch. Also register grid_sampler as a recognized custom edge op for the Arm test-name validator. Signed-off-by: Baris Demir <baris.demir@arm.com> Change-Id: I495d4283b09de7dedebce5d2f24b240e79d37dfc

Change-Id: Iae92c23da049949f5f1d0719922d84581893d4f7

…h#19547 Signed-off-by: Rob Elliott <Robert.Elliott@arm.com> Change-Id: Ic30ddbbdbf0a08d724d2dde9d2f6432918091932 Signed-off-by: Rob Elliott <Robert.Elliott@arm.com>

digantdesai

Thanks, and apologies for the lag.

@digantdesai

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani --------- Signed-off-by: Baris Demir <baris.demir@arm.com>

bdemirb requested a review from digantdesai as a code owner May 13, 2026 12:28

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 13, 2026

github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels May 13, 2026

pytorch-bot Bot added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label May 13, 2026

pytorch-bot Bot added the release notes: arm Changes to the ARM backend delegate label May 13, 2026

bdemirb changed the title ~~Baris mletorch 1945 part2 2 vgf integration~~ Arm backend: Lower grid_sampler_2d to VGF TOSA CUSTOM May 13, 2026

robell reviewed May 13, 2026

View reviewed changes

Comment thread pyproject.toml

zingo added the help wanted Extra attention is needed label May 13, 2026

digantdesai reviewed May 13, 2026

View reviewed changes

Comment thread backends/arm/vgf/shaders/grid_sampler.spirv.b64

digantdesai reviewed May 13, 2026

View reviewed changes

Comment thread pyproject.toml

bdemirb force-pushed the baris_mletorch-1945-Part2_2_vgf_integration branch from 271a05b to 1020f0f Compare May 13, 2026 16:48

digantdesai reviewed May 20, 2026

View reviewed changes

Comment thread backends/arm/vgf/backend.py

bdemirb force-pushed the baris_mletorch-1945-Part2_2_vgf_integration branch from 288e48f to b0c25a2 Compare June 1, 2026 17:47

bdemirb added 4 commits June 2, 2026 07:38

Arm backend: Package VGF custom shader passes

52830fe

Change-Id: Iae92c23da049949f5f1d0719922d84581893d4f7

bdemirb force-pushed the baris_mletorch-1945-Part2_2_vgf_integration branch from b0c25a2 to 52830fe Compare June 2, 2026 06:39

Merge branch 'main' into baris_mletorch-1945-Part2_2_vgf_integration

be760f4

digantdesai approved these changes Jun 3, 2026

View reviewed changes

digantdesai merged commit b5f8155 into pytorch:main Jun 3, 2026
478 of 485 checks passed

robell mentioned this pull request Jun 3, 2026

Arm backend: Add support for shader segment in VGF runtime #19940

Merged

Conversation

bdemirb commented May 13, 2026 • edited by pytorch-bot Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19547

❌ 3 New Failures, 2 Unrelated Failures, 2 Unclassified Failures

Uh oh!

bdemirb commented May 13, 2026

Uh oh!

bdemirb commented May 13, 2026

Uh oh!

robell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zingo commented May 13, 2026

Uh oh!

Uh oh!

Uh oh!

meta-codesync Bot commented May 13, 2026

Uh oh!

zingo commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

digantdesai commented Jun 1, 2026

Uh oh!

linux-foundation-easycla Bot commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

digantdesai commented Jun 1, 2026

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bdemirb commented May 13, 2026 •

edited by pytorch-bot Bot

Loading

pytorch-bot Bot commented May 13, 2026 •

edited

Loading

zingo commented May 18, 2026 •

edited

Loading

linux-foundation-easycla Bot commented Jun 1, 2026 •

edited

Loading