Skip to content

[ET-VK] Add ANY_STORAGE support to softmax and log_softmax#18054

Merged
meta-codesync[bot] merged 8 commits intogh/SS-JIA/471/basefrom
gh/SS-JIA/471/head
Mar 18, 2026
Merged

[ET-VK] Add ANY_STORAGE support to softmax and log_softmax#18054
meta-codesync[bot] merged 8 commits intogh/SS-JIA/471/basefrom
gh/SS-JIA/471/head

Conversation

@SS-JIA
Copy link
Copy Markdown
Contributor

@SS-JIA SS-JIA commented Mar 10, 2026

Stack from ghstack (oldest at bottom):

Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: D95970171

Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Mar 10, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18054

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 11 Cancelled Jobs, 1 Unrelated Failure

As of commit 0ac4c10 with merge base 22174fa (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOBS - The following jobs were cancelled. Please retry:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions
Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

ssjia added 4 commits March 11, 2026 09:52
Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
ssjia added 2 commits March 17, 2026 11:27
Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
Add a buffer shader variant for softmax using workgroup-based reduction with
shared memory. The buffer shader dispatches one workgroup per "row" along the
reduction dimension, with NWORKERS=4 threads per workgroup cooperating on the
max and sum reductions before writing outputs.

In Softmax.cpp, the existing texture dispatch is renamed add_softmax_texture_node
and new add_softmax_buffer_node / add_softmax_node dispatch functions are added.
op_registry.py is updated from ANY_TEXTURE to ANY_STORAGE so the partitioner
accepts buffer-storage inputs. Test cases now also exercise buffer storage.

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)

[ghstack-poisoned]
@meta-codesync meta-codesync Bot merged commit ffe5a6d into gh/SS-JIA/471/base Mar 18, 2026
205 of 220 checks passed
@meta-codesync meta-codesync Bot deleted the gh/SS-JIA/471/head branch March 18, 2026 01:47
@meta-codesync meta-codesync Bot temporarily deployed to cherry-pick-bot March 18, 2026 01:47 Inactive
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
SS-JIA pushed a commit that referenced this pull request Mar 18, 2026
Modernize softmax and log_softmax to support ANY_STORAGE. Migrate both
buffer and texture shaders from indexing_utils.h to indexing.glslh with
BufferMetadata/TextureMetadata UBOs. Merge separate texture and buffer
dispatch functions into a unified add_softmax_node using
add_storage_type_suffix and graph.meta_ubo().

Pull Request resolved: #18054
ghstack-source-id: 353546688
@exported-using-ghexport

Differential Revision: [D95970171](https://our.internmc.facebook.com/intern/diff/D95970171/)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants