Adding Mamba SSM Arm SVE Kernel#664
Closed
hrushitfujitsu wants to merge 1 commit into
Closed
Conversation
Member
|
Thanks a lot for making a kernel and contributing to the kernels ecosystem 🤗! This kernel looks like awesome work. The scope of the
We encourage kernel developers to make kernels available through their own GitHub repositories and upload them to their own organization or user on the Hugging Face Hub. In that way, you also get all the credits for your work as a kernel developer. If this kernel should be integrated into the |
Member
|
@hrushitfujitsu let us know :) |
Member
|
Closing because of no activity. |
|
Hi Sayak,
Greetings !!
Apologies for late reply on this PR.
We have been busy in the other project,
We would like to re-open this PR and work with you to merge the same.
This is foundational work from our side and helps to accelerate the Mamba Model and other SSM models as well in transformers library on Arm Ecosystem
Currently this kernel can give up to 5x acceleration on inference.
Earlier we raised this PR on transformers lib:
Add SVE implementation for Mamba Sequential Scan Algorithm by vineelabhinav · Pull Request #38185 · huggingface/transformers<huggingface/transformers#38185>
(We are trying to merge this from an year now, have been contributing to Kernel ecosystem in HF on our side, we build the arm enablement as well on our side but the work was completed at HuggingFace end just before we were ready to push)
We also tried to support SVE in Kernels repo
Added SVE SIMD flag for CPU kernels by hrushitfujitsu · Pull Request #330 · huggingface/kernel-builder<huggingface/kernel-builder#330>
But it was notified and confirmed that all SIMD related changes should be pushed through kernels itself.
Hosting and maintaining through the main kernel community repo is always advantageous in terms of getting maximum usage as well as making it available to all Arm developers.
Please support and help to share your thoughts.
Looking forward to connecting and conclude this.
Regards,
Abhishek Jain
From: Sayak Paul ***@***.***>
Sent: 12 June 2026 18:52
To: huggingface/kernels-community ***@***.***>
Cc: Jain, Abhishek ***@***.***>; Mention ***@***.***>
Subject: Re: [huggingface/kernels-community] Adding Mamba SSM Arm SVE Kernel (PR #664)
[https://avatars.githubusercontent.com/u/22957388?s=20&v=4]sayakpaul left a comment (huggingface/kernels-community#664)<#664 (comment)>
Closing because of no activity.
—
Reply to this email directly, view it on GitHub<#664?email_source=notifications&email_token=BBGF5OKTWTV5KKC74EDPYJD47P7XXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRZGE3DIOBSGU22M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-4691648255>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BBGF5OIFS6ACBZGOOHFO3JD47P7XXAVCNFSNUABGKJSXA33TNF2G64TZHMYTANZRGQ3TKNJSHE5US43TOVSTWNBSHEZTAMZXGA3DFILWAI>.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS<https://github.com/notifications/mobile/ios/BBGF5OJWFBQM3P3WV2UV5ZT47P7XXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRZGE3DIOBSGU22M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJKTGN5XXIZLSL5UW64Y> and Android<https://github.com/notifications/mobile/android/BBGF5OKFDFXPO2W3AFNVSPL47P7XXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTINRZGE3DIOBSGU22M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>. Download it today!
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With reference to the following PR huggingface/transformers#38185, and as per our knowledge since the kernels raised to this repo are automatically built and uploaded to the hub, we would like to raise this kernel to kernels-community
This kernel was successfully built and tested on G3E, this correction does not have any effect on the accuracy
Implementation
The new kernel vectorizes the selective scan computation using ARM SVE intrinsics. The implementation is intended to:
Performance Check
We also integrated this kernel to transformers repo(https://github.com/huggingface/transformers) locally, this kernel is 3-4x faster than the current implementation
Task 32 input tokens, 1 Generated token
The above table represents the overall generation time (in seconds), this benchmarking was also done on G3E(64 cores)
Co-authored by @hrushitfujitsu and @abhijain1204fujitsu