QVAC-21901 perf(mtmd/gemma4v): Mali Vulkan encoder profiling and measured sweep#182
Closed
Alok-Ranjan23 wants to merge 3 commits into
Closed
QVAC-21901 perf(mtmd/gemma4v): Mali Vulkan encoder profiling and measured sweep#182Alok-Ranjan23 wants to merge 3 commits into
Alok-Ranjan23 wants to merge 3 commits into
Conversation
…to std-norm
Mali Vulkan encoder optimization for the Gemma-4V projector tail. Instead of
scaling the full pooled activation tensor [n_embd, n_tokens] by sqrt(n_embd),
fold that magnitude into the per-channel std-norm params (tiny [n_embd] vectors):
(k*p - b)*s == (p - b/k)*(s*k)
Trades one full-tensor elementwise op for two vector-sized ones. Algebraically
exact (quality unchanged).
Gated by QVAC_G4V_STDFOLD (default on) as a QVAC-21361-style sweep knob so the
on-device Pixel 9 / Mali encode A/B measures fold-on vs fold-off; the winning
config is baked in and the knob removed before the PR ships.
Candidate #1 of the QVAC-21901 Mali fusion sweep. Remaining candidates
(per-layer 2D-RoPE collapse, conv2d patch-embed, coop-matrix/KleidiAI) are
added + measured after the on-device profile identifies the post-#174 hotspot.
Co-authored-by: Cursor <cursoragent@cursor.com>
…embd) into std-norm" This reverts commit ec874ef.
Alok-Ranjan23
changed the title
…snapshots The existing Vulkan perf logger only prints debug output, which is not retained by the Android Device Farm benchmark path. Capture its aggregated per-op timing map as compact JSON after each completed graph and expose it through a dynamic Vulkan registry extension. mtmd retrieves the latest vision-graph snapshot immediately after image encode, without adding generic-backend dependencies. The API is a no-op for non-Vulkan/unprofiled backends. The addon benchmark will forward the JSON through runtime stats in a follow-up bench-only change, letting Pixel 9 runs rank CONV_2D, MUL_MAT, RMS_NORM, and attention operations before selecting the next production optimization. Co-authored-by: Cursor <cursoragent@cursor.com>
Author
|
Closing without merge. QVAC-21901 measured the Pixel 9 Mali-G715 Gemma-4V path:
Conclusion: retain #172/#174, keep the projector CPU-default on Mali (GPU remains ~1.33x CPU), and do not ship profiling/sweep code. Evidence is recorded on QVAC-21901. |
gianni-cor
deleted the
feat/QVAC-21901-close-mali-vulkan-encoder-gpu-cpu-gap
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status — Draft / DO NOT MERGE
QVAC-21901 tracks the measured effort to close the Gemma-4V Mali Vulkan projector GPU↔CPU gap. Base:
temp-9341.Candidate #1 — rejected and removed
The original
sqrt(n_embd)std-norm affine fold was benchmarked on Pixel 9 / Mali with the cognitive preset:Device Farm units vary, so this is not an exact regression claim; however it showed no win and an unfavorable direction. The candidate and its
QVAC_G4V_STDFOLDknob have been reverted. The branch intentionally has no net source diff while profiling determines the next candidate.Next: measured op profile
A Pixel 9 GPU-only Vulkan profiling run completed. The first environment-only profiler attempt did not emit per-op timings to Device Farm logcat, so the next bench-only step enables the port's
force-profilerfeature and verifiesVulkan Profiling Resultsappear before selecting a kernel candidate.Potential targets (only after profile): patch-embedding
CONV_2D, repeated Gemma-4V 2D-RoPE / V RMS-norm dispatches, or explicit-attention / matmul shapes.Validation method
QVAC-21361-style Device Farm measurement: Gemma-4 E2B Q4 + Q8 projector, Pixel 9 Mali-G715, CPU-vs-GPU projector cells, cognitive preset, quality identity gate. Keep only measured winners and remove all sweep/profile knobs before review.
Dependencies / related work
mmproj-use-gpunow enables the Mali GPU projector cell.