feat(whisper): Vulkan GPU acceleration for AMD/Intel GPUs#1185
Merged
Conversation
Wire in the whisper-server Vulkan binaries shipped since binaries release 0.0.7 so AMD Radeon and Intel Arc/iGPU users on Windows/Linux get the same one-click "Enable GPU" flow NVIDIA users get with CUDA. - Extract src/helpers/gpuBinaryManager.js: the shared download/extract/ install pipeline behind whisperCudaManager and llamaVulkanManager (both are now thin config subclasses), with streaming SHA256 verification against in-app pinned digests (fail closed) or the GitHub API asset digest. - Add whisperVulkanManager: pinned tag 0.0.8 + pinned digests, statically linked binaries, no companion libs. - whisperServer: preferVulkan binary resolution, sticky useVulkan flag, 120s Vulkan startup timeout, and CPU fallback on ANY Vulkan startup rejection (early exit, late VRAM-OOM death, or hang). CUDA keeps its existing early-exit rule. An intentional stop() during a pending startup now rejects instead of resurrecting the server. - Thread useVulkan through pre-warm, per-dictation starts, and wake re-warm (which now replays lastStartOptions instead of hardcoding CUDA). - IPC: get/download/cancel/delete vulkan-whisper channels (download and delete stop the server first to avoid Windows file locks; CUDA delete now does the same), WHISPER_VULKAN_ENABLED persisted, gpu-fallback relay beside the existing cuda-fallback one. - UI: the CUDA card in TranscriptionModelPicker is now a GPU card that offers CUDA on NVIDIA machines and Vulkan otherwise; ControlPanel's unified GPU banner detects whisper-vulkan too. Reuses the existing gpu.* i18n keys. - Characterization tests for the shared pipeline: asset resolution, digest fail-closed, divergent cancel semantics, cleanup on failure. Deliberate behavior changes kept from the refactor: the CUDA disk pre-check tightens 2x -> 2.5x, its release fetch now sends GITHUB_TOKEN when set (rate limits), and download archives stage in the OS temp dir instead of userData/bin.
gabrielste1n
force-pushed
the
feat/whisper-vulkan-gpu
branch
from
July 15, 2026 17:37
f1d8c78 to
505516b
Compare
This was referenced Jul 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Local Whisper is GPU-accelerated on macOS (Metal) and NVIDIA (CUDA), but AMD Radeon and Intel Arc/iGPU users on Windows/Linux run CPU-only — the largest unaccelerated population. Since binaries release 0.0.7,
OpenWhispr/whisper.cppshipswhisper-server-{win32,linux}-x64-vulkan.zip(statically linked; the only runtime dependency is the Vulkan loader every modern GPU driver provides), but nothing consumed them. This wires them in, mirroring the existing CUDA (whisper) and Vulkan (llama) flows: a one-click "Enable GPU" card, 2–5× faster transcription on larger models, automatic CPU fallback if Vulkan misbehaves.Implements
whisper-vulkan-gpu-acceleration.md.What changed
Shared refactor (net-negative LOC)
src/helpers/gpuBinaryManager.jsextracts the download→verify→extract→install pipeline thatwhisperCudaManagerandllamaVulkanManagerhad duplicated; both are now thin config subclasses. A third copy for whisper-Vulkan would have hit the 3-occurrence DRY threshold.download()throws"Download cancelled by user"on abort, llama returns{ success: false, cancelled: true }; delete result shapes unchanged. Covered by new characterization tests (test/helpers/gpuBinaryManager.test.js).GITHUB_TOKENwhen set (rate limits), and archives stage in the OS temp dir instead ofuserData/bin.Whisper Vulkan backend
whisperVulkanManager: pinned tagWHISPER_CPP_VERSION || "0.0.8"with pinned per-asset digests, exact asset names, no companion libs (statically linked — also avoids clobbering DLLs in the shareduserData/bin).whisperServer: resolveswhisper-server-{platform}-{arch}-vulkanwhenpreferVulkan, stickyuseVulkanbesideuseCuda(CUDA wins on conflict), and a dedicatedVULKAN_STARTUP_TIMEOUT_MS = 120000— llama-Vulkan cold starts run 30–40s on Windows and whisper-large loads ~3 GiB before the port binds (see fix(llama): raise Vulkan startup timeout, stop server before re-download #698 for the timeout precedent).stop()during a pending startup now rejects the startup instead of resurrecting the server behind the caller (previously possible on the CUDA path too).useVulkanthreaded through startup pre-warm, per-dictation starts, and the Feature: Option to leave local transcription model loaded in VRAM while idle #766 wake re-warm, which now replayslastStartOptionsinstead of hardcodinguseCuda: true.IPC / UI
get-vulkan-whisper-status(bundles Vulkan availability +hasNvidiaGpuso the renderer gates precedence in one call),download-/cancel-/delete-vulkan-whisper-binary, plusvulkan-whisper-download-progress. Download/delete stop the server first (Windows file-lock lesson); CUDA delete now does the same.WHISPER_VULKAN_ENABLEDjoinsPERSISTED_KEYS;gpu-fallbackrelayed beside the existingcuda-fallbackbroadcast.TranscriptionModelPickeris now a GPU card: NVIDIA → CUDA flow (unchanged), otherwise Vulkan-capable GPU → same card on the vulkan API. The Vulkan offer never shows on NVIDIA machines (CUDA is faster) or macOS.ControlPanel's unified GPU banner detects whisper-Vulkan too.gpu.*keys are backend-agnostic; the one hardcoded"Cancel"now usest("gpu.cancel").Verification
npm test(432 tests),npm run lint,npm run typecheck,npm run i18n:checkall pass.whisper-server-{win32-x64,linux-x64}-vulkan(.exe)and that the pinned digests match the release assets.sidecarReaper'swhisper-serverfragment already matches the vulkan binary name — no sidecar changes needed.Before merge (from the plan)
whisper-server-win32-x64-vulkan.exefrom release 0.0.8 on at least one discrete AMD/Intel GPU and one AMD/Intel iGPU laptop; confirm stderr containswhisper_backend_init_gpu: using Vulkan0 backend(MSVC static builds can silently register no Vulkan backend and still transcribe correctly on CPU), record cold/warm startup times against the 120s timeout, and verify output.OpenWhispr/whisper.cpp(0.0.8 is currently mutable) so the pinned tag + digests can't be swapped.