[FP16] Improved performance by fusing dequantize with compute in kernels: 20-30% Inference Speedup#78

Merged

mikepapadim merged 68 commits intomainfrom

feat/deq-n-compute

Dec 11, 2025

Commits on Dec 3, 2025

Implement FP16 support in TornadoVM by introducing HalfFloat arrays, optimized matrix-vector kernels, and SiLU-GLU activation
mikepapadim
committed
Introduce matrix-vector kernel with residual addition and enhance FP16 task graph setup
mikepapadim
committed
Fused Q/K/V matrix-vector multiplication into a single kernel to reduce overhead, improve cache utilization, and update task graph setup to integrate fused kernel.
mikepapadim
committed
Fuse RoPE rotation and KV cache copy into a single kernel, update task graph to integrate `ropeRotationWithCacheCopy` kernel, and remove redundant kernels (`rope` and `copyToCaches`).
mikepapadim
committed
Add `mapContextWithQuantize` kernel, integrate into task graph, and deprecate `mapContext` and `quantizeXb`
mikepapadim
committed

Commits on Dec 4, 2025

Commits on Dec 7, 2025

Commits on Dec 8, 2025

Commits on Dec 9, 2025

Commits on Dec 10, 2025

Commits on Dec 11, 2025