[SYCL][DOCKER] Increased Gemma 4 26b performance with updated docker dependencies

[miloszkolber]

Name and Version

Current server-intel
"build_commit": "354ebac8c",
"build_number": 9468

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

Docker image

Command line

docker run --rm --entrypoint /app/llama-bench \
  --device /dev/dri/card1:/dev/dri/card0 \
  --device /dev/dri/renderD129:/dev/dri/renderD128 \
  --group-add 991 \
  -e ONEAPI_DEVICE_SELECTOR=level_zero:0 \
  -e GGML_SYCL_ENABLE_LEVEL_ZERO=1 \
  -e ZES_ENABLE_SYSMAN=1 \
  -e UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1 \
  -e SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 \
  -e GGML_SYCL_ENABLE_FLASH_ATTN=1 \
  -e GGML_SYCL_DISABLE_GRAPH=1 \
  -v /data/models:/models:ro \
  <image> \
  -m /models/<model>.gguf \
  -dev SYCL0 -ngl 999 -sm none -t 6 -b 2048 \
  -ctk <kv-cache-k> -ctv <kv-cache-v> -fa on \
  -p 512 -n 128 -r 3 -o json

Problem description & steps to reproduce

Summary

I made a little test and ran llama-bench inside the upstream full-intel image (build #9468, commit 354ebac8c) and same build, but rebased with Ubuntu 26.04 (probably doesn't matter this much with Docker) and latest stuff from kobuk-ppa (this is more important). It might be a complete outlier, but Gemma 4 26B shows around 40% improvement in token generation, so it might be worth checking out. It seems to affect mostly MoE models (e.g. LFM-2.5 with 6.4% improvement), dense models are within margin of error.

Environment

• Host: Ubuntu 26.04 LTS, kernel 7.0.0-22-generic
• GPU: Intel Arc Pro B50 (PCI ID 0xe212, Battlemage, latest firmware
• Host packages (for reference):
  • libze-intel-gpu1 26.18.38308.1-1~26.04~ppa1
  • intel-igc-core-2 2.34.4, intel-igc-opencl-2 2.34.4
  • libigdgmm12 22.10.0-1~26.04~ppa1
  • libze1 1.28.2-2
• llama-bench: build 9468 commit 354ebac8c (identical in both images)

Image comparison

Component	Current image	Updated image
Base OS	Ubuntu 24.04	Ubuntu 26.04
intel-igc-core-2 / intel-igc-opencl-2	2.20.5	2.34.4
libigdgmm12	22.8.2	22.10.0
libze-intel-gpu1 (compute-runtime)	25.40.35563.10-0	26.18.38308.1
libze-intel-gpu-raytracing	1.2.2	1.15.38308+1 (Kobuk)
libze1 (upstream Level Zero loader)	1.27.0	1.28.2
llama-bench binary	#9468 354ebac8c	#9468 354ebac8c

The difference is basically the entire stack: newer IGC, newer compute-runtime, newer Level Zero, newer libze-intel-gpu-raytracing.

Reproduction

docker run --rm --entrypoint /app/llama-bench \
  --device /dev/dri/card1:/dev/dri/card0 \
  --device /dev/dri/renderD129:/dev/dri/renderD128 \
  --group-add 991 \
  -e ONEAPI_DEVICE_SELECTOR=level_zero:0 \
  -e GGML_SYCL_ENABLE_LEVEL_ZERO=1 \
  -e ZES_ENABLE_SYSMAN=1 \
  -e UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1 \
  -e SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1 \
  -e GGML_SYCL_ENABLE_FLASH_ATTN=1 \
  -e GGML_SYCL_DISABLE_GRAPH=1 \
  -v /data/models:/models:ro \
  <image> \
  -m /models/<model>.gguf \
  -dev SYCL0 -ngl 999 -sm none -t 6 -b 2048 \
  -ctk <kv-cache-k> -ctv <kv-cache-v> -fa on \
  -p 512 -n 128 -r 3 -o json

Results

Parameters held constant across all runs: -dev SYCL0 -ngl 999 -sm none -t 6 -b 2048 -fa on -p 512 -n 128 -r 3 -o json.

Model	Quant	KV cache	Test	Official tok/s	Local tok/s	Δ
gemma-4-26b (A4B MoE)	Q3_K - Medium	q4_0 / q4_0	prompt	386.22	388.52	+0.6%
gemma-4-26b (A4B MoE)	Q3_K - Medium	q4_0 / q4_0	generation	17.23	24.64	+43.0%
lfm-2.5-8b (A1B MoE)	Q4_K - Medium	q8_0 / q8_0	prompt	1279.79	1384.65	+8.2%
lfm-2.5-8b (A1B MoE)	Q4_K - Medium	q8_0 / q8_0	generation	103.37	110.00	+6.4%
qwen-3.5-9b (dense)	Q4_K - Medium	q8_0 / q8_0	prompt	398.71	395.74	-0.7%
qwen-3.5-9b (dense)	Q4_K - Medium	q8_0 / q8_0	generation	29.35	29.70	+1.2%
ministral-3-14b (dense)	Q4_K - Medium	q8_0 / q8_0	prompt	241.69	240.67	-0.4%
ministral-3-14b (dense)	Q4_K - Medium	q8_0 / q8_0	generation	21.05	21.05	+0.0%

Observations

1. Gemma 4 26B generation change surprised me the most (17.23 -> 24.64 tok/s).
2. Dense models are unaffected, qwen-3.5-9b and ministral-3-14b are within run-to-run noise.
3. The other MoE models are moderately affected, but my testing capabilities of MoE models are limited due to 16GB of VRAM (maybe someone with B60 or B70 can test this?) — lfm-2.5-8b shows +6-8%, glm-4.7-flash shows +2-3%
4. Seems like in my local image GPU is reported as Intel(R) Arc(TM) Pro B50 Graphics, rather than the PCI ID 0xe212`.

What can be done?

I didn't publish this as PR outright, because at first – I'm not really a developer, second – current docker image is quite stable as is. So I kinda see two ways now:

• Update compute-runtime/level zero dependencies in main SYCL build.
• Create a separate build (e.g. "-edge") that would use updated dependencies.

Step-by-step

1. I was using server-intel image for quite some time now. Observed around 17 tok/s generation with Gemma 4 26B.
2. Had a random thought and "upgraded" the docker image and verified the same llama-server could then reach 24-25 tok/s with Gemma 4 26B.
3. Built a dockerfile.intel-bench image (Ubuntu 26.04 + kobuk + llama-bench).
4. Ran llama-bench 10 times (5 models × 2 images) with identical settings (also sampled drawn power, insignificant). Result were stable across runs.

First Bad Commit

No response

Relevant log output

Raw JSON exports
local-gemma-4-26b.json
local-glm-4.7-flash.json
local-lfm-2.5-8b.json
local-ministral-3-14b.json
local-qwen-3.5-9b.json
official-gemma-4-26b.json
official-glm-4.7-flash.json
official-lfm-2.5-8b.json
official-ministral-3-14b.json
official-qwen-3.5-9b.json

Dockerfile
intel-bench.txt
intel-runtime.txt

[miloszkolber]

Ran additional test with mradermacher/Qwen3.6-28B-REAP-GGUF with Q3_K_M quant.

Image	Test	tok/s	stddev	Avg Power (W)	Time (s)
official	prompt	383.48	2.00	25	95
local	prompt	406.61	1.03	19	88
official	generation	30.52	0.03	25	95
local	generation	31.58	0.06	19	88

• prompt Δ: +6.03%
• generation Δ: +3.47%
• Power: −6W (−24%)

Improvements are modest, but second largest after Gemma 4 26B. Interesting thing is that is uses 6W less than upstream image to achieve this.

Edit: Ubuntu version actually doesn't matter at all, so I'm just updating packages on top of existing image and getting the same results.

dockerfile.txt

[arthw]

@miloszkolber
Yes, this issue is fixed by PR: #24070.

Could you check with latest version?

Thank you!

[miloszkolber]

Sorry for the late response – that PR fixes the issue. Thanks!