Skip to content

Commit c6a5128

Browse files
authored
fix: Automatically disable mmap for Intel SYCL backends (#9012) (#9015)
* fix: Automatically disable mmap for Intel SYCL backends Fixes issue #9012 where Qwen3.5 models fail to load on Intel Arc GPU with RPC EOF error. The Intel SYCL backend has a known issue where mmap enabled causes the backend to hang. This change automatically disables mmap when detecting Intel or SYCL backends. References: - #9012 - Documentation mentions: SYCL hangs when mmap: true is set * feat: Add logging for mmap auto-disable on Intel SYCL backends As requested in PR review, add xlog.Info call to log when mmap is automatically disabled for Intel SYCL backends. This helps with debugging and confirms the auto-disable logic is working. --------- Co-authored-by: localai-bot <localai-bot@users.noreply.github.com>
1 parent 8752510 commit c6a5128

1 file changed

Lines changed: 11 additions & 0 deletions

File tree

core/backend/options.go

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package backend
22

33
import (
4+
"strings"
45
"math/rand"
56
"os"
67
"path/filepath"
@@ -109,6 +110,16 @@ func grpcModelOpts(c config.ModelConfig, modelPath string) *pb.ModelOptions {
109110
mmap = *c.MMap
110111
}
111112

113+
// Intel SYCL backend has issues with mmap enabled
114+
// See: https://github.com/mudler/LocalAI/issues/9012
115+
// Automatically disable mmap for Intel SYCL backends
116+
if c.Backend != "" {
117+
if strings.Contains(strings.ToLower(c.Backend), "intel") || strings.Contains(strings.ToLower(c.Backend), "sycl") {
118+
mmap = false
119+
xlog.Info("Auto-disabling mmap for Intel SYCL backend", "backend", c.Backend)
120+
}
121+
}
122+
112123
ctxSize := 4096
113124
if c.ContextSize != nil {
114125
ctxSize = *c.ContextSize

0 commit comments

Comments
 (0)