Skip to content

Commit 69342c3

Browse files
mudlerlocalai-org-maint-bot
authored andcommitted
fix(buun-llama-cpp): backport logit_bias_eog field to grpc-server copy
LocalAI's shared grpc-server.cpp reaches ctx_server.get_meta().logit_bias_eog twice (the twin params_from_json_cmpl callsites). That accessor was added to server_context_meta upstream after buun's 2026-04-05 fork-point, so compiling against buun errors with 'struct server_context_meta' has no member named 'logit_bias_eog'. Rewrite the call sites — only in the buun grpc-server.cpp copy — to source the vector from params_base.sampling.logit_bias_eog instead. That vector is the underlying data the upstream meta accessor eventually returns (buun still carries common_params_sampling::logit_bias_eog at common.h:280), so the substitution yields identical behavior on both trees. The sed is guarded by a grep for the call site, so this patch is self-disabling once buun rebases past the upstream refactor. Assisted-by: Claude:Opus-4.7 [Read] [Edit] [Bash] [WebFetch] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 parent 9301504 commit 69342c3

1 file changed

Lines changed: 15 additions & 0 deletions

File tree

backend/cpp/buun-llama-cpp/patch-grpc-server.sh

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,21 @@ else
125125
echo "==> DFlash option-handler patch OK"
126126
fi
127127

128+
if grep -q 'ctx_server\.get_meta()\.logit_bias_eog' "$SRC"; then
129+
echo "==> patching $SRC to source logit_bias_eog from params_base.sampling (buun predates server_context_meta::logit_bias_eog accessor)"
130+
# Upstream llama.cpp exposes logit_bias_eog through server_context_meta
131+
# after buun's 2026-04-05 fork-point. Buun still carries the underlying
132+
# data on common_params_sampling::logit_bias_eog (the struct field the
133+
# meta accessor eventually returns). Rewriting the call site to read
134+
# params_base.sampling.logit_bias_eog works against both trees — upstream
135+
# still populates that same vector the newer accessor returns.
136+
sed 's/ctx_server\.get_meta()\.logit_bias_eog/params_base.sampling.logit_bias_eog/g' "$SRC" > "$SRC.tmp"
137+
mv "$SRC.tmp" "$SRC"
138+
echo "==> logit_bias_eog substitution OK"
139+
else
140+
echo "==> $SRC has no ctx_server.get_meta().logit_bias_eog call, skipping logit_bias_eog patch"
141+
fi
142+
128143
if grep -q 'get_media_marker()' "$SRC"; then
129144
echo "==> patching $SRC to replace get_media_marker() with legacy \"<__media__>\" literal"
130145
# Only one call site today (ModelMetadata), but replace all occurrences to

0 commit comments

Comments
 (0)