Skip to content

Commit 04f1a02

Browse files
authored
fix(ik-llama-cpp): adapt to common_grammar struct in sampling.h (#9512)
Upstream ik_llama.cpp commit e0596bf6 ("Autoparser") changed common_params_sampling::grammar from std::string to a common_grammar struct (type + grammar), which broke our two direct accesses: - JSON ingest fed the field through json_value<common_grammar>(...), for which nlohmann has no from_json adapter. - JSON export emitted the struct directly, for which nlohmann has no to_json adapter. Wrap the incoming JSON string in common_grammar{COMMON_GRAMMAR_TYPE_USER, ...} and serialize via the inner .grammar member, mirroring upstream's examples/server/server-context.cpp. Also bump IK_LLAMA_VERSION to 286ce324baed17c95faec77792eaa6bdb1c7a5f5 so the local-ai side lines up with the dependency bump in #9496. Assisted-by: Claude-Code:claude-opus-4-7
1 parent 181ebb6 commit 04f1a02

2 files changed

Lines changed: 12 additions & 3 deletions

File tree

backend/cpp/ik-llama-cpp/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11

2-
IK_LLAMA_VERSION?=d4824131580b94ffa7b0e91c955e2b237c2fe16e
2+
IK_LLAMA_VERSION?=286ce324baed17c95faec77792eaa6bdb1c7a5f5
33
LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
44

55
CMAKE_ARGS?=

backend/cpp/ik-llama-cpp/grpc-server.cpp

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -686,7 +686,16 @@ struct llama_server_context
686686
slot->sparams.mirostat_eta = json_value(data, "mirostat_eta", default_sparams.mirostat_eta);
687687
slot->params.n_keep = json_value(data, "n_keep", slot->params.n_keep);
688688
slot->sparams.seed = json_value(data, "seed", default_sparams.seed);
689-
slot->sparams.grammar = json_value(data, "grammar", default_sparams.grammar);
689+
{
690+
// upstream changed common_params_sampling::grammar from std::string to
691+
// the common_grammar struct (type + grammar). The incoming JSON still
692+
// carries a plain string, so build the user-provided grammar here and
693+
// fall back to the server default when the request omits it.
694+
std::string grammar_str = json_value(data, "grammar", std::string());
695+
slot->sparams.grammar = grammar_str.empty()
696+
? default_sparams.grammar
697+
: common_grammar{COMMON_GRAMMAR_TYPE_USER, std::move(grammar_str)};
698+
}
690699
slot->sparams.n_probs = json_value(data, "n_probs", default_sparams.n_probs);
691700
slot->sparams.min_keep = json_value(data, "min_keep", default_sparams.min_keep);
692701
slot->sparams.grammar_triggers = grammar_triggers;
@@ -1232,7 +1241,7 @@ struct llama_server_context
12321241
// {"logit_bias", slot.sparams.logit_bias},
12331242
{"n_probs", slot.sparams.n_probs},
12341243
{"min_keep", slot.sparams.min_keep},
1235-
{"grammar", slot.sparams.grammar},
1244+
{"grammar", slot.sparams.grammar.grammar},
12361245
{"samplers", samplers}
12371246
};
12381247
}

0 commit comments

Comments
 (0)