Skip to content

Commit 29f93ea

Browse files
kirklandsignfacebook-github-bot
authored andcommitted
Add grammar fields to GenerationConfig for constrained decoding
Summary: Add grammar (string) and grammar_type (string) fields to GenerationConfig. Empty-string defaults, no behavior change when empty. Prepares for grammar-constrained decoding integration (e.g. JSON schema, regex). Differential Revision: D105277261
1 parent 08a584b commit 29f93ea

1 file changed

Lines changed: 8 additions & 0 deletions

File tree

extension/llm/runner/irunner.h

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,14 @@ struct GenerationConfig {
3333
// Whether to echo the input prompt in the output
3434
bool echo = true;
3535

36+
// Grammar definition for constrained decoding (e.g. a JSON schema, regex,
37+
// Lark CFG, or GBNF grammar). Empty string means no constraint.
38+
std::string grammar;
39+
40+
// Grammar format: "json_schema", "regex", "lark", or "gbnf".
41+
// Only used when grammar is non-empty.
42+
std::string grammar_type;
43+
3644
// Whether to ignore EOS token and continue generating until max_new_tokens
3745
bool ignore_eos = false;
3846

0 commit comments

Comments
 (0)