feat(vllm): add grammar and structured output support - #8806

Open

fury0928 wants to merge 8 commits into

mudler:masterfrom

fury0928:feat/vllm-structured-output

Commits on Jul 29, 2026

feat(backend): pass JSONSchema and ResponseFormat through gRPC

fury0928
authored and
localai-org-maint-bot
committed
feat(endpoints): extract raw JSON schema for structured output

fury0928
authored and
localai-org-maint-bot
committed
feat(vllm): add structured output support via guided decoding

fury0928
authored and
localai-org-maint-bot
committed
fix: refine vLLM structured output implementation

fury0928
authored and
localai-org-maint-bot
committed
fix(vllm): support both vLLM API versions and add grammar passthrough

fury0928
authored and
localai-org-maint-bot
committed
docs: update constrained grammars with vLLM structured output support

fury0928
authored and
localai-org-maint-bot
committed
refactor: use Metadata map instead of dedicated proto fields for structured output

fury0928
authored and
localai-org-maint-bot
committed
Merge branch 'master' into feat/vllm-structured-output
localai-org-maint-bot
authored