Fix Gemma4 tool parser: support hyphenated function names and braces in string args#1150
Merged
angeloskath merged 1 commit intoml-explore:mainfrom Apr 21, 2026
Merged
Conversation
- Allow hyphens in function names ([\w-]+ instead of \w+) so tools like
manim-video are parsed correctly
- Make brace-balancing regex string-aware so {/} inside <|"|>...<|"|>
string arguments no longer break the match
- Catch parse errors in ToolFormatter to prevent server crashes on
malformed or truncated tool calls
- Add tests for both parser bug fixes
Contributor
Author
|
@angeloskath would you mind taking a look when you get a chance? |
Contributor
Author
|
hey @kernelpool can you take a look at this if possible thanks :) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two bugs in
mlx_lm/tool_parsers/gemma4.pycausedValueError: No function providedwhen parsing certain valid Gemma 4 tool calls:
Hyphenated function names — the regex used
\w+which doesn't match-,so tools named e.g.
manim-videowould always fail to parse.Braces inside string arguments — the brace-balancing regex treated
{/}inside
<|"|>...<|"|>string literals as structural braces, breaking the matchwhen arguments contained code snippets, markdown, or any text with
{}.Both caused an unhandled exception that crashed the server request.
Fix
[\w-]+instead of\w+for function name matching<|"|>...<|"|>literals are matched atomicallyso braces inside them don't affect brace-balance counting
ToolFormatter.__call__inserver.pynow catches parse errors gracefullyinstead of crashing the request
Tests
Added two test cases to
tests/test_tool_parsing.pycovering both bugs.