Skip to content

Commit 55c3c5d

Browse files
abrichrclaude
andauthored
fix: unbounded regex quantifiers prevent Outlines DFA state explosion (#203)
* fix: use outlines v1.2 get_regex_logits_processor API The outlines v1.2 API requires: 1. Wrapping the HF model+tokenizer in outlines.Transformers 2. Calling get_regex_logits_processor(None, wrapped, regex) Prior code tried to construct OutlinesLogitsProcessor directly with a tokenizer= kwarg that doesn't exist in v1.2. The error was caught and silently fell back to unconstrained generation. Tests now verify the ACTUAL API surface (import paths + factory function signature) instead of just checking class names exist. This would have caught all three prior Outlines bugs: - PR #197: wrong class name (RegexLogitsProcessor) - PR #201: wrong constructor (tokenizer= kwarg) - This PR: wrong API pattern (direct constructor vs factory) 33/33 tests pass with outlines 1.2.12 installed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use unbounded regex quantifiers to prevent DFA state explosion Bounded quantifiers like {1,500} create counting DFA states that cross-product with every alternative in the regex. The Thought prefix alone created 1,500 states, exceeding Outlines' 2^31 limit. Changes: - [^\n]{1,500} → [^\n]+ (Thought prefix: 1 state vs 1,500) - [^"]{0,200} → [^"]* (TYPE text: 1 state vs 200) - \d{1,3} → \d+ (coordinates: 1 state vs 3) max_new_tokens=512 provides the actual length limit. The DFA doesn't need to count characters. New test: test_no_bounded_quantifiers_in_regex asserts no quantifier in the regex exceeds {N,10}, preventing future regressions. 34/34 tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 583ed56 commit 55c3c5d

2 files changed

Lines changed: 28 additions & 6 deletions

File tree

openadapt_evals/training/standalone/trainer.py

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -91,19 +91,24 @@ def __init__(
9191
# --- Constrained decoding -------------------------------------------
9292

9393
# Regex matching the required Thought/Action format from SYSTEM_PROMPT.
94-
# The model gets up to 500 chars of chain-of-thought reasoning, then
95-
# MUST output exactly one valid action. This preserves the model's
96-
# ability to reason while guaranteeing parseable output.
94+
# The model reasons freely, then MUST output exactly one valid action.
9795
#
9896
# Format: Thought: <reasoning>\nAction: <action>
97+
#
98+
# IMPORTANT: All repetitions use unbounded quantifiers (+, *) instead
99+
# of bounded ({1,N}). Bounded quantifiers create counting DFA states
100+
# that explode combinatorially — {1,500} alone creates 1,500 states
101+
# cross-producted with every action alternative. Unbounded repetitions
102+
# are single-state self-loops that Outlines handles efficiently.
103+
# max_new_tokens provides the actual length limit.
99104
_ACTION_RE = (
100-
r"CLICK\(x=0\.\d{1,3},\s*y=0\.\d{1,3}\)"
101-
r'|TYPE\(text="[^"]{0,200}"\)'
105+
r"CLICK\(x=0\.\d+,\s*y=0\.\d+\)"
106+
r'|TYPE\(text="[^"]*"\)'
102107
r"|WAIT\(\)"
103108
r"|DONE\(\)"
104109
)
105110
_ACTION_REGEX = (
106-
r"Thought: [^\n]{1,500}\nAction: (" + _ACTION_RE + r")"
111+
r"Thought: [^\n]+\nAction: (" + _ACTION_RE + r")"
107112
)
108113
# Sentinel: None = not yet attempted, list = success, False = failed
109114
_constrained_processor_cache: Any = None

tests/test_standalone_trainer.py

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,23 @@ def test_invalid_text_rejected(self, text: str) -> None:
8080
def test_action_only_regex_matches(self, action: str) -> None:
8181
assert re.match(self.action_regex, action), f"Expected match: {action!r}"
8282

83+
def test_no_bounded_quantifiers_in_regex(self) -> None:
84+
"""Regression test: bounded quantifiers ({N,M}) cause DFA state explosion.
85+
86+
Outlines compiles regex to a DFA. Bounded quantifiers like {1,500}
87+
create counting states that cross-product with every alternative,
88+
exceeding the 2^31 state limit. All repetitions must use +, *, or
89+
small bounds ({1,3} is OK, {0,200} is not).
90+
"""
91+
import re as _re
92+
# Find all bounded quantifiers in the full regex
93+
bounds = _re.findall(r'\{(\d+),(\d+)\}', self.full_regex)
94+
for lo, hi in bounds:
95+
assert int(hi) <= 10, (
96+
f"Bounded quantifier {{{lo},{hi}}} in ACTION_REGEX will cause "
97+
f"DFA state explosion in Outlines. Use + or * instead."
98+
)
99+
83100

84101
# ---------------------------------------------------------------------------
85102
# Constrained decoding cache tests

0 commit comments

Comments
 (0)