Priority Level
High (Major functionality broken)
Describe the bug
MAX_AST_DEPTH=10 too restrictive for conditional Jinja templates; error message swallowed by sanitize_user_exceptions
Description
Data Designer's Jinja sandbox (ginja) enforces MAX_AST_DEPTH = 10 on user-provided templates. This limit is too tight for common prompt-engineering patterns — a single inline conditional with 3 guarded variables exceeds it.
The problem is compounded by sanitize_user_exceptions (in environment.py), which catches the informative UserTemplateError("Jinja template too complex, simplify your template.") and re-raises it as the generic "User provided prompt generation template is invalid." — making the root cause impossible to diagnose without reading Data Designer source code.
Root cause
Jinja2 represents chained and conditions as nested binary And nodes: And(left, And(left, And(...))). Each and keyword adds one level of AST depth. A practical guard pattern like:
{{ '- Body: BMI ' + bmi + ', BP ' + bp + ', BS ' + bs
if bmi and bmi != 'N/A' and bp and bp != 'N/A' and bs and bs != 'N/A'
else '' }}
produces an AST with depth 11 (6 and clauses → 5 nesting levels, plus CondExpr → Concat/Add → Output → Template), which trips the MAX_AST_DEPTH = 10 check in _assert_template_ast_complexity.
The error is then swallowed at:
# environment.py lines 387-401
@sanitize_user_exceptions
def prepare_jinja2_template_renderer(...):
...
# where sanitize_user_exceptions does:
except (UserTemplateError, TemplateSyntaxError):
raise UserTemplateError(USER_PROMPT_TEMPLATE_ERROR_MESSAGE)
# USER_PROMPT_TEMPLATE_ERROR_MESSAGE = "User provided prompt generation template is invalid."
Steps/Code to reproduce bug
import data_designer.config as dd
import pandas as pd
from data_designer.interface import DataDesigner
df = pd.DataFrame({
'a': ['X'], 'b': ['Y'], 'c': ['Z'],
})
model_configs = [dd.ModelConfig(
alias='test', model='openai/gpt-oss-120b', provider='nvidia',
skip_health_check=True,
inference_parameters=dd.ChatCompletionInferenceParams(
max_tokens=100, temperature=1.0, top_p=1.0,
timeout=60, max_parallel_requests=1,
),
)]
designer = DataDesigner()
# WORKS — AST depth 7
cb1 = dd.DataDesignerConfigBuilder(model_configs=model_configs)
cb1.with_seed_dataset(dd.DataFrameSeedSource(df=df))
cb1.add_column(dd.ExpressionColumnConfig(
name='test',
expr="{{ a + b if a and a != 'SKIP' and b and b != 'SKIP' else '' }}",
))
designer.preview(cb1, num_records=1) # OK
# FAILS — AST depth 11
cb2 = dd.DataDesignerConfigBuilder(model_configs=model_configs)
cb2.with_seed_dataset(dd.DataFrameSeedSource(df=df))
cb2.add_column(dd.ExpressionColumnConfig(
name='test',
expr="{{ a + b + c if a and a != 'SKIP' and b and b != 'SKIP' and c and c != 'SKIP' else '' }}",
))
designer.preview(cb2, num_records=1) # PromptTemplateRenderError
Expected behavior
Not fail + provide better error messaging
Suggested Fixes
- Raise MAX_AST_DEPTH from 10 to ~20. The node-count limit (MAX_AST_NODE_COUNT = 600) already caps overall complexity; the depth limit just penalizes and-chains disproportionately.
- Preserve the original error message in sanitize_user_exceptions instead of replacing it with a generic string. At minimum, log it so users can diagnose the issue.
Agent Diagnostic / Prior Investigation
No response
Additional context
No response
Checklist
Priority Level
High (Major functionality broken)
Describe the bug
MAX_AST_DEPTH=10too restrictive for conditional Jinja templates; error message swallowed bysanitize_user_exceptionsDescription
Data Designer's Jinja sandbox (
ginja) enforcesMAX_AST_DEPTH = 10on user-provided templates. This limit is too tight for common prompt-engineering patterns — a single inline conditional with 3 guarded variables exceeds it.The problem is compounded by
sanitize_user_exceptions(inenvironment.py), which catches the informativeUserTemplateError("Jinja template too complex, simplify your template.")and re-raises it as the generic"User provided prompt generation template is invalid."— making the root cause impossible to diagnose without reading Data Designer source code.Root cause
Jinja2 represents chained
andconditions as nested binaryAndnodes:And(left, And(left, And(...))). Eachandkeyword adds one level of AST depth. A practical guard pattern like:{{ '- Body: BMI ' + bmi + ', BP ' + bp + ', BS ' + bs if bmi and bmi != 'N/A' and bp and bp != 'N/A' and bs and bs != 'N/A' else '' }}produces an AST with depth 11 (6 and clauses → 5 nesting levels, plus CondExpr → Concat/Add → Output → Template), which trips the MAX_AST_DEPTH = 10 check in _assert_template_ast_complexity.
The error is then swallowed at:
Steps/Code to reproduce bug
Expected behavior
Not fail + provide better error messaging
Suggested Fixes
Agent Diagnostic / Prior Investigation
No response
Additional context
No response
Checklist