Skip to content

Fix pydantic v2 dataclass lookaround regex engine#3370

Merged
koxudaxi merged 5 commits into
koxudaxi:mainfrom
trail-of-forks:fix-pydantic-v2-dataclass-lookaround-regex-engine
Jun 16, 2026
Merged

Fix pydantic v2 dataclass lookaround regex engine#3370
koxudaxi merged 5 commits into
koxudaxi:mainfrom
trail-of-forks:fix-pydantic-v2-dataclass-lookaround-regex-engine

Conversation

@DarkaMaul

@DarkaMaul DarkaMaul commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

A string field whose schema pattern uses a lookaround assertion ((?=, (?!, (?<=, (?<!), generated with --output-model-type pydantic_v2.dataclass, produced constr(pattern=...) without ConfigDict(regex_engine="python-re").

Pydantic v2's default rust regex engine cannot compile lookaround, so building the generated dataclass raises SchemaError at class definition time. The pydantic_v2.BaseModel output already handles this case.

Summary by CodeRabbit

  • New Features
    • Enhanced Pydantic v2 dataclass generation to detect lookaround regex patterns, including when patterns are reachable through referenced aliases, and automatically select the python-re regex engine when needed.
  • Bug Fixes
    • Ensured regex engine configuration is applied early enough so it reliably appears in the generated dataclass output.
  • Tests
    • Added/updated OpenAPI and JSON Schema coverage and expected fixtures for lookaround-constrained dataclasses, including reference scenarios, and adjusted relevant conformance exclusions.

DarkaMaul and others added 3 commits June 12, 2026 11:25
…patterns

Pydantic v2's default rust regex engine cannot compile lookaround
assertions, so BaseModel generation injects
ConfigDict(regex_engine="python-re") when a field pattern contains one.
The pydantic_v2.dataclass path lacked this handling, so generated
dataclasses raised SchemaError at class-build time.

Hoist the lookaround detection from BaseModel into a shared
has_lookaround_pattern helper and apply it on the DataClass config
path as well.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lass-lookaround-regex-engine

# Conflicts:
#	src/datamodel_code_generator/model/pydantic_v2/base_model.py
#	src/datamodel_code_generator/model/pydantic_v2/dataclass.py
Verifies the generated dataclass actually builds at runtime, since the
bug manifests as a SchemaError at class-build time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6e6cabad-1649-449d-a82b-f9d11449f1be

📥 Commits

Reviewing files that changed from the base of the PR and between b6a6fdd and f1ff0bc.

📒 Files selected for processing (3)
  • tests/data/expected/main/jsonschema/lookaround_anyof_nullable_pydantic_v2_dataclass.py
  • tests/main/jsonschema/test_main_jsonschema.py
  • tests/model/pydantic_v2/test_dataclass.py
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/main/jsonschema/test_main_jsonschema.py
  • tests/model/pydantic_v2/test_dataclass.py
  • tests/data/expected/main/jsonschema/lookaround_anyof_nullable_pydantic_v2_dataclass.py

📝 Walkthrough

Walkthrough

The PR refactors lookaround pattern detection into a shared module-level helper function, removes the duplicated class method from BaseModel, and applies the helper to DataClass configuration. Unit tests validate edge cases with referenced aliases, while integration test fixtures and parametrized test cases demonstrate end-to-end behavior for both OpenAPI and JSONSchema schemas with default and field-constraints modes.

Changes

Regex Lookaround Pattern Detection and Configuration

Layer / File(s) Summary
Lookaround pattern detection helper and BaseModel integration
src/datamodel_code_generator/model/pydantic_v2/base_model.py
New module-level has_lookaround_pattern() function scans field patterns for lookaround assertions in both Constraints.pattern and nested data_type.kwargs["pattern"], with optional recursive traversal of referenced fields using cycle detection. BaseModel.__init__ calls this helper to set regex_engine to "python-re", replacing the removed class method.
DataClass regex engine configuration
src/datamodel_code_generator/model/pydantic_v2/dataclass.py
DataClass imports and uses has_lookaround_pattern() to detect lookaround patterns in generated fields and conditionally set regex_engine to "python-re" in the config dict. A memoized method ensures the configuration is applied at the correct lifecycle point during imports and rendering.
Unit tests for lookaround pattern detection
tests/model/pydantic_v2/test_dataclass.py
Unit test imports has_lookaround_pattern() and validates that lookaround pattern detection correctly skips references to non-model sources without fields when recursion is enabled.
OpenAPI integration fixtures and test
tests/data/expected/main/openapi/pattern_with_lookaround_pydantic_v2_dataclass*.py, tests/main/openapi/test_main_openapi.py
Two expected output fixtures demonstrate Pydantic v2 dataclass generation with negative lookbehind patterns and regex_engine="python-re" configuration, supporting both default and field-constraints modes. Parametrized test validates both output variants with execution validation enabled.
JSONSchema integration fixtures and test
tests/data/expected/main/jsonschema/lookaround_anyof_nullable_pydantic_v2_dataclass.py, tests/main/jsonschema/test_main_jsonschema.py
Expected output fixture and parametrized test demonstrate lookaround pattern handling for nullable anyOf schemas, with a type alias for the constrained string and a dataclass configured with regex_engine="python-re". Execution validation is enabled to verify correctness.
Conformance test exclusion update
tests/main/payload_validation/conformance.py
The exclusion reason for a lookaround mixed constraints test case is updated to clarify how root-level oneOf produces a TypeAliasType without consuming dataclass and how TypeAdapter still rejects lookaround patterns despite regex_engine='python-re' configuration.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • koxudaxi/datamodel-code-generator#2721: Both PRs touch src/datamodel_code_generator/model/pydantic_v2/base_model.py logic that detects lookaround patterns to set regex_engine="python-re" and rely on that flag to drive RootModel/DataClass rendering/config—so the changes are directly related at the regex-engine/lookaround handling code level.
  • koxudaxi/datamodel-code-generator#3350: Both PRs modify Pydantic v2 config generation in src/datamodel_code_generator/model/pydantic_v2/base_model.py (and dataclass.py)—Deduplicate Pydantic v2 config helpers #3350 refactors shared ConfigDict/config-parameter assembly while the main PR adds lookaround detection to conditionally set regex_engine="python-re" during that same config construction path.

Suggested labels

breaking-change-analyzed

Poem

🐰 A pattern with a lookbehind's gleam,
Now Python-re fulfills the regex dream!
Lookarounds detected, configs aligned,
Shared helpers make the logic refined. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: fixing Pydantic v2 dataclass code generation for lookaround regex patterns.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/main/openapi/test_main_openapi.py`:
- Around line 1897-1919: Add the same Black-compatibility skip decorator used on
the neighboring Pydantic v2 BaseModel test to the dataclass test: annotate
test_main_openapi_pattern_with_lookaround_pydantic_v2_dataclass with
`@pytest.mark.skipif`(black.__version__.split(".")[0] < "22", reason="Black <22
incompatible with pydantic_v2.dataclass formatting/parsing"), ensuring the black
symbol is available in the test module; this will skip the dataclass OpenAPI
test when Black major version is less than 22.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 439addbd-8850-4c27-a24e-f8d5ec025a42

📥 Commits

Reviewing files that changed from the base of the PR and between 6192212 and e9eab73.

📒 Files selected for processing (5)
  • src/datamodel_code_generator/model/pydantic_v2/base_model.py
  • src/datamodel_code_generator/model/pydantic_v2/dataclass.py
  • tests/data/expected/main/openapi/pattern_with_lookaround_pydantic_v2_dataclass.py
  • tests/data/expected/main/openapi/pattern_with_lookaround_pydantic_v2_dataclass_field_constraints.py
  • tests/main/openapi/test_main_openapi.py

Comment thread tests/main/openapi/test_main_openapi.py
@codspeed-hq

codspeed-hq Bot commented Jun 12, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 11 untouched benchmarks
⏩ 98 skipped benchmarks1


Comparing trail-of-forks:fix-pydantic-v2-dataclass-lookaround-regex-engine (f1ff0bc) with main (53a25ab)

Open in CodSpeed

Footnotes

  1. 98 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Comment thread src/datamodel_code_generator/model/pydantic_v2/dataclass.py Outdated

@koxudaxi koxudaxi left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for creating this PR. I left one review comment.

@codecov

codecov Bot commented Jun 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (6192212) to head (f1ff0bc).
⚠️ Report is 52 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##              main     #3370     +/-   ##
===========================================
  Coverage   100.00%   100.00%             
===========================================
  Files          129       140     +11     
  Lines        28549     30240   +1691     
  Branches      3448      3569    +121     
===========================================
+ Hits         28549     30240   +1691     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

A string field whose lookaround pattern lives in a generated type alias or root
type (e.g. `value: LookaroundAlias | None`) produced a pydantic v2 dataclass
without `ConfigDict(regex_engine="python-re")`, so it failed at import with a
SchemaError under pydantic's default rust regex engine.

- `has_lookaround_pattern()` gains an opt-in `follow_references` that walks into
  referenced models. References are only linked after `__init__`, so the
  dataclass applies it lazily (memoized) via `imports`/`render`.
- Drop the now-passing dataclass lookaround payload-validation exclusions
  (lookaround_anyof_nullable, lookaround_dict, lookaround_union_types,
  nested_lookaround_array); keep lookaround_mixed_constraints, whose root-level
  oneOf has no consuming dataclass to carry the config.
- Add unit tests covering the alias-detection path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@DarkaMaul DarkaMaul force-pushed the fix-pydantic-v2-dataclass-lookaround-regex-engine branch from e93ae63 to 25e6341 Compare June 15, 2026 11:13
The dataclass regex_engine unit tests hand-built the internal Reference graph
and asserted on render() string fragments, testing the implementation rather
than observable behavior (the negative case only proved "did not crash").

- Replace test_data_class_regex_engine_via_referenced_alias with an end-to-end
  fixture test driving lookaround_anyof_nullable.json through pydantic_v2.dataclass,
  with force_exec_validation so the generated module must import under pydantic's
  default rust regex engine -- the exact failure this PR fixes.
- Replace the "skips reference without fields" render() test with a direct
  has_lookaround_pattern() contract test for the sourceless-reference branch,
  which no generated schema exercises.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@DarkaMaul DarkaMaul force-pushed the fix-pydantic-v2-dataclass-lookaround-regex-engine branch from b6a6fdd to f1ff0bc Compare June 15, 2026 13:31
@koxudaxi koxudaxi merged commit 00dfab6 into koxudaxi:main Jun 16, 2026
61 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Breaking Change Analysis

Result: No breaking changes detected

Reasoning: PR #3370 is a bug fix for the pydantic v2 dataclass output path. Previously, pydantic v2 dataclasses whose fields (or referenced type aliases/root types) used regex lookaround assertions were generated WITHOUT ConfigDict(regex_engine="python-re"), so the generated code failed at class-build time with a SchemaError under pydantic's default rust regex engine. The PR now emits that config for the dataclass path, mirroring existing BaseModel behavior. The only change to generated output affects code that was previously non-importable/broken, so no working user setup can be broken by it. Supporting changes are non-breaking: has_lookaround_pattern() is an internal helper (not public API), and its new follow_references parameter is opt-in with a default of False that preserves prior behavior. There are no CLI option changes, no default-behavior changes to existing valid output, no required template updates, no Python version drop, and no error-handling contract changes.


This analysis was performed by Claude Code Action

@DarkaMaul DarkaMaul deleted the fix-pydantic-v2-dataclass-lookaround-regex-engine branch June 19, 2026 10:05
@github-actions

Copy link
Copy Markdown
Contributor

🎉 Released in 0.64.1

This PR is now available in the latest release. See the release notes for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants