FXC-3378 Pydantic v2

[yaugenst-flex]

Summary

Migrates the tidy3d python client from Pydantic v1 → v2 across models, validators, serialization, and schema generation.

This PR also includes the CI/tooling updates needed to keep the v2 codebase clean (type-only import handling and mypy checks).

Key changes

Pydantic v2 migration

• Standardize imports on pydantic / pydantic_settings; remove v1 compatibility imports.
• Replace @validator / @root_validator with @field_validator / @model_validator (incl. mode='before'|'after'|'wrap' where appropriate), updating signatures (ValidationInfo) and using object.__setattr__ when mutating frozen models.
• Replace .dict()/.json()/.parse_obj() with model_dump* / model_validate, and .schema() with model_json_schema().
• Migrate model configuration to model_config = ConfigDict(...).
• Migrate BaseSettings usage to pydantic_settings.BaseSettings.
• Update custom types to the v2 hooks:
  • __get_pydantic_core_schema__ / __get_pydantic_json_schema__
  • refactors toward Annotated[...] validators/serializers (BeforeValidator, PlainSerializer, etc.)
• Regenerate and commit updated schema artifacts under schemas/.

Pydantic-v2-specific fixes + regressions locked by tests

• Fix ValueMap validation bug caused by Mapping.keys/values name collisions; add regression tests.
• Add regression test for lazy proxy materialization that touches __pydantic_* internals (guards future Pydantic bumps).
• Preserve YAML serialization behavior for Infinity / -Infinity / NaN (string-based, JSON-aligned); add regression test and document why to_yaml() intentionally uses a JSON round-trip.
• Guard info.data[...] cross-field access in field validators to avoid KeyError when prerequisite fields fail validation; add regression tests.

Tooling / CI

• Add/update pre-commit + CI steps around type-only import organization and mypy type-signature checking (needed to keep the v2 migration maintainable).

Compatibility notes

• Goal: no intentional public API drift. The generated JSON schemas are treated as part of the public API and were regenerated/committed as part of the migration.
• tidy3d continues to raise domain-specific exceptions (tidy3d.exceptions.ValidationError, SetupError, etc.) for many validation failures; Pydantic core parse errors may differ slightly from v1 for invalid inputs.

How to test

• poetry run pre-commit run --all-files
• poetry run pytest -q

Follow-ups (post-merge)

• Refactor validator sprawl in Simulation/Medium into orchestrated helpers while preserving warning/error ordering.
• Audit load-only aliases and consider validation_alias / serialization_alias where appropriate (may require schema review).
• Evaluate removing JSON encode/decode round-trips only where behavior stays identical.

---

Note

Medium Risk
Primarily CI/tooling and dependency constraint changes; moderate risk due to expanding mypy coverage to all of tidy3d and introducing a new import-rewrite gate that could fail PRs or require code changes.

Overview
Adds a new CI and pre-commit verification (scripts/move_type_imports.py) that enforces type-only imports are moved under if TYPE_CHECKING, and wires this into the workflow’s required checks.

Updates verification tooling to standardize on Python 3.11 via a shared VERIFICATIONS_PY_VERSION, adjusts pytest config formatting, and broadens mypy to run across the full tidy3d package.

Tightens the Pydantic v2 dependency spec (pydantic >=2.9,<3) and adds pydantic-settings, plus includes libcst in dev extras to support the new import-guard tooling; also ignores simulation.json in git.

^{Written by Cursor Bugbot for commit bdf3a7a. This will update automatically on new commits. Configure here.}

[momchil-flex]

It seems like fundamentally there's just a few things that change? Maybe I'm missing something important/tricky. Could you explain if there's anything like that, and what changes going forward?

Also very important - we will certainly need to update the backend python code too and might need to update denormalizer, metadata api, and who knows what else.

[yaugenst-flex]

Yes the biggest changes are related to custom types and serialization, everything else is relatively straightforward. I'll write something up.

[mahlau-flex]

I have added a commit where all of the conflicts are resolved that can be resolved automatically. This is a bit tricky, because I had to make this a commit that does not resolve the other merge conflicts with some git magic (create two clones of the repo, make changes in one and copy them to the other as a "normal" commit). With the same strategy we should be able to bring this branch back on track piece by piece. In the end, we can squash all of these commits into a single rebase.

@yaugenst-flex if this sounds good to you, I will continue to make more commits here that resolve more and more conflicts. For some of them I will need help though, because at first glance it does not always seem clear what the best strategy for merging is.

[marcorudolphflex]

we should also update the readme here:

tidy3d/tidy3d/components/README.md

Line 137 in d2bc9d4

The `Simulation` can be exported as .json-like dictionary with `Simulation.json()`

with model_dump_json

[daquinteroflex]

Massive effort 🎉 💪

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}