chore(xtest): Shared Scenario/Instance Pydantic schema in otdf-sdk-mgr#450
chore(xtest): Shared Scenario/Instance Pydantic schema in otdf-sdk-mgr#450dmihalcik-virtru wants to merge 8 commits into
Conversation
…X-3302)
Introduces otdf_sdk_mgr.schema as the canonical Pydantic v2 model layer
for the multi-instance test harness. Both otdf-sdk-mgr and otdf-local
will read scenarios.yaml / instance.yaml through these models so the
on-disk YAML format has exactly one definition.
Models:
- SourceRef, PlatformPin, KasPin, SdkPin (with mutually-exclusive
dist|source|image validation on the platform/KAS pins)
- PortsConfig, Metadata, Fixtures
- Instance (apiVersion/kind/metadata/platform/ports/kas/...)
- ScenarioSdks (encrypt + decrypt maps mirroring xtest's
--sdks-encrypt / --sdks-decrypt convention)
- Suite (pytest select + flags)
- Scenario (composes Instance + ScenarioSdks + Suite)
Includes load_scenario / load_instance / dump_instance helpers and a
`python -m otdf_sdk_mgr.schema validate <path>` CLI entry that dispatches
on `kind:` so the same command validates both Scenario and Instance YAML.
Adds pydantic + ruamel.yaml to otdf-sdk-mgr's deps and a 6-test smoke
suite covering round-trips, pin validation, encrypt/decrypt union dedup,
and unknown-field rejection.
Refs: https://virtru.atlassian.net/browse/DSPX-3302
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new schema module for OpenTDF scenarios and instances, utilizing Pydantic models to ensure a canonical YAML definition across different tools. The changes include adding pydantic and ruamel.yaml as dependencies, implementing load/dump logic, and providing comprehensive smoke tests. The review feedback suggests several improvements to the implementation, specifically regarding the enforcement of UTF-8 encoding for file operations, the removal of redundant YAML configuration settings, and the use of more specific exception handling for validation and parsing errors.
There was a problem hiding this comment.
Pull request overview
First PR of a five-part stack introducing a multi-instance test harness for OpenTDF. This change adds only the shared Pydantic v2 schema in otdf-sdk-mgr (no consumers yet), plus a small CLI validator and unit tests. It establishes the on-disk shape for scenarios.yaml / instance.yaml so downstream PRs in the stack (otdf-local, xtest/conftest.py, Claude plugin) can import a single canonical definition.
Changes:
- Adds
otdf_sdk_mgr.schemawith strict (extra="forbid") v2 models:Scenario,Instance,PlatformPin,KasPin,SdkPin,ScenarioSdks,Suite, plus helpersload_scenario/load_instance/dump_instanceand apython -m otdf_sdk_mgr.schema validate <path>entrypoint. - Adds
pydantic>=2.6.0andruamel.yaml>=0.18.0to project dependencies (with correspondinguv.lockentries). - Adds 6 schema unit tests covering scenario round-trip, pin "exactly one source" invariant, KasPin features pass-through, SDK union, instance dump/load, and unknown-field rejection.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| otdf-sdk-mgr/pyproject.toml | Adds pydantic and ruamel.yaml runtime dependencies. |
| otdf-sdk-mgr/uv.lock | Auto-generated lockfile updates for the new dependencies (pydantic, pydantic-core, annotated-types, typing-inspection, ruamel-yaml). |
| otdf-sdk-mgr/src/otdf_sdk_mgr/schema.py | New module with Pydantic models, YAML load/dump helpers, and a validate CLI entry point. |
| otdf-sdk-mgr/tests/test_schema.py | Smoke tests for the new schema (round-trip, pin invariants, union, extra-forbid rejection). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… format xtest's --sdks / --sdks-encrypt / --sdks-decrypt accept whitespace- separated `sdk@version` tokens after #446 (e.g. `go@v0.24.0`, `go@main`, `go@*`). The version segment must match an actual directory under `xtest/sdk/<lang>/dist/`. Scenario version fields can be aliases (`lts`, `tip`) that only resolve to a concrete dist name once `otdf-sdk-mgr install scenario` runs, so we can't translate scenarios → pytest args from the scenario YAML alone. Adds two helpers so the scenario→pytest bridge has one canonical implementation: installed_json_for(scenario_path): The conventional sibling file `otdf-sdk-mgr install scenario` writes. `xtest/scenarios/x.yaml` → `xtest/scenarios/x.installed.json`. scenario_to_pytest_sdks(scenario, installed_json_path) -> dict: Returns `{"encrypt": ["go@v0.24.0", ...], "decrypt": [...]}`, reading the dist directory names recorded in installed.json. Raises FileNotFoundError with a `run install scenario first` hint when the record is missing (aliases can't be passed verbatim to xtest, so a clean error beats a confusing pytest failure). Raises ValueError when the scenario references an SDK the install record doesn't cover. Both `otdf-local scenario run` and `xtest/conftest.py`'s `--scenario`-default path will switch to this helper in the following PRs so they no longer drop the version when forwarding to pytest. Refs: https://virtru.atlassian.net/browse/DSPX-3302 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dmihalcik-virtru
changed the title
|
X-Test Failure Report |
| `dist` references a built binary at `xtest/platform/dist/<dist>/service` | ||
| produced by `otdf-sdk-mgr install platform:<version>`. `source.ref` is a | ||
| git ref to build from on demand. `image` is reserved for forward-compat | ||
| once container images are published; rejected at run time today. |
Summary
First PR in a five-part stack that introduces a multi-instance test harness and a Claude plugin for OpenTDF bug reproduction. This PR adds only the shared Pydantic schema in
otdf-sdk-mgr— no consumers yet.otdf_sdk_mgr.schemawith v2 models:Scenario,Instance,PlatformPin,KasPin,SdkPin,ScenarioSdks,Suite, etc.ScenarioSdks.encrypt/.decryptmirror xtest's existing--sdks-encrypt/--sdks-decryptconvention so a→b-only scenarios are first-class.python -m otdf_sdk_mgr.schema validate <path>validates either a Scenario or an Instance file based on itskind:.pydantic+ruamel.yamltootdf-sdk-mgr/pyproject.toml.Stack
install scenarioinotdf-sdk-mgr(builds on this)otdf-localmulti-instance refactor + new CLI subcommandsxtest/conftest.pyintegration (--scenario,--instance).claude/skills/, settings, plugin manifest)Test plan
cd otdf-sdk-mgr && uv run pytest tests/test_schema.py— all 6 passuv run python -m otdf_sdk_mgr.schema validate <path>accepts a valid scenarios.yaml and rejects unknown fieldsJira: https://virtru.atlassian.net/browse/DSPX-3302
🤖 Generated with Claude Code