Skip to content

Commit c69afd6

Browse files
feat(otdf-local): multi-instance test environments (DSPX-3302)
Refactors otdf-local from a single-instance CLI (one platform checkout, fixed ports, hardcoded six KAS instances) into a multi-instance harness where each named instance under tests/instances/<name>/ owns its own opentdf.yaml, keys, KAS configs, and port range. Why --- A single bug report often describes a *combination* — platform v0.9.0 with Java SDK 0.7.8 and a KAS at a pre-release. Today a developer has to hand-edit configs and re-checkout the platform to reproduce. After this change: otdf-local instance init java-078 --from-scenario .../scenario.yaml otdf-local --instance java-078 up brings up exactly the topology the scenario describes, using platform binaries that otdf-sdk-mgr already provisioned (each instance, and each KAS within an instance, can reference a different pinned version). Two instances on disjoint ports.base can coexist on a developer laptop. What changes ------------ otdf-local now depends on otdf-sdk-mgr via a uv path source so both tools share the canonical Scenario/Instance schema. Settings (otdf_local.config.settings): - New instance_name (env-overridable via OTDF_LOCAL_INSTANCE_NAME), instance_dir, instances_root, instance_yaml properties. - platform_dir becomes optional; legacy sibling-discovery only kicks in when no per-instance configuration is present. - platform_binary_for(dist) resolves to the otdf-sdk-mgr-managed xtest/platform/dist/<dist>/service binary. - keys_dir, logs_dir, config_dir, platform_config, and get_kas_config_path switch to per-instance paths whenever instance.yaml exists; legacy behavior is preserved otherwise. - load_instance() reads the per-instance manifest via the shared Pydantic model. Ports (otdf_local.config.ports): - KAS_OFFSETS exposes the offset table (alpha=+101, beta=+202, ..., km2=+606) so multiple instances on different bases get disjoint port ranges. The legacy 8080-based constants are preserved as defaults. - get_kas_port(name, base=...) computes the port relative to base. Services (otdf_local.services.platform / .kas): - PlatformService.start() and KASService.start() use the pinned dist binary at xtest/platform/dist/<dist>/service when an instance is loaded, with cwd set to the recorded worktree so the binary finds its embedded resources. Legacy `go run ./service` path runs unchanged when no instance is active. - KASService.is_key_management defers to the manifest's `mode` field instead of the legacy name-based heuristic; per-KAS features (e.g. ec_tdf_enabled) pass through to opentdf.yaml. - KASManager constructs only the KAS instances listed in instance.yaml's kas: map. start_standard / start_km filter on is_key_management so subset topologies still work. utils.keys.setup_golden_keys: - Writes key files into the target directory (per-instance keys_dir or legacy platform_dir) and uses absolute paths in the generated keys_config so the binary finds them regardless of cwd. CLI: - New top-level --instance option threads through every command via OTDF_LOCAL_INSTANCE_NAME. - New `instance` subcommand group: init [--from-scenario PATH], ls --json, rm. - New `scenario` subcommand: `run <path>` translates the scenario's suite block into `pytest --sdks-encrypt ... --sdks-decrypt ... --containers ...` under xtest/ with OTDF_LOCAL_INSTANCE_NAME set. Tests (otdf-local/tests/test_multi_instance.py): - Port arithmetic at default and alternate bases. - Settings round-trip with and without an instance.yaml. - platform_binary_for resolves under the otdf-sdk-mgr-managed xtest/platform/ tree. .gitignore additions: - tests/instances/ (per-instance config and logs) - xtest/scenarios/*.installed.json (provisioning records) - .claude/tmp/ Backward compatibility: - `otdf-local up` with no --instance flag keeps working against a sibling platform/ checkout. Refs: https://virtru.atlassian.net/browse/DSPX-3302 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent c6a7895 commit c69afd6

12 files changed

Lines changed: 719 additions & 69 deletions

File tree

.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,10 @@ xtest/sdk/java/cmdline.jar
3131
/xtest/otdfctl/
3232

3333
/tmp/
34+
35+
# Multi-instance test harness state (DSPX-3302). Per-instance config, logs, and
36+
# keys live under tests/instances/; otdf-sdk-mgr install scenario writes
37+
# .installed.json next to each scenarios.yaml.
38+
/instances/
39+
xtest/scenarios/*.installed.json
40+
.claude/tmp/

otdf-local/pyproject.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,16 @@ readme = "README.md"
66
requires-python = ">=3.11"
77
dependencies = [
88
"httpx>=0.27.0",
9+
"otdf-sdk-mgr",
910
"pydantic-settings>=2.2.0",
1011
"rich>=13.7.0",
1112
"ruamel.yaml>=0.18.0",
1213
"typer>=0.12.0",
1314
]
1415

16+
[tool.uv.sources]
17+
otdf-sdk-mgr = { path = "../otdf-sdk-mgr", editable = true }
18+
1519
[dependency-groups]
1620
dev = [
1721
"pyright>=1.1.408",

otdf-local/src/otdf_local/cli.py

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
"""Typer CLI for otdf_local - OpenTDF test environment management."""
22

33
import json
4+
import os
45
import shutil
56
import sys
67
import time
7-
from typing import Annotated
8+
from pathlib import Path
9+
from typing import Annotated, Optional
810

911
import httpx
1012
import typer
@@ -44,6 +46,18 @@
4446
)
4547

4648

49+
def _register_subapps() -> None:
50+
"""Defer imports so the schema dependency only loads when needed."""
51+
from otdf_local.cli_instance import instance_app
52+
from otdf_local.cli_scenario import scenario_app
53+
54+
app.add_typer(instance_app, name="instance")
55+
app.add_typer(scenario_app, name="scenario")
56+
57+
58+
_register_subapps()
59+
60+
4761
def _show_provision_error(result: ProvisionResult, target: str) -> None:
4862
"""Display provisioning error with stderr details."""
4963
print_error(f"{target} provisioning failed (exit code {result.return_code})")
@@ -75,9 +89,19 @@ def main(
7589
is_eager=True,
7690
),
7791
] = False,
92+
instance: Annotated[
93+
Optional[str],
94+
typer.Option(
95+
"--instance",
96+
help='Named instance under tests/instances/. Defaults to "default" (or $OTDF_LOCAL_INSTANCE_NAME).',
97+
),
98+
] = None,
7899
) -> None:
79100
"""OpenTDF test environment management CLI."""
80-
pass
101+
if instance is not None:
102+
os.environ["OTDF_LOCAL_INSTANCE_NAME"] = instance
103+
# Invalidate the cached Settings so subsequent commands see the new value
104+
get_settings.cache_clear()
81105

82106

83107
@app.command()
Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
"""`otdf-local instance` subcommands: init / ls / rm."""
2+
3+
from __future__ import annotations
4+
5+
import shutil
6+
from pathlib import Path
7+
from typing import Annotated, Optional
8+
9+
import typer
10+
from otdf_sdk_mgr.schema import Instance, Metadata, PlatformPin, PortsConfig, dump_instance
11+
12+
from otdf_local.config.settings import get_settings
13+
14+
instance_app = typer.Typer(help="Manage named test environment instances.")
15+
16+
17+
@instance_app.command("init")
18+
def init(
19+
name: Annotated[str, typer.Argument(help="Instance name (used as directory name)")],
20+
from_scenario: Annotated[
21+
Optional[Path],
22+
typer.Option("--from-scenario", help="Initialize from a scenarios.yaml or instance.yaml"),
23+
] = None,
24+
ports_base: Annotated[
25+
int,
26+
typer.Option("--ports-base", help="Base port (KAS ports computed as base+N*101)"),
27+
] = 8080,
28+
platform_dist: Annotated[
29+
Optional[str],
30+
typer.Option("--platform", help="Platform dist version (e.g., v0.9.0)"),
31+
] = None,
32+
) -> None:
33+
"""Scaffold a new instance directory at tests/instances/<name>/."""
34+
settings = get_settings()
35+
instance_dir = settings.instances_root / name
36+
37+
if from_scenario is not None:
38+
_init_from_scenario(name, from_scenario, instance_dir)
39+
else:
40+
if platform_dist is None:
41+
typer.echo("Error: --platform <dist> is required when not using --from-scenario", err=True)
42+
raise typer.Exit(2)
43+
_init_minimal(name, instance_dir, ports_base, platform_dist)
44+
45+
_validate_port_uniqueness(settings.instances_root, name)
46+
typer.echo(f" Initialized instance '{name}' at {instance_dir}")
47+
48+
49+
def _init_from_scenario(name: str, scenario_path: Path, instance_dir: Path) -> None:
50+
"""Copy the embedded Instance from a Scenario or load a standalone Instance."""
51+
from otdf_sdk_mgr.schema import load_instance, load_scenario
52+
from ruamel.yaml import YAML
53+
54+
y = YAML(typ="safe")
55+
raw = y.load(scenario_path.read_text())
56+
if not isinstance(raw, dict):
57+
raise typer.BadParameter(f"{scenario_path} top-level YAML must be a mapping")
58+
kind = raw.get("kind")
59+
if kind == "Scenario":
60+
scenario = load_scenario(scenario_path)
61+
instance = scenario.instance
62+
elif kind == "Instance":
63+
instance = load_instance(scenario_path)
64+
else:
65+
raise typer.BadParameter(f"{scenario_path} has unknown kind {kind!r}")
66+
# Ensure the metadata name matches the chosen directory name.
67+
instance.metadata = Metadata(**{**instance.metadata.model_dump(exclude_none=True), "name": name})
68+
instance_dir.mkdir(parents=True, exist_ok=True)
69+
(instance_dir / "kas").mkdir(parents=True, exist_ok=True)
70+
(instance_dir / "keys").mkdir(mode=0o700, parents=True, exist_ok=True)
71+
(instance_dir / "logs").mkdir(parents=True, exist_ok=True)
72+
dump_instance(instance, instance_dir / "instance.yaml")
73+
74+
75+
def _init_minimal(name: str, instance_dir: Path, ports_base: int, platform_dist: str) -> None:
76+
"""Create a barebones instance.yaml with default KAS layout."""
77+
instance = Instance(
78+
metadata=Metadata(name=name),
79+
platform=PlatformPin(dist=platform_dist),
80+
ports=PortsConfig(base=ports_base),
81+
kas={},
82+
)
83+
instance_dir.mkdir(parents=True, exist_ok=True)
84+
(instance_dir / "kas").mkdir(parents=True, exist_ok=True)
85+
(instance_dir / "keys").mkdir(mode=0o700, parents=True, exist_ok=True)
86+
(instance_dir / "logs").mkdir(parents=True, exist_ok=True)
87+
dump_instance(instance, instance_dir / "instance.yaml")
88+
89+
90+
def _validate_port_uniqueness(instances_root: Path, new_name: str) -> None:
91+
"""Warn if another instance shares the same `ports.base`."""
92+
from otdf_sdk_mgr.schema import load_instance
93+
94+
new_yaml = instances_root / new_name / "instance.yaml"
95+
if not new_yaml.exists():
96+
return
97+
new_inst = load_instance(new_yaml)
98+
new_base = new_inst.ports.base
99+
if not instances_root.exists():
100+
return
101+
for child in instances_root.iterdir():
102+
if not child.is_dir() or child.name == new_name:
103+
continue
104+
other_yaml = child / "instance.yaml"
105+
if not other_yaml.is_file():
106+
continue
107+
try:
108+
other = load_instance(other_yaml)
109+
except Exception:
110+
continue
111+
if other.ports.base == new_base:
112+
typer.echo(
113+
f" Warning: instance '{child.name}' already uses ports.base={new_base}; "
114+
f"running both simultaneously will collide. Change one with `otdf-local instance init`.",
115+
err=True,
116+
)
117+
118+
119+
@instance_app.command("ls")
120+
def ls(
121+
as_json: Annotated[bool, typer.Option("--json", "-j", help="Emit JSON")] = False,
122+
) -> None:
123+
"""List known instances."""
124+
import json as _json
125+
126+
from otdf_sdk_mgr.schema import load_instance
127+
128+
settings = get_settings()
129+
root = settings.instances_root
130+
if not root.exists():
131+
if as_json:
132+
typer.echo(_json.dumps([]))
133+
else:
134+
typer.echo(" (no instances yet)")
135+
return
136+
rows: list[dict[str, object]] = []
137+
for child in sorted(root.iterdir()):
138+
if not child.is_dir():
139+
continue
140+
ymp = child / "instance.yaml"
141+
if not ymp.is_file():
142+
continue
143+
try:
144+
inst = load_instance(ymp)
145+
except Exception as e:
146+
rows.append({"name": child.name, "error": str(e)})
147+
continue
148+
rows.append(
149+
{
150+
"name": child.name,
151+
"platform": (
152+
inst.platform.dist
153+
or (inst.platform.source.ref if inst.platform.source else inst.platform.image)
154+
),
155+
"ports_base": inst.ports.base,
156+
"kas": list(inst.kas.keys()),
157+
}
158+
)
159+
if as_json:
160+
typer.echo(_json.dumps(rows, indent=2))
161+
else:
162+
for row in rows:
163+
typer.echo(f" {row}")
164+
165+
166+
@instance_app.command("rm")
167+
def rm(
168+
name: Annotated[str, typer.Argument(help="Instance to remove")],
169+
yes: Annotated[bool, typer.Option("--yes", "-y", help="Skip confirmation")] = False,
170+
) -> None:
171+
"""Remove an instance directory."""
172+
settings = get_settings()
173+
instance_dir = settings.instances_root / name
174+
if not instance_dir.exists():
175+
typer.echo(f"Error: instance '{name}' not found at {instance_dir}", err=True)
176+
raise typer.Exit(1)
177+
if not yes:
178+
confirm = typer.confirm(f"Delete {instance_dir}?", default=False)
179+
if not confirm:
180+
typer.echo("aborted")
181+
raise typer.Exit(1)
182+
shutil.rmtree(instance_dir)
183+
typer.echo(f" Removed {instance_dir}")
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
"""`otdf-local scenario` subcommands.
2+
3+
Today's surface area is intentionally narrow — `run` is the only command
4+
that's part of the bug-repro MVP. Bisect and other higher-level loops are
5+
deferred (see plan §9).
6+
"""
7+
8+
from __future__ import annotations
9+
10+
import os
11+
import subprocess
12+
from pathlib import Path
13+
from typing import Annotated
14+
15+
import typer
16+
from otdf_sdk_mgr.schema import Scenario, load_scenario
17+
18+
from otdf_local.config.settings import get_settings
19+
20+
scenario_app = typer.Typer(help="Run scenarios.yaml against a healthy instance.")
21+
22+
23+
def _build_pytest_args(scenario: Scenario) -> list[str]:
24+
"""Translate the scenario's `suite` block into pytest CLI args."""
25+
suite = scenario.suite
26+
args: list[str] = [suite.select]
27+
28+
encrypt_sdks = list(scenario.sdks.encrypt.keys())
29+
decrypt_sdks = list(scenario.sdks.decrypt.keys())
30+
if encrypt_sdks:
31+
args.extend(["--sdks-encrypt", " ".join(encrypt_sdks)])
32+
if decrypt_sdks:
33+
args.extend(["--sdks-decrypt", " ".join(decrypt_sdks)])
34+
if suite.containers:
35+
args.extend(["--containers", suite.containers])
36+
if suite.markers:
37+
args.extend(["-m", suite.markers])
38+
args.extend(suite.extra_args)
39+
return args
40+
41+
42+
@scenario_app.command("run")
43+
def run(
44+
path: Annotated[Path, typer.Argument(help="Path to scenarios.yaml")],
45+
instance: Annotated[
46+
str | None,
47+
typer.Option(
48+
"--instance",
49+
help="Override which instance to use (defaults to scenario.instance.metadata.name)",
50+
),
51+
] = None,
52+
extra: Annotated[
53+
list[str] | None,
54+
typer.Argument(help="Extra args passed through to pytest (after --)"),
55+
] = None,
56+
) -> None:
57+
"""Run the pytest suite declared by the scenario against its instance."""
58+
if not path.exists():
59+
typer.echo(f"Error: {path} not found", err=True)
60+
raise typer.Exit(1)
61+
62+
scenario = load_scenario(path)
63+
instance_name = instance or scenario.instance.metadata.name
64+
if not instance_name:
65+
typer.echo("Error: scenario.instance.metadata.name not set; pass --instance", err=True)
66+
raise typer.Exit(2)
67+
68+
settings = get_settings()
69+
# Force the chosen instance via env so child pytest invocations agree.
70+
os.environ["OTDF_LOCAL_INSTANCE_NAME"] = instance_name
71+
72+
xtest_root = settings.xtest_root
73+
if not xtest_root.exists():
74+
typer.echo(f"Error: xtest root not found at {xtest_root}", err=True)
75+
raise typer.Exit(1)
76+
77+
pytest_args = _build_pytest_args(scenario)
78+
if extra:
79+
pytest_args.extend(extra)
80+
81+
cmd = ["uv", "run", "pytest", *pytest_args]
82+
typer.echo(f" Running: {' '.join(cmd)} (cwd={xtest_root})")
83+
completed = subprocess.run(cmd, cwd=xtest_root)
84+
raise typer.Exit(completed.returncode)

otdf-local/src/otdf_local/config/ports.py

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,14 +33,40 @@ class Ports:
3333
"km2": "KAS_KM2",
3434
}
3535

36+
# Offset of each KAS port from `base` (which is the platform port).
37+
# The defaults at base=8080 reproduce the historical 8181/8282/... layout.
38+
KAS_OFFSETS: ClassVar[dict[str, int]] = {
39+
"alpha": 101,
40+
"beta": 202,
41+
"gamma": 303,
42+
"delta": 404,
43+
"km1": 505,
44+
"km2": 606,
45+
}
46+
3647
@classmethod
37-
def get_kas_port(cls, name: str) -> int:
38-
"""Get port for a KAS instance by name."""
48+
def get_kas_port(cls, name: str, *, base: int | None = None) -> int:
49+
"""Get port for a KAS instance by name.
50+
51+
When `base` is provided, the port is computed as `base + offset` so
52+
multiple instances can coexist on disjoint port ranges. Otherwise the
53+
legacy class constants are returned (base=8080 layout).
54+
"""
55+
if base is not None:
56+
offset = cls.KAS_OFFSETS.get(name)
57+
if offset is None:
58+
raise ValueError(f"Unknown KAS instance: {name}")
59+
return base + offset
3960
attr = cls._KAS_NAMES.get(name)
4061
if attr is None:
4162
raise ValueError(f"Unknown KAS instance: {name}")
4263
return getattr(cls, attr)
4364

65+
@classmethod
66+
def platform_port_for(cls, base: int) -> int:
67+
"""Return the platform port for a given `base`. Trivially `base` today."""
68+
return base
69+
4470
@classmethod
4571
def all_kas_names(cls) -> list[str]:
4672
"""Return all KAS instance names."""

0 commit comments

Comments
 (0)