Skip to content

Commit 1863b05

Browse files
authored
feat: enforce runtime domain invariants (#45)
## What - reject blank task, session, MCP, tool, artifact, and result identities - require positive execution hints and finite non-negative budgets/counts - reject ambiguous session start/resume state, duplicate MCP names, and conflicting tool filters - canonicalize sequence value fields while rejecting scalar strings - add AgentResult.is_success - represent unknown Usage and cost as None while preserving reported zero - preserve partial provider telemetry without inventing cache counts - fix Claude resume result fallback when the SDK omits its session id ## Why Several invalid or ambiguous states were constructible, and missing telemetry was indistinguishable from a real zero. ## Root cause The frozen dataclasses provided immutability but did not own their cross-field invariants, while adapter coercion helpers normalized missing and malformed counters to zero. ## Checks - 321 passed, 12 skipped - independent domain audit completed - ruff check src tests - mypy - uv lock --check - wheel and sdist build ## Compatibility This intentionally rejects previously accepted invalid constructor inputs and changes unknown Usage fields from zero to None. ## Stack - Base: #44 - Next: #46
1 parent 5ef007f commit 1863b05

15 files changed

Lines changed: 570 additions & 37 deletions

CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1515
absent or rejected structured output.
1616
- All built-in adapters and public fake runtimes now validate returned values
1717
against `output_schema` locally, including textual JSON fallbacks.
18+
- `AgentResult.is_success` provides one canonical success predicate.
1819

1920
### Changed
2021

2122
- Pydantic structured-output parsing now requests strict validation, so values
2223
such as `"42"` no longer coerce into integer fields.
24+
- BREAKING: task and value-object constructors now reject blank identities,
25+
non-positive execution hints, invalid budgets, ambiguous session inputs,
26+
duplicate MCP names, and conflicting tool filters.
27+
- BREAKING: unknown `Usage` values, including cost, are `None` rather than zero;
28+
an explicit provider-reported zero remains `0`/`0.0`.
2329

2430
### Fixed
2531

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -143,9 +143,11 @@ silently dropping it.
143143

144144
`AgentResult` returns output, finish reason (see `FinishReason`), locally
145145
validated structured output, usage, cost, session id, tool-call audits, and
146-
provider metadata. `parsed_output_available` distinguishes an absent payload
147-
from valid JSON `null` (both use `parsed_output=None`). `artifacts` is a reserved
148-
field: no built-in runtime populates it yet, so it is always an empty tuple today.
146+
provider metadata. `is_success` is true only for a natural, error-free
147+
completion. Unknown usage and cost fields are `None`; reported zero remains
148+
distinct. `parsed_output_available` distinguishes an absent payload from valid
149+
JSON `null` (both use `parsed_output=None`). `artifacts` is a reserved field: no
150+
built-in runtime populates it yet, so it is always an empty tuple today.
149151

150152
## Docs
151153

docs/api-stability.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,13 @@ import the names from the top-level package instead.
3333
convention and compare by value. The wrappers remain plain `dict` subclasses,
3434
so `dataclasses.asdict`, `pickle`, `copy.deepcopy`, and JSON serialization
3535
keep working.
36+
- **Constructors enforce domain invariants.** Goals and identifiers are
37+
non-blank; execution hints are positive; counts and monetary values are
38+
finite and non-negative; session start and resume inputs are mutually
39+
exclusive; MCP server names and tool filters are unambiguous.
40+
- **Unknown telemetry is not zero.** Every `Usage` field is nullable. `None`
41+
means the provider did not report a value; `0` and `0.0` mean it explicitly
42+
reported zero.
3643
- **Unsupported inputs raise, they are not dropped.** An adapter that cannot honor
3744
a task field raises `UnsupportedTaskInputError`; the one exception is
3845
vendor-option drift, which is recorded in

docs/providers.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,10 @@ vocabulary (for example `claude-agent-sdk` 0.2.x accepts
3838
`low`/`medium`/`high`/`xhigh`/`max`), and an SDK too old to accept `effort` at
3939
all records the drop in `AgentResult.metadata["dropped_options"]`.
4040

41+
Usage fields are `None` when a provider omits its usage breakdown or cost.
42+
Provider-reported zeros remain numeric zero, so callers can distinguish a free
43+
or zero-token run from missing telemetry.
44+
4145
Claude uses the `claude-agent-sdk` package and maps working directory,
4246
permissions, filesystem access (a `READ_ONLY` filesystem forces `plan` mode),
4347
MCP servers, sessions, structured output, tool allow/deny lists, runtime

src/agent_runtime_kit/_types.py

Lines changed: 150 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
from collections.abc import Mapping
66
from dataclasses import dataclass, field
77
from enum import Enum
8+
from math import isfinite
89
from pathlib import Path
910
from typing import Any, Generic, NoReturn, Protocol, TypeVar, cast, runtime_checkable
1011
from uuid import uuid4
@@ -81,6 +82,44 @@ def _coerce_enum(enum_cls: type[_EnumT], value: Any, field_name: str) -> _EnumT:
8182
raise ValueError(f"invalid {field_name} {value!r}; valid values: {valid}") from None
8283

8384

85+
def _require_nonblank(value: str, field_name: str) -> None:
86+
if not isinstance(value, str) or not value.strip():
87+
raise ValueError(f"{field_name} must be a non-empty string")
88+
89+
90+
def _require_unique_nonblank(values: tuple[str, ...], field_name: str) -> None:
91+
for value in values:
92+
_require_nonblank(value, field_name)
93+
duplicates = sorted({value for value in values if values.count(value) > 1})
94+
if duplicates:
95+
raise ValueError(f"{field_name} contains duplicates: {duplicates}")
96+
97+
98+
def _tuple_value(value: Any, field_name: str) -> tuple[Any, ...]:
99+
if isinstance(value, (str, bytes)):
100+
raise ValueError(f"{field_name} must be a sequence, not a scalar string")
101+
try:
102+
return tuple(value)
103+
except TypeError as exc:
104+
raise ValueError(f"{field_name} must be an iterable") from exc
105+
106+
107+
def _validate_optional_count(value: int | None, field_name: str) -> None:
108+
if value is None:
109+
return
110+
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
111+
raise ValueError(f"{field_name} must be a non-negative integer or None")
112+
113+
114+
def _validate_optional_amount(value: float | None, field_name: str) -> None:
115+
if value is None:
116+
return
117+
if isinstance(value, bool) or not isinstance(value, (int, float)):
118+
raise ValueError(f"{field_name} must be a non-negative finite number or None")
119+
if value < 0 or not isfinite(float(value)):
120+
raise ValueError(f"{field_name} must be a non-negative finite number or None")
121+
122+
84123
class AgentRuntimeKind(str, Enum):
85124
"""Supported runtime families."""
86125

@@ -257,6 +296,17 @@ class McpServerConfig:
257296
env: Mapping[str, str] = field(default_factory=dict)
258297

259298
def __post_init__(self) -> None:
299+
_require_nonblank(self.name, "McpServerConfig.name")
300+
_require_nonblank(self.command, "McpServerConfig.command")
301+
object.__setattr__(self, "args", _tuple_value(self.args, "McpServerConfig.args"))
302+
if not all(isinstance(arg, str) for arg in self.args):
303+
raise ValueError("McpServerConfig.args must contain only strings")
304+
if not isinstance(self.env, Mapping):
305+
raise ValueError("McpServerConfig.env must be a mapping")
306+
for key, value in self.env.items():
307+
_require_nonblank(key, "McpServerConfig.env key")
308+
if not isinstance(value, str):
309+
raise ValueError("McpServerConfig.env values must be strings")
260310
object.__setattr__(self, "env", _freeze_mapping(self.env))
261311

262312

@@ -285,6 +335,25 @@ def __post_init__(self) -> None:
285335
"filesystem",
286336
_coerce_enum(FilesystemAccess, self.filesystem, "filesystem"),
287337
)
338+
object.__setattr__(
339+
self,
340+
"allowed_tools",
341+
_tuple_value(self.allowed_tools, "PermissionProfile.allowed_tools"),
342+
)
343+
object.__setattr__(
344+
self,
345+
"disallowed_tools",
346+
_tuple_value(self.disallowed_tools, "PermissionProfile.disallowed_tools"),
347+
)
348+
_require_unique_nonblank(self.allowed_tools, "PermissionProfile.allowed_tools")
349+
_require_unique_nonblank(self.disallowed_tools, "PermissionProfile.disallowed_tools")
350+
overlap = sorted(set(self.allowed_tools) & set(self.disallowed_tools))
351+
if overlap:
352+
raise ValueError(
353+
"PermissionProfile cannot both allow and disallow tools: " + ", ".join(overlap)
354+
)
355+
if self.network is not None and not isinstance(self.network, bool):
356+
raise ValueError("PermissionProfile.network must be bool or None")
288357

289358

290359
@dataclass(frozen=True)
@@ -298,6 +367,14 @@ class ToolCallAudit:
298367
duration_ms: int = 0
299368

300369
def __post_init__(self) -> None:
370+
_require_nonblank(self.tool_name, "ToolCallAudit.tool_name")
371+
_require_nonblank(self.status, "ToolCallAudit.status")
372+
if (
373+
isinstance(self.duration_ms, bool)
374+
or not isinstance(self.duration_ms, int)
375+
or self.duration_ms < 0
376+
):
377+
raise ValueError("ToolCallAudit.duration_ms must be a non-negative integer")
301378
object.__setattr__(self, "arguments", _freeze_mapping(self.arguments))
302379

303380

@@ -310,6 +387,8 @@ class ArtifactRef:
310387
metadata: Mapping[str, Any] = field(default_factory=dict)
311388

312389
def __post_init__(self) -> None:
390+
_require_nonblank(self.uri, "ArtifactRef.uri")
391+
_require_nonblank(self.kind, "ArtifactRef.kind")
313392
object.__setattr__(self, "metadata", _freeze_mapping(self.metadata))
314393

315394

@@ -325,25 +404,39 @@ class SessionResumeState:
325404
session_id: str
326405
transcript: tuple[Any, ...] = ()
327406

407+
def __post_init__(self) -> None:
408+
_require_nonblank(self.session_id, "SessionResumeState.session_id")
409+
object.__setattr__(
410+
self,
411+
"transcript",
412+
_tuple_value(self.transcript, "SessionResumeState.transcript"),
413+
)
414+
328415

329416
@dataclass(frozen=True)
330417
class Usage:
331418
"""Token and cost metadata reported by a runtime.
332419
333420
``input_tokens`` counts prompt tokens excluding Anthropic-style cache reads and
334421
cache creation, which are reported separately in ``cache_read_tokens`` and
335-
``cache_creation_tokens``. ``total_tokens`` is the vendor-reported total when the
336-
runtime provides one, and ``None`` when it does not (so "unknown" is
337-
distinguishable from zero). ``cost_usd`` is ``0.0`` when the provider reports no
338-
cost.
422+
``cache_creation_tokens``. Every field is ``None`` when unknown, so a reported
423+
zero remains distinguishable from missing telemetry.
339424
"""
340425

341-
input_tokens: int = 0
342-
output_tokens: int = 0
343-
cache_read_tokens: int = 0
344-
cache_creation_tokens: int = 0
426+
input_tokens: int | None = None
427+
output_tokens: int | None = None
428+
cache_read_tokens: int | None = None
429+
cache_creation_tokens: int | None = None
345430
total_tokens: int | None = None
346-
cost_usd: float = 0.0
431+
cost_usd: float | None = None
432+
433+
def __post_init__(self) -> None:
434+
_validate_optional_count(self.input_tokens, "Usage.input_tokens")
435+
_validate_optional_count(self.output_tokens, "Usage.output_tokens")
436+
_validate_optional_count(self.cache_read_tokens, "Usage.cache_read_tokens")
437+
_validate_optional_count(self.cache_creation_tokens, "Usage.cache_creation_tokens")
438+
_validate_optional_count(self.total_tokens, "Usage.total_tokens")
439+
_validate_optional_amount(self.cost_usd, "Usage.cost_usd")
347440

348441

349442
@dataclass(frozen=True)
@@ -375,6 +468,30 @@ class AgentTask:
375468
metadata: Mapping[str, Any] = field(default_factory=dict)
376469

377470
def __post_init__(self) -> None:
471+
_require_nonblank(self.goal, "AgentTask.goal")
472+
_require_nonblank(self.task_id, "AgentTask.task_id")
473+
if self.model is not None:
474+
_require_nonblank(self.model, "AgentTask.model")
475+
if self.reasoning_effort is not None:
476+
_require_nonblank(self.reasoning_effort, "AgentTask.reasoning_effort")
477+
if isinstance(self.sdk_executions, bool) or not isinstance(self.sdk_executions, int):
478+
raise ValueError("AgentTask.sdk_executions must be a positive integer")
479+
if self.sdk_executions < 1:
480+
raise ValueError("AgentTask.sdk_executions must be a positive integer")
481+
_validate_optional_amount(self.budget_usd, "AgentTask.budget_usd")
482+
if self.session_id is not None:
483+
_require_nonblank(self.session_id, "AgentTask.session_id")
484+
if self.session_id is not None and self.resume_from is not None:
485+
raise ValueError("AgentTask.session_id and resume_from are mutually exclusive")
486+
object.__setattr__(
487+
self,
488+
"mcp_servers",
489+
_tuple_value(self.mcp_servers, "AgentTask.mcp_servers"),
490+
)
491+
server_names = tuple(server.name for server in self.mcp_servers)
492+
duplicates = sorted({name for name in server_names if server_names.count(name) > 1})
493+
if duplicates:
494+
raise ValueError(f"AgentTask.mcp_servers contains duplicate names: {duplicates}")
378495
object.__setattr__(self, "metadata", _freeze_mapping(self.metadata))
379496
if self.output_schema is not None:
380497
# Local import avoids an import cycle: _schema's typed errors import
@@ -405,16 +522,39 @@ class AgentResult:
405522
parsed_output_available: bool = False
406523

407524
def __post_init__(self) -> None:
525+
_require_nonblank(self.finish_reason, "AgentResult.finish_reason")
526+
if self.error is not None:
527+
_require_nonblank(self.error, "AgentResult.error")
528+
if self.finish_reason == FinishReason.DONE:
529+
raise ValueError("AgentResult.error requires a non-success finish_reason")
530+
if isinstance(self.rounds, bool) or not isinstance(self.rounds, int) or self.rounds < 0:
531+
raise ValueError("AgentResult.rounds must be a non-negative integer")
532+
object.__setattr__(
533+
self,
534+
"tool_calls",
535+
_tuple_value(self.tool_calls, "AgentResult.tool_calls"),
536+
)
537+
object.__setattr__(
538+
self,
539+
"artifacts",
540+
_tuple_value(self.artifacts, "AgentResult.artifacts"),
541+
)
408542
object.__setattr__(self, "metadata", _freeze_mapping(self.metadata))
409543
if self.parsed_output is not None and not self.parsed_output_available:
410544
object.__setattr__(self, "parsed_output_available", True)
411545

412546
@property
413-
def cost_usd(self) -> float:
547+
def cost_usd(self) -> float | None:
414548
"""Return the reported task cost in USD."""
415549

416550
return self.usage.cost_usd
417551

552+
@property
553+
def is_success(self) -> bool:
554+
"""Whether the runtime completed naturally without an error."""
555+
556+
return self.finish_reason == FinishReason.DONE and self.error is None
557+
418558

419559
_ParsedT = TypeVar("_ParsedT")
420560

src/agent_runtime_kit/adapters/_common.py

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
import os
77
from collections.abc import Iterable, Mapping
88
from importlib import metadata, util
9+
from math import isfinite
910
from typing import Any
1011

1112
from agent_runtime_kit._errors import UnsupportedTaskInputError
@@ -259,13 +260,27 @@ def field_value(value: Any, name: str, default: Any = None) -> Any:
259260
return getattr(value, name, default)
260261

261262

262-
def optional_int(value: Any) -> int:
263-
"""Coerce a vendor-reported value to ``int``, treating ``None``/junk as 0."""
263+
def optional_int(value: Any) -> int | None:
264+
"""Coerce a non-negative vendor count, preserving unknown as None."""
264265

266+
if value is None or isinstance(value, bool):
267+
return None
268+
if isinstance(value, float):
269+
if not isfinite(value) or not value.is_integer():
270+
return None
271+
if isinstance(value, str) and not value.strip():
272+
return None
265273
try:
266-
return int(value or 0)
267-
except (TypeError, ValueError):
268-
return 0
274+
parsed = int(value)
275+
except (TypeError, ValueError, OverflowError):
276+
return None
277+
if not isinstance(value, (int, float, str)):
278+
try:
279+
if value != parsed:
280+
return None
281+
except Exception:
282+
return None
283+
return parsed if parsed >= 0 else None
269284

270285

271286
def optional_str(value: Any) -> str | None:

src/agent_runtime_kit/adapters/antigravity.py

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -918,14 +918,24 @@ def _default_data_dir() -> Path:
918918

919919

920920
def _usage_from(value: Any) -> Usage:
921+
if value is None:
922+
return Usage()
921923
prompt_tokens = optional_int(getattr(value, "prompt_token_count", None))
922924
output_tokens = optional_int(getattr(value, "candidates_token_count", None))
923925
thoughts = optional_int(getattr(value, "thoughts_token_count", None))
924926
cache_read = optional_int(getattr(value, "cached_content_token_count", None))
925927
total = optional_int(getattr(value, "total_token_count", None))
926928
return Usage(
927-
input_tokens=max(prompt_tokens - cache_read, 0),
928-
output_tokens=output_tokens + thoughts,
929+
input_tokens=(
930+
max(prompt_tokens - cache_read, 0)
931+
if prompt_tokens is not None and cache_read is not None
932+
else None
933+
),
934+
output_tokens=(
935+
output_tokens + thoughts
936+
if output_tokens is not None and thoughts is not None
937+
else None
938+
),
929939
cache_read_tokens=cache_read,
930940
total_tokens=total,
931941
)

0 commit comments

Comments
 (0)