Skip to content

Commit bf770ed

Browse files
Cover fan-out degrade refinements (0069) (#164)
* Cover fan-out degrade refinements (0069) Adopt proposal 0069 (spec v0.59.0), which refines 0066's fan-out degrade contribution. python's runtime already satisfied all three refinements, so this is conformance coverage with no library behavior change: - Fixture 069 is a mixed fixture; its FI-degrade cases run in the pipeline-utilities runner and its crash_injection/resume case in the checkpoint runner. The two runners partition cases on a checkpointer / resume / crash_injection marker. The checkpoint runner gains a small static-degrade FailureIsolationMiddleware translator for the degrade. - A unit test pins the strict-element-reducer caveat: a degrade that nulls a slot under concat_flatten / merge_all still raises ReducerError (the engine merge runs the reducer; the fan-in does not suppress it). Advance the pinned spec to v0.59.0. * Guard against vacuous cases-fixture skips (0069) Address PR review on the fixture-069 split: - Both cases-loops now count the cases they drive and assert at least one ran, so a cases-shaped fixture whose cases are all skipped (a routing mistake) fails loudly instead of passing vacuously. This is the structural guard for the silent-skip hazard the split introduced. - _translate_fi_instance_middleware validates that a failure_isolation entry carries a degraded_update and names the fan-out node in its errors, so a malformed fixture fails actionably instead of with a bare KeyError.
1 parent 22aad2a commit bf770ed

10 files changed

Lines changed: 176 additions & 18 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The
1818
- **Observer privacy flag `disable_llm_payload` renamed to `disable_provider_payload`** (proposal 0059, observability §5.5.4, spec v0.54.0). The observer-level flag on both bundled observers (`OTelObserver` and `LangfuseObserver`) is renamed, and its scope broadens from LLM-completion payload to any provider-call payload (LLM completion today; embedding and rerank when those land). This is a breaking change to both observer constructors: config passing `disable_llm_payload=True` (or `False`) updates to `disable_provider_payload=...` with no other change. The default stays `True` (payload suppressed), and the gating behavior for `LlmCompletionEvent` / `LlmFailedEvent` rendering is unchanged at every existing site. The rename is the only part of proposal 0059 adopted this cycle: the retrieval-provider capability itself (the `EmbeddingProvider` protocol, the `EmbeddingEvent` / `EmbeddingFailedEvent` typed variants, and the embedding span / observation mapping) is not yet implemented and rides as `not-yet` in `conformance.toml`. The §5.5.4 rename touches existing LLM-payload gating, so it lands with the pin.
1919
- **Fan-out failure-isolation degrade contribution implemented** (proposal 0066, pipeline-utilities §9.3 / §9.8 / §11.7, spec v0.56.0). When `FailureIsolationMiddleware` degrades a fan-out instance, that instance is a success whose contribution is its `degraded_update`, read in subgraph-field-name space and never merged onto the failed instance's pre-failure state. This also fixes a latent bug: an instance `degraded_update`'s `extra_outputs` values were previously looked up by the parent field name and silently dropped (`collect_field` was unaffected). A static `degraded_update` that omits the node's `collect_field` is now a compile-time error (`FanOutDegradedUpdateMissingCollectField`); a callable `degraded_update` that omits it yields a graceful null slot rather than raising, preserving one collection slot per item. The parallel-branches counterpart (a branch `degraded_update` omitting a projected `outputs` field skips that field) was already correct as of the parallel-branches fix above and is now pinned by fixture 065. Success-path and resume behavior for correctly-configured fan-outs is unchanged.
2020
- **Failure-isolation events carry the full structured cause chain** (proposal 0068, pipeline-utilities §6.3, spec v0.57.0). `FailureIsolatedEvent.caught_exception` gains a `chain`: an ordered list of `CauseLink` records (each carrying `category`, `message`, and a `carrier` flag), from the caught exception (outermost) to the originating raise (innermost), with graph-engine `node_exception` carrier wrappers flagged `carrier=True`. The existing `category` and `message` are retained and redefined as a derivation over the chain: the category of the outermost non-carrier link whose category is a non-empty string (else `category` is `null` and `message` is the outermost non-carrier link's message). This supersedes proposal 0065's single "originating cause" representation, which was ambiguous once the post-carrier chain held more than one non-carrier link; the derivation reproduces 0065's single-carrier values, so fixture 064 is unchanged. A new `CauseLink` type is exported from `openarmature.graph`. The bundled OTel and Langfuse observers continue to render the derived `category`; surfacing the full chain is left to custom observers. The change is additive to the event shape, and catch/degrade behavior is unchanged. Conformance fixture 066 (three cases: an instance-site carrier chain, a node-level single non-carrier link, and an uncategorized null-category cause) passes.
21-
- **Pinned spec advances v0.53.0 → v0.58.0 across the v0.14.0 cycle**, in five steps: v0.54.0 (proposal 0059, the observer-flag rename above), v0.55.1 (proposal 0065 above; the v0.55.1 patch also carries an observability §11 span-links text reconciliation that narrows an *Out of scope* bullet, with no python-observable change), v0.56.0 (proposal 0066, the fan-out degrade contribution above), v0.57.0 (proposal 0068, the failure-isolation cause chain above), and v0.58.0 (proposal 0070, conformance-adapter crash-injection and cause-chaining test vocabulary: a `crash_injection` directive and a recursive mock `cause`, with conformance fixtures 067 and 068, no library behavior change). `conformance.toml` records 0065, 0066, 0068, and 0070 as `implemented` and 0059 as `not-yet` (only its cross-spec flag rename was adopted).
21+
- **Pinned spec advances v0.53.0 → v0.59.0 across the v0.14.0 cycle**, in six steps: v0.54.0 (proposal 0059, the observer-flag rename above), v0.55.1 (proposal 0065 above; the v0.55.1 patch also carries an observability §11 span-links text reconciliation that narrows an *Out of scope* bullet, with no python-observable change), v0.56.0 (proposal 0066, the fan-out degrade contribution above), v0.57.0 (proposal 0068, the failure-isolation cause chain above), v0.58.0 (proposal 0070, conformance-adapter crash-injection and cause-chaining test vocabulary: a `crash_injection` directive and a recursive mock `cause`, with conformance fixtures 067 and 068, no library behavior change), and v0.59.0 (proposal 0069, fan-out degrade contribution refinements to 0066: an omitted `extra_outputs` source is a positional null slot, an absent `collect_field` is a null slot the fan-in does not raise on except under a strict-element reducer, and a degraded slot survives resume; python already satisfied these, so the change is conformance coverage via fixture 069 plus a strict-reducer unit test, no library behavior change). `conformance.toml` records 0065, 0066, 0068, 0070, and 0069 as `implemented` and 0059 as `not-yet` (only its cross-spec flag rename was adopted).
2222

2323
### Fixed
2424

conformance.toml

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@
3232

3333
[manifest]
3434
implementation = "openarmature-python"
35-
spec_pin = "v0.58.0"
35+
spec_pin = "v0.59.0"
3636

3737
# Status values:
3838
# implemented — shipped behavior matches the proposal's contract
@@ -663,3 +663,19 @@ since = "0.14.0"
663663
[proposals."0070"]
664664
status = "implemented"
665665
since = "0.14.0"
666+
667+
# Spec v0.59.0 (proposal 0069). Fan-out degrade contribution refinements
668+
# (pipeline-utilities §9.3, refining 0066). Three refinements python already
669+
# satisfied: (1) an omitted ``extra_outputs`` source contributes NULL at the
670+
# instance's positional slot (index-aligned with target_field), not "not
671+
# contributed"; (2) an absent ``collect_field`` on any fan-in path is a null
672+
# slot and the fan-in MUST NOT raise -- with the caveat that under a strict-
673+
# element reducer (``concat_flatten`` / ``merge_all``) a null contribution
674+
# still raises ``ReducerError`` (python does not suppress it; the reducer runs
675+
# in the engine merge); (3) a degraded slot survives a checkpoint + resume
676+
# round-trip. No library behavior change. Fixture 069's FI-degrade cases run
677+
# in test_pipeline_utilities, its crash_injection/resume case in
678+
# test_checkpoint; the strict-reducer caveat has a focused unit test.
679+
[proposals."0069"]
680+
status = "implemented"
681+
since = "0.14.0"

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ Specification = "https://github.com/LunarCommand/openarmature-spec"
6363
openarmature = "openarmature.cli:main"
6464

6565
[tool.openarmature]
66-
spec_version = "0.58.0"
66+
spec_version = "0.59.0"
6767

6868
[dependency-groups]
6969
dev = [

src/openarmature/AGENTS.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# OpenArmature — Agent documentation
22

3-
*This is the agent guide bundled with the openarmature Python package, version 0.13.0 (spec v0.58.0). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*
3+
*This is the agent guide bundled with the openarmature Python package, version 0.13.0 (spec v0.59.0). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*
44

55
## TL;DR
66

@@ -10,7 +10,7 @@ OpenArmature is a workflow framework for LLM pipelines and tool-calling agents:
1010

1111
## Capability contracts
1212

13-
_Sourced from openarmature-spec v0.58.0. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md` verbatim — including additions from accepted proposals that this Python implementation may not yet ship. For per-proposal implementation status (implemented / partial / textual-only / not-yet), see the `conformance.toml` manifest at the repo root. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._
13+
_Sourced from openarmature-spec v0.59.0. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md` verbatim — including additions from accepted proposals that this Python implementation may not yet ship. For per-proposal implementation status (implemented / partial / textual-only / not-yet), see the `conformance.toml` manifest at the repo root. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._
1414

1515
### Capability: `graph-engine`
1616

src/openarmature/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
"""
2626

2727
__version__ = "0.13.0"
28-
__spec_version__ = "0.58.0"
28+
__spec_version__ = "0.59.0"
2929
# Proposal 0052 (spec observability §5.1 / §8.4.1): canonical
3030
# package-registry name for this implementation. Surfaces on every
3131
# OTel invocation span as ``openarmature.implementation.name`` and on

tests/conformance/test_checkpoint.py

Lines changed: 75 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
NodePosition,
5050
)
5151
from openarmature.graph import (
52+
FailureIsolationMiddleware,
5253
RuntimeGraphError,
5354
State,
5455
)
@@ -68,8 +69,11 @@
6869
# rather than relying on the test runner's file-glob to filter the
6970
# missing fixture out. 067 (crash-injection fan-out resume, proposal
7071
# 0070) is a crash/resume fixture this runner owns; it joined at v0.58.0.
72+
# 069 (fan-out degrade refinements, proposal 0069, v0.59.0) is a mixed
73+
# fixture: this runner drives its crash_injection/resume case and skips the
74+
# plain FI-degrade cases (owned by test_pipeline_utilities.py).
7175
_CHECKPOINT_FIXTURE_NUMBERS: frozenset[int] = frozenset(
72-
(set(range(24, 32)) - {28}) | set(range(48, 57)) | {67}
76+
(set(range(24, 32)) - {28}) | set(range(48, 57)) | {67, 69}
7377
)
7478

7579
# Fixtures that need resume-aware test seams the conformance adapter
@@ -277,12 +281,31 @@ async def test_checkpoint_fixture(fixture_path: Path) -> None:
277281
)
278282
spec = _load(fixture_path)
279283
if "cases" in spec:
284+
cases_run = 0
280285
for case in cast("list[dict[str, Any]]", spec["cases"]):
281286
case_name = case.get("name", "<unnamed>")
287+
# This runner drives the checkpoint cases. A mixed fixture (069)
288+
# interleaves plain FI-degrade cases owned by
289+
# test_pipeline_utilities.py; skip a case with no checkpoint
290+
# concern. The marker is checkpointer / resume / crash_injection —
291+
# NOT resume alone: fixtures like 024 / 026 / 030 / 055 assert
292+
# checkpoint behavior (saves, record shape, not-found,
293+
# schema_version) with a checkpointer but no resume.
294+
if not any(k in case for k in ("checkpointer", "resume", "crash_injection")):
295+
continue
296+
cases_run += 1
282297
try:
283298
await _run_one_case(case, top_level=spec)
284299
except AssertionError as e:
285300
raise AssertionError(f"case {case_name!r}: {e}") from e
301+
# A cases-shaped fixture in this runner's set that drives zero cases
302+
# (all skipped as non-checkpoint) would pass vacuously; fail loudly
303+
# instead so a routing mistake surfaces.
304+
assert cases_run > 0, (
305+
f"{fixture_id}: cases-shaped fixture drove zero cases in this runner "
306+
f"(all skipped as non-checkpoint). Fix the routing or remove it from "
307+
f"_CHECKPOINT_FIXTURE_NUMBERS."
308+
)
286309
return
287310
await _run_one_case(spec, top_level=spec)
288311

@@ -367,6 +390,56 @@ def _find_crash_injection(spec: Mapping[str, Any]) -> tuple[int | None, str | No
367390
return None, None, None
368391

369392

393+
def _translate_fi_instance_middleware(
394+
spec: Mapping[str, Any],
395+
) -> dict[str, list[FailureIsolationMiddleware]]:
396+
"""Translate a fan-out node's ``instance_middleware: [failure_isolation]``
397+
into FailureIsolationMiddleware instances keyed by node name, for
398+
build_graph's ``fan_out_instance_middleware``. Scoped to the static
399+
``degraded_update`` mapping form (the only shape the checkpoint fixtures
400+
use, e.g. fixture 069 Case 3's degrade-survives-resume); the callable
401+
forms are owned by test_pipeline_utilities.py, which drives the plain
402+
FI-degrade cases."""
403+
out: dict[str, list[FailureIsolationMiddleware]] = {}
404+
nodes = cast("dict[str, dict[str, Any]]", spec.get("nodes") or {})
405+
for node_name, node_spec in nodes.items():
406+
fan_out = node_spec.get("fan_out")
407+
if not isinstance(fan_out, dict):
408+
continue
409+
entries = cast(
410+
"list[dict[str, Any]]",
411+
cast("Mapping[str, Any]", fan_out).get("instance_middleware") or [],
412+
)
413+
mws: list[FailureIsolationMiddleware] = []
414+
for entry in entries:
415+
# Only failure_isolation is translated here. Other instance
416+
# middleware (e.g. fixture 053's retry) is left unwired, as this
417+
# runner did before — those fixtures drive their behavior via
418+
# flaky_per_index seams, not a wired middleware.
419+
if entry.get("type") != "failure_isolation":
420+
continue
421+
if "degraded_update" not in entry:
422+
raise ValueError(
423+
f"fan-out node {node_name!r}: failure_isolation instance middleware "
424+
f"entry is missing the required 'degraded_update'"
425+
)
426+
degraded = entry["degraded_update"]
427+
if not isinstance(degraded, dict):
428+
raise ValueError(
429+
f"fan-out node {node_name!r}: checkpoint runner supports only the static "
430+
f"degraded_update form for instance middleware"
431+
)
432+
mws.append(
433+
FailureIsolationMiddleware(
434+
degraded_update=dict(cast("Mapping[str, Any]", degraded)),
435+
event_name=entry.get("event_name", "degraded"),
436+
)
437+
)
438+
if mws:
439+
out[node_name] = mws
440+
return out
441+
442+
370443
def _strip_abort_directive(spec: Mapping[str, Any]) -> Mapping[str, Any]:
371444
"""Return a fresh spec dict with any ``abort_after_instance``
372445
directive removed from fan-out nodes. The engine doesn't recognize
@@ -421,6 +494,7 @@ async def _run_one_case(spec: Mapping[str, Any], *, top_level: Mapping[str, Any]
421494
trace=trace,
422495
flaky_per_index_attempt_recorders=flaky_per_index_recorders,
423496
instance_execution_recorders=instance_execution_recorders,
497+
fan_out_instance_middleware=_translate_fi_instance_middleware(sanitized_spec),
424498
)
425499
builder = built.builder
426500

tests/conformance/test_pipeline_utilities.py

Lines changed: 25 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -84,15 +84,16 @@ def _load(path: Path) -> dict[str, Any]:
8484
# the `cases:` shape carries seeded-record + migrations + resume blocks.
8585
_LAST_DRIVEN_FIXTURE = 38
8686

87-
# Failure-isolation fixtures (058-066 + 068, proposals 0050 §6.3 / 0065 /
88-
# 0066 / 0068 / 0070) are middleware fixtures this runner handles. They sit
89-
# past _LAST_DRIVEN_FIXTURE only because the 039-057 range (state migration /
90-
# checkpoint fan-out) is owned by dedicated runners (test_state_migration.py
91-
# / test_checkpoint.py), not because this runner can't drive them. Fixture
92-
# 066 (cause chain, 0068) joined at v0.57.0; 068 (failure-mock cause chain,
93-
# 0070) at v0.58.0. Fixture 067 (crash-injection fan-out resume) is a
94-
# checkpoint fixture owned by test_checkpoint.py, hence the gap at 67.
95-
_FAILURE_ISOLATION_FIXTURES = frozenset(range(58, 67)) | {68}
87+
# Failure-isolation fixtures (058-066, 068, 069, proposals 0050 §6.3 / 0065 /
88+
# 0066 / 0068 / 0070 / 0069) are middleware fixtures this runner handles. They
89+
# sit past _LAST_DRIVEN_FIXTURE only because the 039-057 range (state migration
90+
# / checkpoint fan-out) is owned by dedicated runners (test_state_migration.py
91+
# / test_checkpoint.py), not because this runner can't drive them. Fixture 066
92+
# (cause chain, 0068) joined at v0.57.0; 068 (failure-mock cause chain, 0070)
93+
# at v0.58.0; 069 (fan-out degrade refinements, 0069) at v0.59.0 — this runner
94+
# drives its FI-degrade cases and skips its crash_injection/resume case (owned
95+
# by test_checkpoint.py, which also owns fixture 067, hence the gap at 67).
96+
_FAILURE_ISOLATION_FIXTURES = frozenset(range(58, 67)) | {68, 69}
9697

9798

9899
def _fixture_paths() -> list[Path]:
@@ -541,8 +542,15 @@ async def test_pipeline_utility_fixture(
541542
shared_subgraph_blocks = {
542543
k: spec[k] for k in ("subgraph", "subgraph_with_idx", "subgraphs") if k in spec
543544
}
545+
cases_run = 0
544546
for case in spec["cases"]:
545547
case_name = case.get("name", "<unnamed>")
548+
# Checkpoint-concern cases (fixture 069 Case 3) are owned by
549+
# test_checkpoint.py; this runner skips them. The marker mirrors
550+
# that runner's: checkpointer / resume / crash_injection.
551+
if any(k in case for k in ("checkpointer", "resume", "crash_injection")):
552+
continue
553+
cases_run += 1
546554
merged: dict[str, Any] = dict(case)
547555
# Compile-error cases (065 Case 2) nest the graph under ``graph:``
548556
# (the graph-engine fixture 007 convention) so it sits beside
@@ -557,6 +565,14 @@ async def test_pipeline_utility_fixture(
557565
await _run_one(merged, monkeypatch)
558566
except AssertionError as e:
559567
raise AssertionError(f"case {case_name!r}: {e}") from e
568+
# A cases-shaped fixture in this runner's set that drives zero cases
569+
# (all skipped as checkpoint-owned) would pass vacuously; fail loudly
570+
# instead so a routing mistake surfaces.
571+
assert cases_run > 0, (
572+
f"{fixture_id}: cases-shaped fixture drove zero cases in this runner "
573+
f"(all skipped as checkpoint-owned). Fix the routing or remove it from "
574+
f"_FAILURE_ISOLATION_FIXTURES."
575+
)
560576
return
561577

562578
if (hit := _unsupported_directive(spec)) is not None:

tests/test_smoke.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
def test_package_versions() -> None:
1111
assert openarmature.__version__ == "0.13.0"
12-
assert openarmature.__spec_version__ == "0.58.0"
12+
assert openarmature.__spec_version__ == "0.59.0"
1313

1414

1515
def test_spec_version_matches_pyproject() -> None:

0 commit comments

Comments
 (0)