Skip to content

Commit 48032cb

Browse files
checkpoint: spec v0.8 checkpointing (proposal 0008) — phase 5
Land the checkpointing capability. Adds `openarmature.checkpoint` with the typed Checkpointer Protocol, CheckpointRecord/NodePosition/ CheckpointSummary shapes, the three §10.10 error categories, and two reference backends — InMemoryCheckpointer and SQLiteCheckpointer (WAL mode, upsert retention, pickle|json serialization knob). Engine integration: GraphBuilder.with_checkpointer registers a backend at compile time; CompiledGraph.invoke gains correlation_id and resume_invocation kwargs. Saves fire after every successful merge for outermost-graph and subgraph-internal nodes, gated off inside fan-out instances (§10.7 atomic-restart). Resume loads the record, restores outermost state from parent_states[0], threads any deeper saved states into the matching subgraph descents, and fast-forwards past nodes whose namespace matches saved completed_positions. Bumps the spec submodule to v0.8.2 — picks up fixture 019 (subgraph depth-3 regression) and the fixture 029 namespace convention fix. Test coverage: 6 of 8 §10 conformance fixtures pass; 027 and 028 are deferred (need resume-aware test seams in the conformance adapter; engine paths covered by unit tests). 23 unit tests cover backend round-trip + durability + serialization mismatch, error categories, schema_version round-trip, default-off behavior, the fan-out save gate, inner-node parent_states preservation, and correlation_id preservation across resume.
1 parent dfbcb02 commit 48032cb

15 files changed

Lines changed: 2073 additions & 26 deletions

File tree

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
"""openarmature.checkpoint — checkpointing capability per spec proposal 0008.
2+
3+
Public surface: the typed :class:`Checkpointer` Protocol,
4+
:class:`CheckpointRecord` / :class:`NodePosition` /
5+
:class:`CheckpointSummary` shapes, the three §10.10 error categories,
6+
and two reference backends (in-memory and SQLite).
7+
8+
Users register a backend at graph build time via
9+
``GraphBuilder.with_checkpointer(...)``; the engine then fires saves
10+
at every ``completed`` event for outermost-graph nodes and
11+
subgraph-internal nodes per §10.3, and ``invoke(resume_invocation=X)``
12+
loads + restores from a prior record per §10.4.
13+
"""
14+
15+
from .backends import InMemoryCheckpointer, SerializationMode, SQLiteCheckpointer
16+
from .errors import (
17+
CheckpointError,
18+
CheckpointNotFound,
19+
CheckpointRecordInvalid,
20+
CheckpointSaveFailed,
21+
)
22+
from .protocol import (
23+
CHECKPOINT_SCHEMA_VERSION,
24+
Checkpointer,
25+
CheckpointFilter,
26+
CheckpointRecord,
27+
CheckpointSummary,
28+
NodePosition,
29+
)
30+
31+
__all__ = [
32+
"CHECKPOINT_SCHEMA_VERSION",
33+
"CheckpointError",
34+
"CheckpointFilter",
35+
"CheckpointNotFound",
36+
"CheckpointRecord",
37+
"CheckpointRecordInvalid",
38+
"CheckpointSaveFailed",
39+
"CheckpointSummary",
40+
"Checkpointer",
41+
"InMemoryCheckpointer",
42+
"NodePosition",
43+
"SQLiteCheckpointer",
44+
"SerializationMode",
45+
]
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
"""Concrete Checkpointer backends.
2+
3+
Each backend satisfies the
4+
:class:`openarmature.checkpoint.Checkpointer` Protocol. The current
5+
catalog ships :class:`InMemoryCheckpointer` (no durability; tests +
6+
short-lived runs) and :class:`SQLiteCheckpointer` (single-host
7+
durable). Sibling-package adapters for Temporal, DBOS, Restate, and
8+
Redis are informative per spec §10.11 — not specified here.
9+
10+
Users typically import from the package root::
11+
12+
from openarmature.checkpoint import InMemoryCheckpointer, SQLiteCheckpointer
13+
"""
14+
15+
from .memory import InMemoryCheckpointer
16+
from .sqlite import SerializationMode, SQLiteCheckpointer
17+
18+
__all__ = [
19+
"InMemoryCheckpointer",
20+
"SQLiteCheckpointer",
21+
"SerializationMode",
22+
]
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
"""In-memory Checkpointer (spec §10.11).
2+
3+
Keeps records in a Python ``dict`` keyed by ``invocation_id``. NOT
4+
durable across process crashes — useful for tests, short-lived runs,
5+
and development. Accepts any state shape (the dict holds the
6+
:class:`CheckpointRecord` directly; nothing is serialized).
7+
"""
8+
9+
from __future__ import annotations
10+
11+
import asyncio
12+
from collections.abc import Iterable
13+
14+
from ..protocol import CheckpointFilter, CheckpointRecord, CheckpointSummary
15+
16+
17+
class InMemoryCheckpointer:
18+
"""Dict-backed Checkpointer.
19+
20+
**Durability:** none. Records live for the lifetime of this
21+
instance only; restarting the process loses everything.
22+
Appropriate for unit tests, the dev loop, and short-lived
23+
in-process pipelines that don't need crash recovery.
24+
25+
**State shape:** any. The record is held by reference, so the
26+
Pydantic state instance the engine produces is what comes back
27+
from :meth:`load` — no serialization round-trip. (This is the
28+
feature: tests can assert on the saved state's identity.)
29+
"""
30+
31+
def __init__(self) -> None:
32+
self._records: dict[str, CheckpointRecord] = {}
33+
self._lock = asyncio.Lock()
34+
35+
async def save(self, invocation_id: str, record: CheckpointRecord) -> None:
36+
async with self._lock:
37+
self._records[invocation_id] = record
38+
39+
async def load(self, invocation_id: str) -> CheckpointRecord | None:
40+
async with self._lock:
41+
return self._records.get(invocation_id)
42+
43+
async def list(self, filter: CheckpointFilter | None = None) -> Iterable[CheckpointSummary]:
44+
async with self._lock:
45+
records = list(self._records.values())
46+
summaries = [
47+
CheckpointSummary(
48+
invocation_id=r.invocation_id,
49+
correlation_id=r.correlation_id,
50+
last_saved_at=r.last_saved_at,
51+
completed_node_count=len(r.completed_positions),
52+
)
53+
for r in records
54+
]
55+
if filter is None or filter.correlation_id is None:
56+
return summaries
57+
return [s for s in summaries if s.correlation_id == filter.correlation_id]
58+
59+
async def delete(self, invocation_id: str) -> None:
60+
async with self._lock:
61+
self._records.pop(invocation_id, None)
62+
63+
64+
__all__ = ["InMemoryCheckpointer"]

0 commit comments

Comments
 (0)