Skip to content

Commit 011e5b7

Browse files
Implement 0086 PromptManager default cache TTL (#227)
* Implement 0086 PromptManager default cache TTL PromptManager construction gains an optional default_cache_ttl_seconds (prompt-management proposal 0086). fetch and get resolve the effective cache_ttl_seconds by precedence: an explicit per-call value (including 0 force-fresh) wins; otherwise the manager default; otherwise None, so the backend's own caching governs. An omitted or explicit-None per-call value both select the default, so resolution is presence-independent. A negative default is rejected at construction; the per-call negative rejection is unchanged. get delegates to fetch and inherits the chain, and the bundled caching backends already honor a resolved value, so no backend changed. Un-defers conformance fixture 036 (runtime and parse), wiring the manager default-cache-ttl construction slot and the {manager: true} target form into the prompt-management harness. Adds seven unit tests, the concepts/prompts doc, and the 0.17.0 changelog entry. * Fix 0086 changelog em dash and stale ttl message Address CoPilot review on #227: - CHANGELOG 0.17.0 header uses a hyphen, not an em dash, per .github/copilot-instructions.md (new entries avoid the em dash; older headers are intentionally left as-is). - The fetch negative-ttl ValueError message no longer says "None preserves current behavior"; with a manager default configured, per-call None selects the default. Reworded to match the precedence. Comment/wording only; no behavior change.
1 parent 1a5f68a commit 011e5b7

8 files changed

Lines changed: 181 additions & 28 deletions

File tree

CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ All notable changes to `openarmature-python` are documented in this file.
44

55
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The package follows [Semantic Versioning](https://semver.org/); pre-1.0 minor bumps may carry behavioral changes per [spec governance](https://github.com/LunarCommand/openarmature-spec/blob/main/GOVERNANCE.md).
66

7+
## [0.17.0] - 2026-07-20
8+
9+
### Added
10+
11+
- **PromptManager service-wide default cache TTL** (proposal 0086, prompt-management §6, spec v0.79.0). `PromptManager` construction accepts an optional `default_cache_ttl_seconds`, applied to any `fetch` or `get` that omits a per-call `cache_ttl_seconds`. Resolution follows a precedence chain: an explicit per-call value (including `0` force-fresh) wins; otherwise the manager default applies; otherwise nothing is forwarded and the backend's own caching governs. An omitted or explicit-`None` per-call value both select the default, so resolution does not depend on argument presence. A negative default is rejected at construction, and the per-call negative rejection is unchanged. `get` delegates to `fetch` and inherits the chain, and the bundled caching backends need no change (they already honor a resolved `cache_ttl_seconds`). Conformance fixture 036 is un-deferred.
12+
713
## [0.16.0] — 2026-07-18
814

915
### Added

conformance.toml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -840,10 +840,12 @@ since = "0.16.0"
840840
note = "The SAVE-side enclosing_fan_out_lineage keying (pipeline-utilities §10.11) shipped in #194: a fan-out instance's checkpoint tracking key carries the enclosing fan-out instance lineage in the in-memory dict and through the checkpoint projection / lookup / cleanup / restore, so concurrent outer instances no longer collide. partial because the RESUME consume-side is not yet shipped: a fan-out nested inside an outer instance re-runs rather than skipping on resume, since the saved record format carries no lineage (tracked as a follow-up). pipeline-utilities fixture 076 is not collected by the test_pipeline_utilities.py _LAST_DRIVEN_FIXTURE number gate (it is not deferred); the resume consume-side plus its fixture wiring land in a later PR."
841841

842842
# Spec v0.79.0 (proposal 0086). Service-wide default cache_ttl_seconds
843-
# on PromptManager (prompt-management §6). Not-yet; prompt-management
844-
# fixture 036 defers with it.
843+
# on PromptManager (prompt-management §6). Implemented since 0.17.0;
844+
# prompt-management fixture 036 pins it.
845845
[proposals."0086"]
846-
status = "not-yet"
846+
status = "implemented"
847+
since = "0.17.0"
848+
note = "PromptManager construction gains an optional default_cache_ttl_seconds (prompt-management §6). fetch/get resolve cache_ttl_seconds by precedence: an explicit per-call value (including 0 force-fresh) wins; else the manager default; else None (the backend's own caching governs). An omitted or explicit-None per-call value both select the default, so resolution does not depend on argument-presence semantics. A negative default is rejected at construction; the per-call negative rejection from 0072 is unchanged and applies to the per-call value before resolution. get() delegates to fetch, so it inherits the chain unchanged; the bundled caching backends already honor a resolved cache_ttl_seconds (0072), so no backend change. Fixture 036 (default applied on omit + advance-clock aging + per-call 0 override) is un-deferred and driven via a manager: {default_cache_ttl_seconds} construction slot + target: {manager: true} fetches."
847849

848850
# Spec v0.82.0 (proposal 0087). Conformance-adapter within-node
849851
# directive execution order (§8.3). Implemented since 0.17.0;

docs/concepts/prompts.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,32 @@ recent = await manager.get(
7373
)
7474
```
7575

76+
### A service-wide default
77+
78+
If most fetches want the same freshness bound, set it once at construction
79+
with `default_cache_ttl_seconds` instead of passing `cache_ttl_seconds` on
80+
every call:
81+
82+
```python
83+
# `backend` is any backend with a client-side cache (e.g. the Langfuse backend);
84+
# a cacheless backend ignores the TTL regardless of where it comes from.
85+
manager = PromptManager(backend, default_cache_ttl_seconds=60)
86+
87+
# Uses the default (60s); no per-call value needed:
88+
prompt = await manager.fetch("greeting", "production")
89+
90+
# A per-call value always wins, so this still force-refreshes:
91+
fresh = await manager.fetch("greeting", "production", cache_ttl_seconds=0)
92+
```
93+
94+
Resolution follows a precedence chain: an explicit per-call value (including
95+
`0`) wins; otherwise the manager default applies; otherwise nothing is
96+
forwarded and the backend's own caching governs. A negative default is
97+
rejected at construction. Once a default is set, an omitted per-call value
98+
resolves to it, so there is no per-call way to defer to the backend's own
99+
behavior for a single fetch while a default is configured. Configure no
100+
default, or pass an explicit value, if you need that.
101+
76102
## Prompt identity
77103

78104
Every `Prompt` carries five identity fields:

src/openarmature/prompts/manager.py

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -76,12 +76,24 @@ def __init__(
7676
self,
7777
*backends: PromptBackend,
7878
label_resolver: LabelResolver | None = None,
79+
default_cache_ttl_seconds: int | None = None,
7980
jinja_undefined: type[jinja2.Undefined] = jinja2.StrictUndefined,
8081
) -> None:
8182
if not backends:
8283
raise ValueError("PromptManager requires at least one backend")
84+
# Proposal 0086 (prompt-management §6): a service-wide default
85+
# cache_ttl_seconds, applied to any fetch/get that omits a per-call
86+
# value. Same per-fetch semantics as the per-call lever (0 = force
87+
# fresh, N > 0 = bound staleness to N seconds); a negative default
88+
# is invalid and is rejected at construction.
89+
if default_cache_ttl_seconds is not None and default_cache_ttl_seconds < 0:
90+
raise ValueError(
91+
f"default_cache_ttl_seconds must be >= 0 (got {default_cache_ttl_seconds!r}); "
92+
"None leaves per-fetch cache control to the backend, 0 forces fresh reads"
93+
)
8394
self._backends: tuple[PromptBackend, ...] = backends
8495
self._label_resolver = label_resolver
96+
self._default_cache_ttl_seconds = default_cache_ttl_seconds
8597
# autoescape disabled by design: render output goes to an LLM
8698
# API call (plain text), not an HTML response. The env is
8799
# per-manager (was module-level) so jinja_undefined can be
@@ -127,21 +139,30 @@ async def fetch(
127139
failures, the manager raises ``PromptStoreUnavailable``.
128140
129141
``cache_ttl_seconds`` is a read-side cache control forwarded to
130-
each backend's ``fetch``: ``None`` keeps
131-
current behavior, ``0`` forces a fresh read, ``N > 0`` bounds a
132-
served entry's staleness to N seconds; a negative value is
133-
rejected. Cacheless backends ignore it.
142+
each backend's ``fetch``: ``0`` forces a fresh read, ``N > 0``
143+
bounds a served entry's staleness to N seconds, and a negative
144+
value is rejected. When omitted (or ``None``), the manager's
145+
``default_cache_ttl_seconds`` applies if one was configured;
146+
otherwise nothing is forwarded and the backend's own caching
147+
governs. An explicit per-call value always overrides the default.
148+
Cacheless backends ignore the resolved value.
134149
"""
135150
if cache_ttl_seconds is not None and cache_ttl_seconds < 0:
136151
raise ValueError(
137152
f"cache_ttl_seconds must be >= 0 (got {cache_ttl_seconds!r}); "
138-
"None preserves current behavior, 0 forces a fresh read"
153+
"None selects the manager default or the backend's own caching, "
154+
"0 forces a fresh read"
139155
)
156+
# Proposal 0086 precedence: an explicit per-call value (including
157+
# 0) wins; else the manager default; else None (the backend's own
158+
# caching governs). An omitted or explicit-None per-call value both
159+
# select the default, so resolution is presence-independent.
160+
resolved_ttl = cache_ttl_seconds if cache_ttl_seconds is not None else self._default_cache_ttl_seconds
140161
resolved_label = self._resolve_label(label, name)
141162
causes: list[BaseException] = []
142163
for backend in self._backends:
143164
try:
144-
return await backend.fetch(name, resolved_label, cache_ttl_seconds=cache_ttl_seconds)
165+
return await backend.fetch(name, resolved_label, cache_ttl_seconds=resolved_ttl)
145166
except PromptNotFound:
146167
raise
147168
except PromptStoreUnavailable as exc:

tests/conformance/harness/prompt_management.py

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,8 +74,12 @@ class FixtureLabelResolverSpec(_StrictModel):
7474

7575

7676
class FixtureManagerSpec(_StrictModel):
77-
backends: list[str]
77+
# backends is optional: fixture 036's manager block omits it and
78+
# defaults to all declared backends in order (proposal 0086).
79+
backends: list[str] | None = None
7880
label_resolver_ref: str | None = None
81+
# Proposal 0086: the manager's service-wide default cache_ttl_seconds.
82+
default_cache_ttl_seconds: int | None = None
7983

8084

8185
# ---------------------------------------------------------------------------
@@ -87,8 +91,17 @@ class BackendTarget(_StrictModel):
8791
backend: str
8892

8993

94+
class ManagerTarget(_StrictModel):
95+
# Proposal 0086: fixture 036 routes fetches through the manager via the
96+
# dict form ``target: {manager: true}`` (mirroring ``{backend: <name>}``),
97+
# alongside the bare-string ``manager`` the other fixtures use. Fixed to
98+
# ``true`` -- ``{manager: false}`` is nonsensical and rejected at parse.
99+
manager: Literal[True]
100+
101+
90102
CallTarget = (
91103
BackendTarget
104+
| ManagerTarget
92105
| Literal[
93106
"manager",
94107
"secondary_manager",

tests/conformance/test_fixture_parsing.py

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -636,11 +636,6 @@ def _id(case: tuple[str, Path]) -> str:
636636
"directive shapes the cross-cap parser does not model, "
637637
"runtime-driven in test_observability"
638638
),
639-
# Proposal 0086 (PromptManager default cache_ttl_seconds, v0.79.0) -- the
640-
# manager default-cache-ttl directive shape.
641-
"prompt-management/036-prompt-manager-default-cache-ttl": (
642-
"Proposal 0086 default cache_ttl_seconds; not implemented"
643-
),
644639
}
645640

646641

tests/conformance/test_prompt_management.py

Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -43,10 +43,12 @@
4343
from ._deferral import skip_if_deferred
4444
from .harness.loader import CONFORMANCE_ROOT
4545
from .harness.prompt_management import (
46+
BackendTarget,
4647
FixtureBackendSpec,
4748
FixtureCall,
4849
FixtureExpectedResultEquivalence,
4950
FixtureManagerSpec,
51+
ManagerTarget,
5052
PromptManagementFixture,
5153
)
5254

@@ -339,10 +341,13 @@ async def _run_call(
339341
members = [captures[ref] for ref in call.members_refs]
340342
return PromptGroup(group_name=call.group_name, members=members), None
341343

342-
if isinstance(target, str) and target in {"manager", "secondary_manager", "tertiary_manager"}:
343-
# All three manager targets dispatch to the currently-active
344-
# manager in the per-pair iteration loop. The naming exists
345-
# only to keep fixture YAML self-describing under a
344+
if isinstance(target, ManagerTarget) or (
345+
isinstance(target, str) and target in {"manager", "secondary_manager", "tertiary_manager"}
346+
):
347+
# The bare-string manager targets and the ``{manager: true}``
348+
# dict form (fixture 036, proposal 0086) both dispatch to the
349+
# currently-active manager in the per-pair iteration loop. The
350+
# string naming keeps fixture YAML self-describing under a
346351
# multi-manager shape (e.g., fixture 015).
347352
assert manager is not None
348353
if operation == "fetch":
@@ -392,7 +397,7 @@ async def _run_call(
392397
raise AssertionError(f"unsupported manager operation: {operation!r}")
393398

394399
# ``target: {backend: <name>}`` — direct backend op.
395-
assert not isinstance(target, str)
400+
assert isinstance(target, BackendTarget)
396401
backend = backends[target.backend]
397402
if operation == "fetch":
398403
assert call.name is not None and call.label is not None
@@ -528,13 +533,23 @@ def _build_manager(
528533
backends_map: dict[str, MockPromptBackend],
529534
resolvers_map: dict[str, MappingLabelResolver],
530535
) -> PromptManager:
531-
ordered = [backends_map[name] for name in spec.backends]
536+
# A manager spec that omits ``backends`` uses all declared backends in
537+
# declaration order (fixture 036's single-backend default-cache-ttl shape).
538+
ordered = (
539+
[backends_map[name] for name in spec.backends]
540+
if spec.backends is not None
541+
else list(backends_map.values())
542+
)
532543
resolver: MappingLabelResolver | None = None
533544
if spec.label_resolver_ref is not None:
534545
if spec.label_resolver_ref not in resolvers_map:
535546
raise AssertionError(f"unknown label_resolver_ref: {spec.label_resolver_ref!r}")
536547
resolver = resolvers_map[spec.label_resolver_ref]
537-
return PromptManager(*ordered, label_resolver=resolver)
548+
return PromptManager(
549+
*ordered,
550+
label_resolver=resolver,
551+
default_cache_ttl_seconds=spec.default_cache_ttl_seconds,
552+
)
538553

539554

540555
def _assert_capture_attrs(capture_name: str, actual: Any, expected: dict[str, Any]) -> None:
@@ -611,12 +626,6 @@ def _message_to_dict_for_compare(message: Message) -> dict[str, Any]:
611626
# accept, and asserts the construct-time prompt_group_invalid raise that
612627
# python does not yet implement. Defers until a later v0.16.0 PR.
613628
"035-prompt-group-arity-rejection": ("Proposal 0080 PromptGroup arity enforcement; not implemented"),
614-
# Proposal 0086 (PromptManager default cache_ttl_seconds, spec v0.79.0)
615-
# -- fixture 036 uses the manager default-cache-ttl construction slot
616-
# python does not yet implement. Defers until a later v0.16.0 PR.
617-
"036-prompt-manager-default-cache-ttl": (
618-
"Proposal 0086 PromptManager default cache_ttl_seconds; not implemented"
619-
),
620629
}
621630

622631

tests/unit/test_prompts.py

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,87 @@ async def fetch(
268268
await manager.fetch("greeting", "production", cache_ttl_seconds=-1)
269269

270270

271+
# ---------------------------------------------------------------------------
272+
# Proposal 0086: PromptManager service-wide default_cache_ttl_seconds
273+
# ---------------------------------------------------------------------------
274+
275+
276+
class _RecordingCacheTtlBackend:
277+
# Records the cache_ttl_seconds the manager forwarded on the last fetch,
278+
# so the resolution precedence can be asserted at the backend boundary.
279+
def __init__(self, prompt: Prompt) -> None:
280+
self._prompt = prompt
281+
self.last_cache_ttl_seconds: int | None = None
282+
283+
async def fetch(
284+
self, name: str, label: str = "production", *, cache_ttl_seconds: int | None = None
285+
) -> Prompt:
286+
self.last_cache_ttl_seconds = cache_ttl_seconds
287+
return self._prompt
288+
289+
290+
def test_construction_rejects_negative_default_cache_ttl() -> None:
291+
# A negative service-wide default is invalid and is rejected at
292+
# construction, before any fetch.
293+
backend = _RecordingCacheTtlBackend(_make_prompt())
294+
with pytest.raises(ValueError, match="default_cache_ttl_seconds must be >= 0"):
295+
PromptManager(backend, default_cache_ttl_seconds=-1)
296+
297+
298+
async def test_default_cache_ttl_applies_when_per_call_omitted() -> None:
299+
# Precedence step 2: an omitted per-call value resolves to the manager
300+
# default, which is forwarded to the backend verbatim.
301+
backend = _RecordingCacheTtlBackend(_make_prompt())
302+
manager = PromptManager(backend, default_cache_ttl_seconds=60)
303+
await manager.fetch("greeting", "production")
304+
assert backend.last_cache_ttl_seconds == 60
305+
306+
307+
async def test_explicit_none_selects_default_cache_ttl() -> None:
308+
# Resolution is presence-independent: an explicit None selects the
309+
# default exactly as an omitted argument does.
310+
backend = _RecordingCacheTtlBackend(_make_prompt())
311+
manager = PromptManager(backend, default_cache_ttl_seconds=60)
312+
await manager.fetch("greeting", "production", cache_ttl_seconds=None)
313+
assert backend.last_cache_ttl_seconds == 60
314+
315+
316+
async def test_per_call_zero_overrides_positive_default_cache_ttl() -> None:
317+
# Precedence step 1: an explicit per-call value wins, so a 0 force-fresh
318+
# overrides a positive manager default.
319+
backend = _RecordingCacheTtlBackend(_make_prompt())
320+
manager = PromptManager(backend, default_cache_ttl_seconds=60)
321+
await manager.fetch("greeting", "production", cache_ttl_seconds=0)
322+
assert backend.last_cache_ttl_seconds == 0
323+
324+
325+
async def test_no_default_forwards_none_cache_ttl() -> None:
326+
# Precedence step 3: with no manager default and no per-call value, None
327+
# is forwarded, so the backend's own caching governs (unchanged).
328+
backend = _RecordingCacheTtlBackend(_make_prompt())
329+
manager = PromptManager(backend)
330+
await manager.fetch("greeting", "production")
331+
assert backend.last_cache_ttl_seconds is None
332+
333+
334+
async def test_zero_default_cache_ttl_is_valid_and_forwarded() -> None:
335+
# A default of 0 is valid (force-fresh-always): accepted at construction,
336+
# and an omitted per-call value resolves to 0, forwarded to the backend.
337+
backend = _RecordingCacheTtlBackend(_make_prompt())
338+
manager = PromptManager(backend, default_cache_ttl_seconds=0)
339+
await manager.fetch("greeting", "production")
340+
assert backend.last_cache_ttl_seconds == 0
341+
342+
343+
async def test_get_inherits_default_cache_ttl() -> None:
344+
# get() delegates to fetch(), so the manager default resolves for get()
345+
# too when the per-call value is omitted.
346+
backend = _RecordingCacheTtlBackend(_make_prompt())
347+
manager = PromptManager(backend, default_cache_ttl_seconds=60)
348+
await manager.get("greeting", "production", {"user": "Alice"})
349+
assert backend.last_cache_ttl_seconds == 60
350+
351+
271352
# ---------------------------------------------------------------------------
272353
# FilesystemPromptBackend
273354
# ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)