Skip to content

Commit afbd47b

Browse files
Add prompt-management surface refinements (proposal 0033 + wishes 1, 5) (#79)
* Add prompt-management surface refinements Implements proposal 0033 (spec v0.26.0) plus python-side wishes 1 (FS flat layout) and 5 (Jinja-undefined opt-out). New fields on Prompt and PromptResult: - sampling: SamplingConfig | None, a RuntimeConfig subclass mirroring the seven declared fields plus extras. Splats directly into provider.complete(config=...) without translation. - observability_entities: dict[str, Any] | None, with the spec-normative key langfuse_prompt holding the Langfuse SDK Prompt reference (replaces the implementation-defined metadata key from proposal 0031's v0.23.0 placeholder). New LabelResolver primitive (openarmature.prompts.LabelResolver) with the spec §7 three-step fallback: per-name override > default override > spec fallback "production". The reference impl MappingLabelResolver is mapping-backed; the Protocol is open to JSON-file or remote-config implementations. PromptManager accepts label_resolver= and jinja_undefined= constructor kwargs; the render Environment is now per-instance to let the undefined-class knob bite without affecting other managers. FilesystemPromptBackend gains layout= (per-label default, flat opt-in) and sampling_source= (none default, per-prompt-sidecar reading <root>/<label>/<name>.config.json, or unified reading <root>/prompt_configs.json once at construction). A latent import cycle between openarmature.llm and openarmature.prompts surfaced once prompt.py imported RuntimeConfig from the llm package (for the SamplingConfig subclass). Deferred the current_prompt_group / current_prompt_result imports in openai.py to function-local; same behavior, no top-level re-entry. Spec submodule bumped to v0.26.0; conformance.toml grows entries for proposals 0033 (PR 2) and 0034 (PR 4), both not-yet pending the release PR. Fixture-parser defers prompt-management/015 and 016 (the PM-specific harness models the new shapes; the cross-capability parser doesn't) and observability/027-030 (PR 4 territory). Tests: four new prompt-management fixtures (013-016) plus six new unit tests covering the python-only ergonomics (jinja opt-out, flat layout, sidecar variants, LabelResolver precedence). Second of 6 PRs in the v0.10.0 batch. * Defensive validation and copy in prompt-management Two defensive fixes from CoPilot PR review on #79: - Unified-mode sampling source: validate each per-prompt entry in prompt_configs.json is a JSON object before calling _sampling_from_dict, raising a structured PromptStoreUnavailable on shape drift. Matches the symmetric top-level guard already in _load_unified_configs. Relaxed _unified_sampling's value type to Any so the runtime isinstance guard remains meaningful (the cast would have made it dead code). - PromptResult construction: shallow-copy prompt.sampling (model_copy) and prompt.observability_entities (dict(...)) so a caller mutating the result can't leak into the source Prompt or whatever instance the backend may be caching.
1 parent d23e3ab commit afbd47b

17 files changed

Lines changed: 791 additions & 85 deletions

File tree

conformance.toml

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929

3030
[manifest]
3131
implementation = "openarmature-python"
32-
spec_pin = "v0.24.0"
32+
spec_pin = "v0.26.0"
3333

3434
# Status values:
3535
# implemented — shipped behavior matches the proposal's contract
@@ -150,8 +150,8 @@ status = "textual-only"
150150
since = "0.9.0"
151151
note = "Drain snapshot semantic and timeout-input validation already implemented as part of the proposal 0010 impl PR (v0.9.0); no additional module-level work needed."
152152

153-
# Spec v0.23.0 + v0.24.0 batch (proposals 0031, 0032). Both proposals
154-
# have impl work landing across the v0.10.0 release cycle; status
153+
# Spec v0.23.0-v0.26.0 batch (proposals 0031, 0032, 0033, 0034). All
154+
# four have impl work landing across the v0.10.0 release cycle; status
155155
# stays `not-yet` until the release PR flips them to `implemented`
156156
# with `since = "0.10.0"`. The pinned spec submodule advances ahead
157157
# of the impl status because newer fixtures need to be visible to
@@ -161,3 +161,9 @@ status = "not-yet"
161161

162162
[proposals."0032"]
163163
status = "not-yet"
164+
165+
[proposals."0033"]
166+
status = "not-yet"
167+
168+
[proposals."0034"]
169+
status = "not-yet"

docs/concepts/prompts.md

Lines changed: 85 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,79 @@ a working-but-wrong prompt, often invisibly. If you need
100100
lenient behavior, wrap your variables in your own defaulting
101101
layer before passing them to `render()`.
102102

103-
The Python implementation uses Jinja2's `StrictUndefined`.
103+
The Python implementation uses Jinja2's `StrictUndefined`. To opt
104+
out, pass a different `Undefined` subclass at `PromptManager`
105+
construction:
106+
107+
```python
108+
import jinja2
109+
110+
manager = PromptManager(backend, jinja_undefined=jinja2.Undefined)
111+
```
112+
113+
`jinja2.Undefined` renders a missing variable as the empty string;
114+
`jinja2.ChainableUndefined` is the other common opt-out for
115+
templates that walk nested attributes. Reach for these only when the
116+
strict default is actively wrong for your workflow.
117+
118+
## Per-prompt sampling parameters
119+
120+
A `Prompt` carries an optional `sampling` field — a `SamplingConfig`
121+
sub-record mirroring `RuntimeConfig`'s seven declared fields
122+
(`temperature`, `max_tokens`, `top_p`, `seed`, `frequency_penalty`,
123+
`presence_penalty`, `stop_sequences`) plus the extras pass-through
124+
bag. Backends that source per-prompt config (Langfuse's
125+
`prompt.config`, a filesystem sidecar) populate it; backends that
126+
don't leave it `None`.
127+
128+
```python
129+
prompt = await manager.fetch("classify", "production")
130+
if prompt.sampling is not None:
131+
response = await provider.complete(messages, config=prompt.sampling)
132+
else:
133+
response = await provider.complete(messages)
134+
```
135+
136+
`SamplingConfig` is a subclass of `RuntimeConfig`, so it splats
137+
directly into `provider.complete()` without translation.
138+
`PromptResult.sampling` carries the value verbatim from the source
139+
`Prompt`; rendering doesn't touch it.
140+
141+
The `FilesystemPromptBackend` reads sidecar config when constructed
142+
with `sampling_source="per-prompt-sidecar"` (reading
143+
`<root>/<label>/<name>.config.json` next to each template) or
144+
`sampling_source="unified"` (reading `<root>/prompt_configs.json`
145+
once at construction, keyed by prompt name).
146+
147+
## Deployment-time label routing with `LabelResolver`
148+
149+
`PromptManager.fetch(name)` without an explicit `label` consults a
150+
configured `LabelResolver` and falls back to `"production"`. This
151+
lets one prompt be A/B-tested or canaried without code changes —
152+
edit the resolver's data, not the call sites.
153+
154+
```python
155+
from openarmature.prompts import MappingLabelResolver, PromptManager
156+
157+
resolver = MappingLabelResolver({
158+
"default": "production",
159+
"experimental_classifier": "staging",
160+
"extract_claims": "variant-a",
161+
})
162+
manager = PromptManager(backend, label_resolver=resolver)
163+
164+
# Resolver returns "staging" — staging template fetched.
165+
classify = await manager.fetch("experimental_classifier")
166+
# Resolver returns "production" (the default) — production fetched.
167+
greet = await manager.fetch("greet")
168+
# Explicit label bypasses the resolver entirely.
169+
audit = await manager.fetch("greet", "audit")
170+
```
171+
172+
`LabelResolver` is a Protocol with one method, `resolve(name) -> str`.
173+
The reference implementation is `MappingLabelResolver`, but any
174+
class with the right shape works (a JSON-file-backed resolver, a
175+
remote-config-service-backed resolver).
104176

105177
## Composite backends and fallback
106178

@@ -212,6 +284,18 @@ Nesting is innermost-wins. If you activate a result inside
212284
another active result, the inner one wins for the duration
213285
of the inner block.
214286

287+
### Backend-keyed observability entity references
288+
289+
A `Prompt` also carries an optional `observability_entities`
290+
mapping for backend-keyed references to first-class entities
291+
the prompt has been registered as in observability backends. The
292+
spec-normative key is `langfuse_prompt`, holding the Langfuse SDK
293+
`Prompt` reference. The Langfuse observer (when it ships) reads
294+
this field to establish the native Generation → Prompt link
295+
rather than reaching into the implementation-defined `metadata`
296+
mapping. Backends that don't surface such references leave the
297+
field `None`.
298+
215299
## Determinism and content-addressed caching
216300

217301
`render` is deterministic: same `Prompt`, same `variables`

openarmature-spec

Submodule openarmature-spec updated 28 files

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Specification = "https://github.com/LunarCommand/openarmature-spec"
5151
openarmature = "openarmature.cli:main"
5252

5353
[tool.openarmature]
54-
spec_version = "0.24.0"
54+
spec_version = "0.26.0"
5555

5656
[dependency-groups]
5757
dev = [

src/openarmature/AGENTS.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# OpenArmature — Agent documentation
22

3-
*This is the agent guide bundled with the openarmature Python package, version 0.9.0 (spec v0.24.0). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*
3+
*This is the agent guide bundled with the openarmature Python package, version 0.9.0 (spec v0.26.0). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*
44

55
## TL;DR
66

@@ -10,7 +10,7 @@ OpenArmature is a workflow framework for LLM pipelines and tool-calling agents
1010

1111
## Capability contracts
1212

13-
_Sourced from openarmature-spec v0.24.0. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md`. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._
13+
_Sourced from openarmature-spec v0.26.0. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md`. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._
1414

1515
### Capability: `graph-engine`
1616

src/openarmature/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,4 @@
2525
"""
2626

2727
__version__ = "0.9.0"
28-
__spec_version__ = "0.24.0"
28+
__spec_version__ = "0.26.0"

src/openarmature/llm/providers/openai.py

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,8 +59,13 @@
5959
current_namespace_prefix,
6060
)
6161
from openarmature.observability.llm_event import LlmEventPayload
62-
from openarmature.prompts.context import current_prompt_group, current_prompt_result
6362

63+
# ``current_prompt_group`` / ``current_prompt_result`` are imported
64+
# lazily inside :meth:`OpenAIProvider.complete` to avoid a module-load
65+
# cycle: ``openarmature.prompts.prompt`` imports ``RuntimeConfig`` from
66+
# this package (for the ``SamplingConfig`` subclass), so a top-level
67+
# import here would re-enter prompts.prompt before its types finish
68+
# defining.
6469
from ..errors import (
6570
LlmProviderError,
6671
ProviderAuthentication,
@@ -310,6 +315,12 @@ async def complete(
310315
# from inside the observer in the worker task returns ``None``
311316
# even when a node body opened a ``with_active_prompt`` block.
312317
# Snapshot here; the observer reads from the event payload.
318+
# Lazy import: see module-level comment for the cycle reason.
319+
from openarmature.prompts.context import (
320+
current_prompt_group,
321+
current_prompt_result,
322+
)
323+
313324
active_prompt = current_prompt_result()
314325
active_prompt_group = current_prompt_group()
315326
# Payload data the §5.5.1 / §5.5.2 / §5.5.3 attributes are

src/openarmature/prompts/__init__.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,19 @@
2020
)
2121
from .group import PromptGroup
2222
from .hashing import compute_rendered_hash, compute_template_hash
23+
from .label_resolver import SPEC_FALLBACK_LABEL, LabelResolver, MappingLabelResolver
2324
from .manager import PromptManager
24-
from .prompt import Prompt, PromptResult
25+
from .prompt import Prompt, PromptResult, SamplingConfig
2526

2627
__all__ = [
2728
"PROMPT_NOT_FOUND",
2829
"PROMPT_RENDER_ERROR",
2930
"PROMPT_STORE_UNAVAILABLE",
3031
"PROMPT_TRANSIENT_CATEGORIES",
32+
"SPEC_FALLBACK_LABEL",
3133
"FilesystemPromptBackend",
34+
"LabelResolver",
35+
"MappingLabelResolver",
3236
"Prompt",
3337
"PromptBackend",
3438
"PromptError",
@@ -38,6 +42,7 @@
3842
"PromptRenderError",
3943
"PromptResult",
4044
"PromptStoreUnavailable",
45+
"SamplingConfig",
4146
"compute_rendered_hash",
4247
"compute_template_hash",
4348
"current_prompt_group",

0 commit comments

Comments
 (0)