Skip to content

Commit 91a8c2f

Browse files
Implement 0080 PromptGroup arity enforcement (#228)
* Implement 0080 PromptGroup arity enforcement PromptGroup already rejected fewer than two members, but with a bare ValueError that pydantic folds into a ValidationError carrying no error category, so the conformance harness had nothing to assert against. Add a categorized PromptGroupInvalid (new prompt_group_invalid category) and raise it from the arity validator. Because PromptGroupInvalid is a PromptError and not a ValueError, pydantic propagates it unwrapped from the model validator, so the caller sees .category. It is non-transient and exported from openarmature.prompts. Wire the rejection fixture 035: make PromptManagementFixture.backends optional (035 is cases-only with per-case backends, one case needing none) and rework the cases-runner to build per-case backends when a case declares its own backends key (else share the top-level, so 016 is unchanged) and to run a manager-less case's direct-target calls instead of skipping them (the empty-group construct needs no manager). Update the two arity unit tests, the concepts/prompts doc, and the 0.17.0 changelog entry. * Assert PROMPT_GROUP_INVALID constant in category test Address CoPilot review on #228: test_error_categories_match_spec pinned PromptGroupInvalid.category but not the exported PROMPT_GROUP_INVALID constant, so a regression in the constant value or export could slip through. Add the constant assertion (and its import) alongside the other three categories. Test-only; no behavior change.
1 parent 011e5b7 commit 91a8c2f

9 files changed

Lines changed: 76 additions & 24 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The
99
### Added
1010

1111
- **PromptManager service-wide default cache TTL** (proposal 0086, prompt-management §6, spec v0.79.0). `PromptManager` construction accepts an optional `default_cache_ttl_seconds`, applied to any `fetch` or `get` that omits a per-call `cache_ttl_seconds`. Resolution follows a precedence chain: an explicit per-call value (including `0` force-fresh) wins; otherwise the manager default applies; otherwise nothing is forwarded and the backend's own caching governs. An omitted or explicit-`None` per-call value both select the default, so resolution does not depend on argument presence. A negative default is rejected at construction, and the per-call negative rejection is unchanged. `get` delegates to `fetch` and inherits the chain, and the bundled caching backends need no change (they already honor a resolved `cache_ttl_seconds`). Conformance fixture 036 is un-deferred.
12+
- **PromptGroup arity enforcement** (proposal 0080, prompt-management §10 / §11, spec v0.75.0). Constructing a `PromptGroup` with fewer than two members (an empty or single-member group) now raises a categorized `PromptGroupInvalid` at construction, before any render or LLM call. This replaces the previous bare `ValueError` (which pydantic folded into a `ValidationError` carrying no error category) and adds a `prompt_group_invalid` category to the prompt-management error set. `PromptGroupInvalid` is non-transient and is exported from `openarmature.prompts`. Conformance fixture 035 is un-deferred.
1213

1314
## [0.16.0] — 2026-07-18
1415

conformance.toml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -795,10 +795,12 @@ note = "Retrieval-provider OpenAI-compatible embeddings wire mapping (§8 / §8.
795795

796796
# Spec v0.75.0 (proposal 0080). PromptGroup arity enforcement
797797
# (prompt-management §10 / §11 -- construct-time raise plus the new
798-
# prompt_group_invalid error category). Not-yet; prompt-management
799-
# fixture 035 defers with it.
798+
# prompt_group_invalid error category). Implemented since 0.17.0;
799+
# prompt-management fixture 035 pins it.
800800
[proposals."0080"]
801-
status = "not-yet"
801+
status = "implemented"
802+
since = "0.17.0"
803+
note = "PromptGroup already rejected fewer than two members, but as a bare ValueError (pydantic folds it into a ValidationError, which has no category). Now raises a categorized PromptGroupInvalid (new category prompt_group_invalid, prompt-management §11) at construction: PromptGroupInvalid is a PromptError (not a ValueError), so pydantic propagates it unwrapped from the model validator and the caller sees .category. Non-transient, not in PROMPT_TRANSIENT_CATEGORIES; exported from openarmature.prompts. Fixture 035 (single-member + empty group, each asserting raises.category = prompt_group_invalid) is un-deferred. Harness: PromptManagementFixture.backends is now optional (035 is cases-only with per-case backends, one case needing none) and the cases-runner builds per-case backends when a case declares its own backends key (else shares the top-level, preserving 016) and runs a manager-less case's direct-target calls instead of skipping them (035's empty-group construct_prompt_group needs no manager)."
802804

803805
# Spec v0.76.0 (proposal 0081). Conformance-adapter value-matcher
804806
# vocabulary (§5.10) -- ratifies the fixture matcher tokens already in

docs/concepts/prompts.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -329,10 +329,11 @@ Canonical patterns the primitive covers:
329329
- **Map-reduce over chunks**: `[chunk_classify_1..N, synthesize]`.
330330

331331
The N=2 case ("classifier + follow-up") is the simplest;
332-
larger groups work under the same primitive. The group rejects
333-
empty and single-member shapes; single-prompt tagging is
334-
already served by the per-prompt observability attributes
335-
below.
332+
larger groups work under the same primitive. Constructing a
333+
group with fewer than two members raises `PromptGroupInvalid`
334+
at construction time, before any render or call; single-prompt
335+
tagging is already served by the per-prompt observability
336+
attributes below.
336337

337338
## Observability propagation
338339

src/openarmature/prompts/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,13 @@
99
with_active_prompt_group,
1010
)
1111
from .errors import (
12+
PROMPT_GROUP_INVALID,
1213
PROMPT_NOT_FOUND,
1314
PROMPT_RENDER_ERROR,
1415
PROMPT_STORE_UNAVAILABLE,
1516
PROMPT_TRANSIENT_CATEGORIES,
1617
PromptError,
18+
PromptGroupInvalid,
1719
PromptNotFound,
1820
PromptRenderError,
1921
PromptStoreUnavailable,
@@ -38,6 +40,7 @@
3840
)
3941

4042
__all__ = [
43+
"PROMPT_GROUP_INVALID",
4144
"PROMPT_NOT_FOUND",
4245
"PROMPT_RENDER_ERROR",
4346
"PROMPT_STORE_UNAVAILABLE",
@@ -57,6 +60,7 @@
5760
"PromptBackend",
5861
"PromptError",
5962
"PromptGroup",
63+
"PromptGroupInvalid",
6064
"PromptManager",
6165
"PromptNotFound",
6266
"PromptRenderError",

src/openarmature/prompts/errors.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
PROMPT_NOT_FOUND = "prompt_not_found"
88
PROMPT_RENDER_ERROR = "prompt_render_error"
99
PROMPT_STORE_UNAVAILABLE = "prompt_store_unavailable"
10+
PROMPT_GROUP_INVALID = "prompt_group_invalid"
1011

1112
# Mirrors openarmature.llm.errors.TRANSIENT_CATEGORIES. Retry-middleware
1213
# classifiers MAY import this to identify transient prompt-management
@@ -123,3 +124,17 @@ def __init__(
123124
self.label = label
124125
self.backends_tried = backends_tried
125126
self.causes = causes
127+
128+
129+
class PromptGroupInvalid(PromptError):
130+
"""Raised when a ``PromptGroup`` construction violates a
131+
group-validity rule. Currently raised when ``members`` contains
132+
fewer than two elements (an empty or single-member group). Raised
133+
at construction time, before any render or LLM call, so an invalid
134+
group never reaches observability emission.
135+
136+
Non-transient: a caller-contract violation. Constructing again with
137+
the same members will not succeed without changing them.
138+
"""
139+
140+
category = PROMPT_GROUP_INVALID

src/openarmature/prompts/group.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
from pydantic import BaseModel, ConfigDict, model_validator
66

7+
from .errors import PromptGroupInvalid
78
from .prompt import PromptResult
89

910

@@ -31,6 +32,10 @@ class PromptGroup(BaseModel):
3132

3233
@model_validator(mode="after")
3334
def _check_min_two_members(self) -> PromptGroup:
35+
# PromptGroupInvalid is not a ValueError, so pydantic propagates it
36+
# unwrapped from the validator (only ValueError / AssertionError get
37+
# folded into a ValidationError) -- the categorized error reaches the
38+
# caller with its ``category``.
3439
if len(self.members) < 2:
35-
raise ValueError("prompt group: members MUST contain at least two PromptResult instances")
40+
raise PromptGroupInvalid("prompt group: members MUST contain at least two PromptResult instances")
3641
return self

tests/conformance/harness/prompt_management.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,10 @@ class FixtureExpectedTopLevel(_PermissiveModel):
235235

236236

237237
class PromptManagementFixture(_StrictModel):
238-
backends: list[FixtureBackendSpec]
238+
# Optional: fixture 035 is cases-only with per-case backends (and an
239+
# empty-group case that needs no backend at all), so the top level
240+
# declares none (proposal 0080).
241+
backends: list[FixtureBackendSpec] = []
239242
# Fixture 016 uses a top-level ``cases:`` list to split into
240243
# independent sub-cases that share the backends declaration but
241244
# each have their own manager + calls. The runner walks the list

tests/conformance/test_prompt_management.py

Lines changed: 24 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -620,12 +620,6 @@ def _message_to_dict_for_compare(message: Message) -> dict[str, Any]:
620620
"032-cross-variable-substring-stability": (
621621
"Proposal 0047 wire-byte stability (expected_shared_prefix directive); queued for v0.13.0"
622622
),
623-
# ----- v0.16.0 spec-pin bump (v0.70.1 -> v0.84.0) -------------------
624-
# Proposal 0080 (PromptGroup arity enforcement, spec v0.75.0) -- fixture
625-
# 035 uses a cases-only shape (no backends) the PM fixture model doesn't
626-
# accept, and asserts the construct-time prompt_group_invalid raise that
627-
# python does not yet implement. Defers until a later v0.16.0 PR.
628-
"035-prompt-group-arity-rejection": ("Proposal 0080 PromptGroup arity enforcement; not implemented"),
629623
}
630624

631625

@@ -671,8 +665,11 @@ async def test_prompt_management_fixture(fixture_path: Path) -> None:
671665
if call.capture_as is not None and raised is None:
672666
captures[call.capture_as] = result
673667

674-
# Cases-form fixtures (016) split into independent sub-cases that
675-
# share the backends but use their own per-case manager + calls.
668+
# Cases-form fixtures split into independent sub-cases sharing the
669+
# captures dict. Fixture 016 shares the top-level backends across cases;
670+
# fixture 035 (proposal 0080) declares backends per case (one case needs
671+
# none), so a case that brings its own ``backends`` key gets fresh
672+
# backends and any other shares the top-level set.
676673
cases = raw.get("cases")
677674
if cases:
678675
for case in cases:
@@ -684,18 +681,32 @@ async def test_prompt_management_fixture(fixture_path: Path) -> None:
684681
**{k: v for k, v in case.items() if k not in {"name", "description"}},
685682
}
686683
case_fixture = PromptManagementFixture.model_validate(case_payload)
684+
# A case that declares its own ``backends`` REPLACES the top-level
685+
# set (not a union) -- the payload merge already gives case keys
686+
# override semantics. No current fixture needs a case to add to the
687+
# shared backends; if one ever does, merge here instead.
688+
case_backends = (
689+
{spec.name: MockPromptBackend(spec) for spec in case_fixture.backends}
690+
if "backends" in case
691+
else backends
692+
)
687693
case_manager_pairs = [
688694
(case_fixture.manager, case_fixture.calls),
689695
(case_fixture.secondary_manager, case_fixture.secondary_calls),
690696
(case_fixture.tertiary_manager, case_fixture.tertiary_calls),
691697
]
692698
for manager_spec, manager_calls in case_manager_pairs:
693-
if manager_spec is None:
694-
continue
695-
manager = _build_manager(manager_spec, backends, resolvers_map)
699+
# Build a manager only when one is declared; a manager-less
700+
# case still runs its direct-target calls (035's empty-group
701+
# construct_prompt_group needs no manager).
702+
manager = (
703+
_build_manager(manager_spec, case_backends, resolvers_map)
704+
if manager_spec is not None
705+
else None
706+
)
696707
for call in manager_calls:
697-
result, raised = await _run_call(call, backends, manager, captures)
698-
_assert_per_call(call, result, raised, backends)
708+
result, raised = await _run_call(call, case_backends, manager, captures)
709+
_assert_per_call(call, result, raised, case_backends)
699710
if call.capture_as is not None and raised is None:
700711
captures[call.capture_as] = result
701712
if case_fixture.expected is not None:

tests/unit/test_prompts.py

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020

2121
from openarmature.llm.messages import Message, UserMessage
2222
from openarmature.prompts import (
23+
PROMPT_GROUP_INVALID,
2324
PROMPT_NOT_FOUND,
2425
PROMPT_RENDER_ERROR,
2526
PROMPT_STORE_UNAVAILABLE,
@@ -28,6 +29,7 @@
2829
Prompt,
2930
PromptError,
3031
PromptGroup,
32+
PromptGroupInvalid,
3133
PromptManager,
3234
PromptNotFound,
3335
PromptRenderError,
@@ -51,9 +53,11 @@ def test_error_categories_match_spec() -> None:
5153
assert PromptNotFound.category == "prompt_not_found"
5254
assert PromptRenderError.category == "prompt_render_error"
5355
assert PromptStoreUnavailable.category == "prompt_store_unavailable"
56+
assert PromptGroupInvalid.category == "prompt_group_invalid"
5457
assert PROMPT_NOT_FOUND == "prompt_not_found"
5558
assert PROMPT_RENDER_ERROR == "prompt_render_error"
5659
assert PROMPT_STORE_UNAVAILABLE == "prompt_store_unavailable"
60+
assert PROMPT_GROUP_INVALID == "prompt_group_invalid"
5761

5862

5963
def test_transient_categories_contains_only_store_unavailable() -> None:
@@ -167,8 +171,13 @@ def test_prompt_result_rejects_empty_messages() -> None:
167171

168172

169173
def test_prompt_group_rejects_zero_members() -> None:
170-
with pytest.raises(ValueError, match="at least two"):
174+
# Categorized prompt_group_invalid (proposal 0080), not a bare ValueError:
175+
# PromptGroupInvalid is not a ValueError, so pydantic propagates it
176+
# unwrapped from the model validator rather than folding it into a
177+
# ValidationError.
178+
with pytest.raises(PromptGroupInvalid, match="at least two") as exc_info:
171179
PromptGroup(group_name="g", members=[])
180+
assert exc_info.value.category == "prompt_group_invalid"
172181

173182

174183
def test_prompt_group_rejects_one_member() -> None:
@@ -184,8 +193,9 @@ def test_prompt_group_rejects_one_member() -> None:
184193
fetched_at=prompt.fetched_at,
185194
rendered_at=datetime.now(UTC),
186195
)
187-
with pytest.raises(ValueError, match="at least two"):
196+
with pytest.raises(PromptGroupInvalid, match="at least two") as exc_info:
188197
PromptGroup(group_name="g", members=[pr])
198+
assert exc_info.value.category == "prompt_group_invalid"
189199

190200

191201
def test_prompt_group_accepts_two_or_more_members() -> None:

0 commit comments

Comments
 (0)