Skip to content

Commit d23e3ab

Browse files
Add RuntimeConfig declared fields + null-skip (proposal 0032) (#78)
* Add RuntimeConfig declared fields + null-skip Implements proposal 0032 (spec v0.24.0). Three OpenAI-standard sampling parameters promoted from the extras pass-through to declared fields on RuntimeConfig: frequency_penalty, presence_penalty, and stop_sequences. The declared name stop_sequences matches the OpenTelemetry GenAI semconv (and Anthropic / Gemini wire-key conventions); the OpenAI-compatible wire mapping translates it to OpenAI's shorter body key stop on emission, per spec §8.1. Null-skip is now uniform across all seven declared fields: a field set to None is omitted from the wire body entirely (no JSON null). The existing four fields already followed this pattern; the three new fields extend it. OTel observer emits gen_ai.request.frequency_penalty, gen_ai.request.presence_penalty, and gen_ai.request.stop_sequences on the LLM span when the corresponding RuntimeConfig field is set (per observability §5.5.2's expanded attribute list). RuntimeConfig.from_partial(**kwargs) classmethod is a pure Python ergonomic that drops None-valued kwargs before construction. The wire layer already null-skips, so this is convenience only. Spec submodule bumped to v0.24.0; conformance.toml grows entries for proposals 0031 (langfuse mapping, not-yet) and 0032 (not-yet here, flips to implemented with since=0.10.0 in the release PR). Tests: fixture llm-provider/032 and observability/025 pass. Langfuse fixtures 022-024 deferred in the parser harness until PR 3 ships the matching directive model. * Strengthen from_partial drop-Nones test The is-None assertions on max_tokens and seed weren't load-bearing: both fields default to None on a base RuntimeConfig, so the test would have passed even if from_partial forwarded the None kwargs. Added model_fields_set assertions, which carry only fields the caller explicitly set during construction, to prove the drop. Addresses CoPilot PR review feedback on #78.
1 parent fcb3718 commit d23e3ab

15 files changed

Lines changed: 190 additions & 26 deletions

File tree

conformance.toml

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929

3030
[manifest]
3131
implementation = "openarmature-python"
32-
spec_pin = "v0.22.1"
32+
spec_pin = "v0.24.0"
3333

3434
# Status values:
3535
# implemented — shipped behavior matches the proposal's contract
@@ -149,3 +149,15 @@ since = "0.9.0"
149149
status = "textual-only"
150150
since = "0.9.0"
151151
note = "Drain snapshot semantic and timeout-input validation already implemented as part of the proposal 0010 impl PR (v0.9.0); no additional module-level work needed."
152+
153+
# Spec v0.23.0 + v0.24.0 batch (proposals 0031, 0032). Both proposals
154+
# have impl work landing across the v0.10.0 release cycle; status
155+
# stays `not-yet` until the release PR flips them to `implemented`
156+
# with `since = "0.10.0"`. The pinned spec submodule advances ahead
157+
# of the impl status because newer fixtures need to be visible to
158+
# the conformance harness as each PR lands.
159+
[proposals."0031"]
160+
status = "not-yet"
161+
162+
[proposals."0032"]
163+
status = "not-yet"

docs/concepts/observability.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -396,8 +396,9 @@ tools). Emitted alongside the OA namespace:
396396
- `gen_ai.request.model` / `gen_ai.response.model` — the bound
397397
model and (when the provider returns one) the more-specific
398398
identifier in the response body.
399-
- `gen_ai.request.temperature` / `max_tokens` / `top_p` / `seed`
400-
— only emitted for fields the caller actually set; absence on
399+
- `gen_ai.request.temperature` / `max_tokens` / `top_p` / `seed` /
400+
`frequency_penalty` / `presence_penalty` / `stop_sequences`
401+
only emitted for fields the caller actually set; absence on
401402
the span means "not supplied," distinct from a zero value.
402403
- `gen_ai.usage.input_tokens` / `output_tokens` — token counts.
403404
- `gen_ai.response.finish_reasons` — single-element string array.
@@ -429,7 +430,8 @@ This surfaces three attributes:
429430
with empty content.
430431
- `openarmature.llm.request.extras` — JSON-encoded `RuntimeConfig`
431432
extras bag (provider-specific pass-through fields like
432-
`frequency_penalty`). Omitted when empty.
433+
`repetition_penalty` for vLLM, or `top_k` for HuggingFace
434+
endpoints). Omitted when empty.
433435

434436
**Default-off is deliberate.** The payload may contain PII the user
435437
hasn't audited; opting in is a separate decision from opting into

openarmature-spec

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Specification = "https://github.com/LunarCommand/openarmature-spec"
5151
openarmature = "openarmature.cli:main"
5252

5353
[tool.openarmature]
54-
spec_version = "0.22.1"
54+
spec_version = "0.24.0"
5555

5656
[dependency-groups]
5757
dev = [

src/openarmature/AGENTS.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# OpenArmature — Agent documentation
22

3-
*This is the agent guide bundled with the openarmature Python package, version 0.9.0 (spec v0.22.1). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*
3+
*This is the agent guide bundled with the openarmature Python package, version 0.9.0 (spec v0.24.0). For the full docs site see [openarmature.ai](https://openarmature.ai). For the canonical spec text see [openarmature.org/capabilities](https://openarmature.org/capabilities/). For project-specific conventions for the code you're editing, see the host project's `AGENTS.md` or `CLAUDE.md`.*
44

55
## TL;DR
66

@@ -10,7 +10,7 @@ OpenArmature is a workflow framework for LLM pipelines and tool-calling agents
1010

1111
## Capability contracts
1212

13-
_Sourced from openarmature-spec v0.22.1. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md`. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._
13+
_Sourced from openarmature-spec v0.24.0. Each entry below reproduces §1 (Purpose) and §2 (Concepts) of the capability's `spec.md`. For the full spec text (execution model, error semantics, determinism, observer hooks, etc.) see the linked docs site._
1414

1515
### Capability: `graph-engine`
1616

@@ -274,13 +274,13 @@ The observability capability defines normative mappings from OpenArmature's runt
274274
well-known external observability backends. The substrate is provider-neutral; the capability is
275275
where each concrete backend's translation lives.
276276

277-
This first version specifies the **OpenTelemetry** mapping. Future proposals add other backends
278-
(Langfuse, etc.) as sibling sections of this same spec; the OTel mapping serves as the reference
279-
shape for cross-backend equivalence.
277+
This spec defines two concrete backend mappings: the **OpenTelemetry** mapping in §3–§7 and the
278+
**Langfuse** mapping in §8. Future proposals add additional backends as further sibling sections
279+
of this same spec; the OTel mapping serves as the reference shape for cross-backend equivalence.
280280

281281
The capability does NOT introduce new graph-engine primitives. It consumes the existing observer
282282
event stream — `started` events open spans, `completed` events close them. An implementation that
283-
emits OTel spans is built on top of §6, not into the engine.
283+
emits OTel spans (or Langfuse observations, per §8) is built on top of §6, not into the engine.
284284

285285
#### 2. Concepts
286286

@@ -310,7 +310,8 @@ and is intended to be visible in every backend the implementation emits to. A us
310310
LLM workflow with both an OTel backend (system traces, logs) and a Langfuse backend
311311
(LLM-specific traces) uses the `correlation_id` as a join key between them: find a slow request
312312
in Langfuse, search for its `correlation_id` in OTel logs, and see the surrounding
313-
infrastructure activity. See §3 (architectural contract) and §5.6 (OTel attribute realization).
313+
infrastructure activity. See §3 (architectural contract), §5.6 (OTel attribute realization),
314+
and §8.5 (Langfuse attribute realization).
314315

315316
### Capability: `prompt-management`
316317

src/openarmature/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,4 @@
2525
"""
2626

2727
__version__ = "0.9.0"
28-
__spec_version__ = "0.22.1"
28+
__spec_version__ = "0.24.0"

src/openarmature/llm/providers/openai.py

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -423,6 +423,11 @@ def _build_request_body(
423423
if tools:
424424
body["tools"] = [_tool_to_wire(t) for t in tools]
425425
if config is not None:
426+
# Per spec §6 null-skip: each declared field with value
427+
# ``None`` is omitted from the wire body. Same-name keys
428+
# for the pre-0032 four; same-name for frequency_penalty /
429+
# presence_penalty per §8.1; ``stop_sequences`` renames to
430+
# OpenAI's body key ``stop`` per §8.1's only rename.
426431
if config.temperature is not None:
427432
body["temperature"] = config.temperature
428433
if config.max_tokens is not None:
@@ -431,9 +436,15 @@ def _build_request_body(
431436
body["top_p"] = config.top_p
432437
if config.seed is not None:
433438
body["seed"] = config.seed
439+
if config.frequency_penalty is not None:
440+
body["frequency_penalty"] = config.frequency_penalty
441+
if config.presence_penalty is not None:
442+
body["presence_penalty"] = config.presence_penalty
443+
if config.stop_sequences is not None:
444+
body["stop"] = config.stop_sequences
434445
# Pass-through any provider-specific extras (extra="allow"
435-
# on RuntimeConfig); spec §6 permits implementations to
436-
# accept additional fields.
446+
# on RuntimeConfig); spec §6 mandates implementations MUST
447+
# accept and forward undeclared fields untouched.
437448
extras = config.model_extra or {}
438449
for k, v in extras.items():
439450
body.setdefault(k, v)
@@ -1111,6 +1122,15 @@ def _request_params_from_config(config: RuntimeConfig | None) -> dict[str, Any]:
11111122
out["top_p"] = config.top_p
11121123
if config.seed is not None:
11131124
out["seed"] = config.seed
1125+
# Three fields promoted in proposal 0032; surfaced under their
1126+
# cross-vendor declared names so the observer emits
1127+
# gen_ai.request.{frequency_penalty,presence_penalty,stop_sequences}.
1128+
if config.frequency_penalty is not None:
1129+
out["frequency_penalty"] = config.frequency_penalty
1130+
if config.presence_penalty is not None:
1131+
out["presence_penalty"] = config.presence_penalty
1132+
if config.stop_sequences is not None:
1133+
out["stop_sequences"] = config.stop_sequences
11141134
return out
11151135

11161136

src/openarmature/llm/response.py

Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,7 @@
1515
1616
``RuntimeConfig`` is the optional per-call sampling-parameter record.
1717
Implementations MAY accept additional provider-specific fields; the
18-
four declared here (temperature, max_tokens, top_p, seed) are the
19-
mandated minimum.
18+
seven declared here are the mandated minimum.
2019
"""
2120

2221
from __future__ import annotations
@@ -107,12 +106,13 @@ class Response(BaseModel):
107106
response_model: str | None = None
108107

109108

109+
# Spec §6 declared-field surface: seven optional fields. Undeclared
110+
# fields supplied by callers MUST be forwarded to the wire body
111+
# untouched (extras pass-through); declared fields with value ``None``
112+
# MUST be omitted from the wire body (null-skip). Both rules are
113+
# enforced by the §8 wire-format mapping, not by RuntimeConfig itself.
110114
class RuntimeConfig(BaseModel):
111-
"""Per-call sampling parameters and budget hints.
112-
113-
All four fields are optional. Implementations MAY accept
114-
additional provider-specific fields; this is the minimum.
115-
"""
115+
"""Per-call sampling parameters and budget hints."""
116116

117117
model_config = ConfigDict(extra="allow")
118118

@@ -122,6 +122,32 @@ class RuntimeConfig(BaseModel):
122122
# Per spec §6: setting ``seed`` does NOT guarantee determinism; see
123123
# §9. Best-effort only, useful for providers that support it.
124124
seed: int | None = None
125+
# Promoted from extras to a declared field in proposal 0032 (spec
126+
# v0.24.0). Cross-vendor: OpenAI / Mistral / Cohere accept this
127+
# name directly; Anthropic and Gemini map to vendor-specific
128+
# equivalents at the wire layer (per §8.2 / §8.3 when those land).
129+
frequency_penalty: float | None = None
130+
presence_penalty: float | None = None
131+
# Declared field name matches the cross-vendor OpenTelemetry GenAI
132+
# semconv (``gen_ai.request.stop_sequences``) and Anthropic /
133+
# Gemini wire-key conventions. The §8.1 OpenAI-compatible wire
134+
# mapping translates ``stop_sequences`` to OpenAI's shorter body
135+
# key ``stop`` on emission. OpenAI is the outlier on the shorter
136+
# name; the declared layer matches the cross-vendor norm.
137+
stop_sequences: list[str] | None = None
138+
139+
# Pure Python ergonomic, not a spec contract. The wire-layer
140+
# null-skip rule already drops ``None``-valued declared fields, so
141+
# this helper exists solely to let callers splat a dict whose
142+
# entries may be ``None`` without filtering at the call site.
143+
@classmethod
144+
def from_partial(cls, **kwargs: Any) -> RuntimeConfig:
145+
"""Construct a config, dropping kwargs whose value is ``None``.
146+
147+
>>> RuntimeConfig.from_partial(temperature=0.7, top_p=None).top_p is None
148+
True
149+
"""
150+
return cls(**{k: v for k, v in kwargs.items() if v is not None})
125151

126152

127153
__all__ = [

src/openarmature/observability/otel/observer.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -765,6 +765,17 @@ def _handle_llm_event(self, event: NodeEvent) -> None:
765765
attrs["gen_ai.request.top_p"] = request_params["top_p"]
766766
if "seed" in request_params:
767767
attrs["gen_ai.request.seed"] = request_params["seed"]
768+
# Three new request-param attrs from proposal 0032
769+
# (spec v0.24.0). The §8.4.3 Langfuse mapping picks
770+
# these up by inclusion via the gen_ai.request.* →
771+
# generation.modelParameters.<suffix> rule with no
772+
# §8 edit.
773+
if "frequency_penalty" in request_params:
774+
attrs["gen_ai.request.frequency_penalty"] = request_params["frequency_penalty"]
775+
if "presence_penalty" in request_params:
776+
attrs["gen_ai.request.presence_penalty"] = request_params["presence_penalty"]
777+
if "stop_sequences" in request_params:
778+
attrs["gen_ai.request.stop_sequences"] = request_params["stop_sequences"]
768779
# §5.5.1 payload attributes (gated by ``disable_llm_payload``).
769780
# ``input.messages`` and ``request.extras`` go on the started
770781
# span; ``output.content`` lands on the completed branch.

tests/conformance/harness/directives.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -346,6 +346,10 @@ class RuntimeConfigSpec(_AllowExtras):
346346
max_tokens: int | None = None
347347
top_p: float | None = None
348348
seed: int | None = None
349+
# Three declared fields promoted by proposal 0032 (spec v0.24.0).
350+
frequency_penalty: float | None = None
351+
presence_penalty: float | None = None
352+
stop_sequences: list[str] | None = None
349353
extras: dict[str, Any] | None = None
350354

351355

0 commit comments

Comments
 (0)