Skip to content

Commit b283f97

Browse files
docs: cumulative strict-mode constraint list + spec-version-drift test
Captures two follow-ups surfaced by the four CoPilot review rounds: - docs/concepts/llms.md "Strict mode" section expanded into the full constraint list. After four rounds of tightening the strict_mode_supported heuristic, the rule set is stable and the user-facing surface should list it directly rather than make callers read provider.py. The page frames the list as the authoritative set: anything not on it trips to non-strict. - docs/model-providers/index.md "Strict mode" subsection trimmed and now links into the concepts page for the full list, following the established split (concepts/ owns the deep-dive, model-providers/ stays terse). - tests/test_smoke.py adds test_spec_version_matches_pyproject: reads pyproject.toml's [tool.openarmature].spec_version and asserts it equals openarmature.__spec_version__. AGENTS.md flags these as required to stay in sync; the previous smoke test only checked internal consistency between __spec_version__ and its asserted value, so the pyproject side could drift silently (and did, in the original submodule-bump commit).
1 parent cddb2b1 commit b283f97

3 files changed

Lines changed: 54 additions & 22 deletions

File tree

docs/concepts/llms.md

Lines changed: 30 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -163,22 +163,38 @@ inspect which path is active.
163163
### Strict mode
164164

165165
OpenAI's native path supports a `strict: true` flag that engages the
166-
model's schema-constrained decoding (the model literally cannot emit
167-
non-conforming tokens). It applies only when the schema satisfies
168-
specific constraints: `additionalProperties` explicitly `false` on every
169-
object, every key in `properties` listed in `required`, no
170-
unresolvable `$ref` targets.
171-
172-
`strict_mode_supported(schema)` performs the deep recursive check. The
173-
provider passes `strict: true` to the wire when the schema satisfies
174-
it, and `strict: false` otherwise. Either way, the provider validates
175-
the response post-receive against the supplied schema. Strict is a
166+
model's schema-constrained decoding: the model literally cannot emit
167+
non-conforming tokens. The framework decides `strict: true` vs
168+
`strict: false` automatically based on whether your schema satisfies
169+
strict-mode constraints. Either way, the framework validates the
170+
response post-receive against the supplied schema; strict is a
176171
wire-level optimization, not a correctness requirement.
177172

178-
If you control the schema, prefer making it strict-compatible:
179-
explicit `additionalProperties: false` plus `required` covering every
180-
property. Pydantic-derived schemas may need a tweak to satisfy this
181-
(`model_config = ConfigDict(extra="forbid")` on the class).
173+
`strict_mode_supported(schema)` (exported from `openarmature.llm`)
174+
performs the deep recursive check. The heuristic is conservative —
175+
anything not on the list below trips to `strict: false`:
176+
177+
- Top-level schema is `type: "object"`.
178+
- For every nested object: `additionalProperties` is **explicitly**
179+
`false`, and every key in `properties` is listed in `required`.
180+
- For every nested array: `items` is present and points to a
181+
verifiable schema (dict, or tuple-form list of dicts).
182+
- Every branch of `anyOf` / `oneOf` / `allOf` independently satisfies
183+
the above.
184+
- Internal `$ref` targets (`#/...` or bare `#`) resolve and their
185+
resolved schema passes. External refs (any other URI) and `$ref`
186+
cycles are handled conservatively.
187+
- Primitive types (`string`, `integer`, `number`, `boolean`, `null`)
188+
are accepted as terminal: no nested structure to verify.
189+
- Empty `{}` schemas and unrecognized-keyword schemas (`const`-only,
190+
`enum`-only, etc.) trip to non-strict; the walker can't statically
191+
verify them.
192+
193+
If you control the schema and want strict mode, the easiest path is to
194+
set `additionalProperties: false` and put every property in `required`
195+
on every object. Pydantic-derived schemas may need `model_config =
196+
ConfigDict(extra="forbid")` on the class to get the
197+
`additionalProperties: false` in the generated JSON Schema.
182198

183199
## Routing on parsed fields
184200

docs/model-providers/index.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -153,14 +153,14 @@ inspect which path is active.
153153
### Strict mode
154154

155155
OpenAI's native path supports a `strict: true` flag that engages
156-
schema-constrained decoding. It applies only when the schema satisfies
157-
specific constraints: `additionalProperties` explicitly `false` on
158-
every object, every key in `properties` listed in `required`, no
159-
unresolvable `$ref` targets. `strict_mode_supported(schema)` (exported
160-
from `openarmature.llm`) performs the deep recursive check; the
161-
provider passes `strict: true` to the wire when the schema satisfies
162-
it, and `strict: false` otherwise. Either way, the provider validates
163-
the response post-receive.
156+
schema-constrained decoding. The provider passes `strict: true` when
157+
the schema satisfies the strict-mode constraints and `strict: false`
158+
otherwise; the full constraint list lives on the
159+
[LLMs concepts page](../concepts/llms.md#strict-mode).
160+
`strict_mode_supported(schema)` is exported from `openarmature.llm`
161+
for callers wanting to check the heuristic directly. Either way, the
162+
provider validates the response post-receive against the supplied
163+
schema.
164164

165165
## A minimal example
166166

tests/test_smoke.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,22 @@
1+
import tomllib
2+
from pathlib import Path
3+
14
import openarmature
25

36

47
def test_package_versions() -> None:
58
assert openarmature.__version__ == "0.5.0"
69
assert openarmature.__spec_version__ == "0.15.0"
10+
11+
12+
def test_spec_version_matches_pyproject() -> None:
13+
# AGENTS.md flags __spec_version__, pyproject.toml's
14+
# [tool.openarmature].spec_version, and the submodule pin as
15+
# required to stay in sync. The test_package_versions check above
16+
# only verifies internal consistency between __spec_version__ and
17+
# its asserted value, so the pyproject side can drift undetected.
18+
# This test catches that class of three-place drift.
19+
pyproject_path = Path(__file__).resolve().parent.parent / "pyproject.toml"
20+
config = tomllib.loads(pyproject_path.read_text())
21+
pyproject_spec_version = config["tool"]["openarmature"]["spec_version"]
22+
assert openarmature.__spec_version__ == pyproject_spec_version

0 commit comments

Comments
 (0)