Skip to content

Commit ee68d7c

Browse files
PredictabilityAtScalePredictabilityAtScale
committed
Implemented gaps in scema
1 parent 0c55e95 commit ee68d7c

21 files changed

Lines changed: 758 additions & 43 deletions

‎README.md‎

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ Supported values for `warnings.contextSize` are `auto`, `off`, `result-only`, `c
162162
- **Overrides** — Environment and tier-based overrides (base → env → tier → runtime)
163163
- **5 provider adapters** — OpenAI (Chat), OpenAI (Responses), Anthropic, Gemini, OpenRouter — body-only output
164164
- **Provider-aware input caching controls** — optional `cache` front matter maps to OpenAI prompt cache hints, Anthropic `cache_control`, and Gemini `cachedContent`
165+
- **Vendor escape hatch** — optional `raw.<provider>` blocks shallow-merge unmodeled request-body fields into the final provider payload
165166
- **Validation** — Zod schema validation, Levenshtein-based "did you mean?" for typos, variable usage checks
166167
- **Context hardening** — structured regexes with flags, `/pattern/i` convenience syntax, and built-in `non_empty` / `reject_secrets` validators
167168
- **Optional short-circuit messages** — validators can return a structured `returnMessage` instead of throwing when configured
@@ -250,6 +251,48 @@ const request = openaiAdapter.render(prompt, {
250251

251252
In browser or client-side code, keep provider credentials on the server. Use the rendered request body with your own server endpoint, server action, or edge function rather than calling a provider directly from the client.
252253

254+
### Provider-specific fields and raw passthrough
255+
256+
Use normalized fields first (`sampling`, `response`, `cache`, `tools`) so prompts stay portable. `response.schema` is the neutral JSON Schema path; adapters emit it as OpenAI/OpenRouter `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
257+
258+
Use `provider_options` when PromptOpsKit has a known provider-specific mapping, such as Anthropic `top_k`, Gemini's native `response_schema`, or OpenRouter routing fields.
259+
260+
```yaml
261+
response:
262+
format: json
263+
schema_name: support_reply
264+
schema_description: Structured support reply
265+
schema:
266+
type: object
267+
properties:
268+
answer:
269+
type: string
270+
provider_options:
271+
openrouter:
272+
provider:
273+
order: ["anthropic", "openai"]
274+
transforms: ["middle-out"]
275+
```
276+
277+
When a provider adds a body field PromptOpsKit does not model yet, use `raw`:
278+
279+
```yaml
280+
raw:
281+
openai:
282+
service_tier: flex
283+
anthropic:
284+
service_tier: auto
285+
gemini:
286+
safetySettings:
287+
- category: HARM_CATEGORY_DANGEROUS_CONTENT
288+
threshold: BLOCK_ONLY_HIGH
289+
openrouter:
290+
usage:
291+
include: true
292+
```
293+
294+
Each adapter reads only its matching raw block and shallow-merges it into the generated request body after normalized mappings. This is intentionally an escape hatch; prefer first-class fields when they exist.
295+
253296
On the server, adapters also provide async prompt-aware helpers so you can use the default `./prompts` and `./.generated-prompts/json` directories without creating a `PromptOpsKit` instance:
254297

255298
```typescript
@@ -561,10 +604,11 @@ Prompt files use YAML front matter with these fields:
561604
| `fallback_models` | `string[]` | Fallback model list |
562605
| `reasoning` | `object` | `{ effort, budget_tokens }` |
563606
| `sampling` | `object` | `{ temperature, top_p, frequency_penalty, presence_penalty, stop, max_output_tokens }` |
564-
| `response` | `object` | `{ format, stream, schema, schema_name, schema_strict }` |
607+
| `response` | `object` | `{ format, stream, schema, schema_name, schema_description, schema_strict }` |
565608
| `cache` | `object` | Provider-specific cache controls (`openai`, `anthropic`, `gemini`/`google`) |
566609
| `tools` | `array` | Tool references (string names or inline definitions) |
567-
| `provider_options` | `object` | Provider-specific non-portable options (`anthropic`, `gemini`) |
610+
| `provider_options` | `object` | Provider-specific non-portable options (`anthropic`, `gemini`, `openrouter`) |
611+
| `raw` | `object` | Provider-scoped request-body passthrough (`openai`, `openai-responses`, `anthropic`, `gemini`/`google`, `openrouter`) |
568612
| `mcp` | `object` | MCP server references |
569613
| `context` | `object` | `{ inputs, history }` — declare expected variables, with optional per-input `max_size`, `trim`, structured or literal `allow_regex`/`deny_regex`, and built-in `non_empty` / `reject_secrets` validators |
570614
| `includes` | `string[]` | Paths to included prompt files |

‎SKILL.md‎

Lines changed: 61 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -63,10 +63,11 @@ the fields required by that specific file:
6363
| `fallback_models` | string[] | no | Ordered fallback model list |
6464
| `reasoning` | object | no | `{ effort: low|medium|high, budget_tokens: number }` |
6565
| `sampling` | object | no | `{ temperature, top_p, frequency_penalty, presence_penalty, stop, max_output_tokens }` |
66-
| `response` | object | no | `{ format: text|json|markdown, stream: boolean, schema?: object, schema_name?: string, schema_strict?: boolean }` |
66+
| `response` | object | no | `{ format: text|json|markdown, stream: boolean, schema?: object, schema_name?: string, schema_description?: string, schema_strict?: boolean }` |
6767
| `cache` | object | no | Provider-specific cache controls (`openai`, `anthropic`, `gemini`/`google`) |
6868
| `tools` | array | no | Tool names (strings) or inline definitions with `{ name, description, input_schema }` |
69-
| `provider_options` | object | no | Provider-specific advanced options (`anthropic`, `gemini`) |
69+
| `provider_options` | object | no | Provider-specific advanced options (`anthropic`, `gemini`, `openrouter`) |
70+
| `raw` | object | no | Provider-scoped request-body passthrough for unmodeled vendor fields |
7071
| `mcp` | object | no | `{ servers: [string | { name, config }] }` |
7172
| `context.inputs` | `Array<string | { name, max_size?, trim?, allow_regex?, deny_regex?, non_empty?, reject_secrets? }>` | no | Declared variable names used in templates, with optional size budgets and runtime hardening controls |
7273
| `context.history` | object | no | `{ max_items: number }` |
@@ -232,12 +233,69 @@ tiers:
232233
```
233234

234235
Overridable fields: `model`, `fallback_models`, `reasoning`, `sampling`,
235-
`response`, `cache`, `tools`, `provider_options`.
236+
`response`, `cache`, `raw`, `tools`, `provider_options`.
236237

237238
Override application order: **base → environment → tier → runtime**.
238239

239240
---
240241

242+
## Provider-specific fields
243+
244+
Prefer portable fields first:
245+
246+
- Use `sampling` for common sampling controls
247+
- Use `response.schema`, `response.schema_name`, `response.schema_description`, and `response.schema_strict` for structured output when possible
248+
- Use `cache` for provider cache hints
249+
- Use `tools` for tool definitions
250+
251+
Treat `response.schema` as the provider-neutral JSON Schema contract. The adapters emit it through provider-specific request fields: OpenAI/OpenRouter `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
252+
253+
Use `provider_options` for known non-portable mappings:
254+
255+
```yaml
256+
provider_options:
257+
anthropic:
258+
top_k: 40
259+
tool_choice:
260+
type: auto
261+
output_config:
262+
format:
263+
type: json_schema
264+
schema:
265+
type: object
266+
gemini:
267+
# Use only when Gemini's native schema dialect is required.
268+
response_schema:
269+
type: object
270+
response_json_schema:
271+
type: object
272+
openrouter:
273+
provider:
274+
order: ["anthropic", "openai"]
275+
transforms: ["middle-out"]
276+
```
277+
278+
Use `raw` only when a vendor request-body field is important and PromptOpsKit does not model it yet:
279+
280+
```yaml
281+
raw:
282+
openai:
283+
service_tier: flex
284+
anthropic:
285+
service_tier: auto
286+
gemini:
287+
safetySettings:
288+
- category: HARM_CATEGORY_DANGEROUS_CONTENT
289+
threshold: BLOCK_ONLY_HIGH
290+
openrouter:
291+
usage:
292+
include: true
293+
```
294+
295+
Raw blocks are provider-scoped (`openai`, `openai-responses`/`openai_responses`, `anthropic`, `gemini`/`google`, `openrouter`) and are shallow-merged into the final request body after normalized fields. When adding `raw`, include a short note in `# Notes` explaining why a first-class field is not being used.
296+
297+
---
298+
241299
## Test sidecars
242300

243301
Create a `.test.yaml` file alongside a prompt to define test cases:

‎docs/api-reference.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -247,7 +247,9 @@ const request = adapter.render(resolvedAsset, {
247247
});
248248
```
249249

250-
`RuntimeRenderOptions` for direct adapter rendering supports `environment`, `tier`, `runtime`, `variables`, `onContextOverflow`, `history`, `toolRegistry`, and `strict`.
250+
`RuntimeRenderOptions` for direct adapter rendering supports `environment`, `tier`, `runtime`, `variables`, `onContextOverflow`, `history`, `toolRegistry`, `strict`, and `openaiResponses`.
251+
252+
Runtime overrides can include the same overridable front matter fields as `environments` and `tiers`, including `raw` provider passthrough blocks. Raw blocks are merged into provider request bodies after normalized fields and provider-specific options.
251253

252254
## Standalone `renderPrompt`
253255

‎docs/overrides.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,12 +51,13 @@ Only these fields can be overridden in `environments` and `tiers`:
5151
| `fallback_models` | `string[]` | Fallback model list |
5252
| `reasoning` | `object` | `{ effort, budget_tokens }` |
5353
| `sampling` | `object` | `{ temperature, top_p, frequency_penalty, presence_penalty, stop, max_output_tokens }` |
54-
| `response` | `object` | `{ format, stream, schema, schema_name, schema_strict }` |
54+
| `response` | `object` | `{ format, stream, schema, schema_name, schema_description, schema_strict }` |
5555
| `cache` | `object` | Provider-specific cache controls (`openai`, `anthropic`, `gemini`/`google`) |
56+
| `raw` | `object` | Provider-specific request-body passthrough blocks |
5657
| `tools` | `array` | Tool references |
57-
| `provider_options` | `object` | Provider-specific advanced options (`anthropic`, `gemini`) |
58+
| `provider_options` | `object` | Provider-specific advanced options (`anthropic`, `gemini`, `openrouter`) |
5859

59-
Object fields (`reasoning`, `sampling`, `response`, `cache`, `provider_options`) are shallow-merged — individual sub-fields are replaced, but you don't need to repeat every sub-field.
60+
Object fields (`reasoning`, `sampling`, `response`, `cache`, `raw`, `provider_options`) are shallow-merged — individual sub-fields are replaced, but you don't need to repeat every sub-field.
6061

6162
## Applying overrides
6263

‎docs/prompt-format.md‎

Lines changed: 55 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,44 @@ cache:
132132
- `gemini.cached_content` (or `google.cached_content`) maps to `cachedContent` for requests that reuse a previously created Gemini cache.
133133
- You can safely include multiple provider blocks in the same prompt. Each adapter only reads its own block (`openai`, `anthropic`, or `gemini`/`google`) and ignores the others.
134134

135+
## Structured JSON output
136+
137+
Use the neutral `response` block for structured JSON whenever possible:
138+
139+
```yaml
140+
response:
141+
format: json
142+
schema_name: support_reply
143+
schema_description: Structured support reply
144+
schema:
145+
type: object
146+
properties:
147+
answer:
148+
type: string
149+
```
150+
151+
Adapters emit this JSON Schema through the provider-specific body shape: OpenAI/OpenRouter `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
152+
153+
Use provider-specific schema fields only when the vendor dialect itself matters, such as `provider_options.gemini.response_schema` for Gemini's native schema form.
154+
155+
## Raw provider passthrough
156+
157+
Use `raw` when a provider supports a request-body field that PromptOpsKit does not expose yet:
158+
159+
```yaml
160+
raw:
161+
openai:
162+
service_tier: flex
163+
anthropic:
164+
service_tier: auto
165+
gemini:
166+
safetySettings:
167+
- category: HARM_CATEGORY_DANGEROUS_CONTENT
168+
threshold: BLOCK_ONLY_HIGH
169+
```
170+
171+
Each adapter reads only its own raw block. Raw values are shallow-merged into the generated body after normalized mappings, so they can override generated fields. Prefer normalized fields (`sampling`, `response`, `cache`, `tools`) and `provider_options` first; reserve `raw` for vendor-specific fields that would otherwise be impossible to send.
172+
135173
## Sections
136174

137175
The Markdown body is split on **H1 headings** into named sections. Three section names are recognized (case-insensitive):
@@ -261,7 +299,20 @@ sampling:
261299
temperature: 0.7
262300
max_output_tokens: 2048
263301
response:
264-
format: text
302+
format: json
303+
schema_name: support_reply
304+
schema_description: Structured support reply
305+
schema:
306+
type: object
307+
properties:
308+
answer:
309+
type: string
310+
required:
311+
- answer
312+
provider_options:
313+
openrouter:
314+
transforms:
315+
- middle-out
265316
context:
266317
inputs:
267318
- user_message
@@ -285,6 +336,9 @@ tiers:
285336
model: gpt-5.4-mini
286337
pro:
287338
model: gpt-5.4
339+
raw:
340+
openai:
341+
service_tier: flex
288342
metadata:
289343
owner: support-platform
290344
review_required: true

0 commit comments

Comments
 (0)