Skip to content

Commit bb839f9

Browse files
FEATURE: Added LLMAsAService provider
1 parent cd41125 commit bb839f9

26 files changed

Lines changed: 877 additions & 63 deletions

README.md

Lines changed: 39 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ To one reviewable asset:
3434
Core capabilities:
3535

3636
- **Markdown prompt assets** — capture prompt text, model config, tool bindings, context rules, and metadata together.
37-
- **Provider-ready output** — render request bodies for OpenAI Chat, OpenAI Responses, Anthropic, Gemini, and OpenRouter while your app owns transport.
37+
- **Provider-ready output** — render request bodies for OpenAI Chat, OpenAI Responses, Anthropic, Gemini, OpenRouter, and LLMAsAService while your app owns transport.
3838
- **Input hardening** — define required values, size limits, allow/deny patterns, and secret rejection close to the prompt template.
3939
- **Reusable composition** — share tone, policy, and safety instructions with `includes`, and apply folder-level standards with `defaults.md`.
4040
- **Environment and tier overrides** — keep dev/prod and plan-specific behavior in one prompt source with explicit, reviewable overrides.
@@ -148,7 +148,7 @@ Supported values for `warnings.contextSize` are `auto`, `off`, `result-only`, `c
148148
- **Composition**`includes` to share system instructions across prompts, with circular detection
149149
- **Folder defaults**`defaults.md` inheritance for shared provider, model, metadata, and system instructions
150150
- **Overrides** — Environment and tier-based overrides (base → env → tier → runtime)
151-
- **5 provider adapters** — OpenAI (Chat), OpenAI (Responses), Anthropic, Gemini, OpenRouter — body-only output
151+
- **6 provider adapters** — OpenAI (Chat), OpenAI (Responses), Anthropic, Gemini, OpenRouter, LLMAsAService
152152
- **Provider-aware input caching controls** — optional `cache` front matter maps to OpenAI prompt cache hints, Anthropic `cache_control`, and Gemini `cachedContent`
153153
- **Vendor escape hatch** — optional `raw.<provider>` blocks shallow-merge unmodeled request-body fields into the final provider payload
154154
- **Validation** — Zod schema validation, Levenshtein-based "did you mean?" for typos, variable usage checks
@@ -203,6 +203,24 @@ result = await kit.renderPrompt({
203203
variables: { name: 'World', app_context: 'Welcome screen' },
204204
});
205205
if (!result.request) throw new Error(result.returnMessage ?? 'Prompt rendering failed.');
206+
207+
// LLMAsAService — OpenAI-compatible gateway with project and customer metadata
208+
result = await kit.renderPrompt({
209+
path: 'hello',
210+
provider: 'llmasaservice',
211+
runtime: {
212+
provider_options: {
213+
llmasaservice: {
214+
project_id: process.env.LLM_GATEWAY_PROJECT_ID,
215+
customer: { customer_id: 'cust_123', customer_name: 'Acme' },
216+
},
217+
},
218+
},
219+
variables: { name: 'World', app_context: 'Welcome screen' },
220+
});
221+
if (!result.request) throw new Error(result.returnMessage ?? 'Prompt rendering failed.');
222+
// result.request.body → { model, messages, customer, ... }
223+
// result.request.headers → { 'x-project-id': '...' }
206224
```
207225

208226
Provider adapters are also available as direct imports:
@@ -213,6 +231,7 @@ import { openaiResponsesAdapter } from 'promptopskit/openai-responses';
213231
import { anthropicAdapter } from 'promptopskit/anthropic';
214232
import { geminiAdapter } from 'promptopskit/gemini';
215233
import { openrouterAdapter } from 'promptopskit/openrouter';
234+
import { llmasaserviceAdapter } from 'promptopskit/llmasaservice';
216235
```
217236

218237
Direct adapter rendering also accepts `environment` and `tier` selectors. This is useful for compiled JSON/ESM assets in browser, edge, or worker code:
@@ -242,9 +261,9 @@ In browser or client-side code, keep provider credentials on the server. Use the
242261

243262
### Provider-specific fields and raw passthrough
244263

245-
Use normalized fields first (`sampling`, `response`, `cache`, `tools`) so prompts stay portable. `response.schema` is the neutral JSON Schema path; adapters emit it as OpenAI/OpenRouter `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
264+
Use normalized fields first (`sampling`, `response`, `cache`, `tools`) so prompts stay portable. `response.schema` is the neutral JSON Schema path; adapters emit it as OpenAI/OpenRouter/LLMAsAService `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
246265

247-
Use `provider_options` when PromptOpsKit has a known provider-specific mapping, such as Anthropic `top_k`, Gemini's native `response_schema`, or OpenRouter routing fields.
266+
Use `provider_options` when PromptOpsKit has a known provider-specific mapping, such as Anthropic `top_k`, Gemini's native `response_schema`, OpenRouter routing fields, or LLMAsAService gateway routing/customer metadata.
248267

249268
```yaml
250269
response:
@@ -261,8 +280,16 @@ provider_options:
261280
provider:
262281
order: ["anthropic", "openai"]
263282
transforms: ["middle-out"]
283+
llmasaservice:
284+
project_id: "llm-project-id"
285+
# Optional default; usually pass the real customer at render time.
286+
customer:
287+
customer_id: "cust_123"
288+
customer_name: "Acme"
264289
```
265290
291+
For LLMAsAService, `provider_options.llmasaservice.customer` is intended to be render-time attribution for the current account/user. A prompt can keep a default, but production calls should normally override it through `runtime.provider_options.llmasaservice.customer`.
292+
266293
When a provider adds a body field PromptOpsKit does not model yet, use `raw`:
267294

268295
```yaml
@@ -278,6 +305,8 @@ raw:
278305
openrouter:
279306
usage:
280307
include: true
308+
llmasaservice:
309+
conversationId: "conv_123"
281310
```
282311

283312
Each adapter reads only its matching raw block and shallow-merges it into the generated request body after normalized mappings. This is intentionally an escape hatch; prefer first-class fields when they exist.
@@ -336,7 +365,7 @@ Use PromptOpsKit when you want:
336365

337366
## Optional UsageTap Tracking
338367

339-
PromptOpsKit can also help you track provider calls with UsageTap.com while keeping the core render API body-only.
368+
PromptOpsKit can also help you track provider calls with UsageTap.com while keeping the core render API transport-light.
340369

341370
```typescript
342371
import { createPromptOpsKit } from 'promptopskit';
@@ -400,7 +429,7 @@ const tracked = await runOpenAIWithUsageTap(usageTap, {
400429

401430
Notes:
402431
- `entitlementMode` defaults to `'off'`. Set it to `'apply'` only when you want UsageTap allowances to mutate a cloned provider request.
403-
- `runOpenRouterWithUsageTap`, `runAnthropicWithUsageTap`, and `runGeminiWithUsageTap` follow the same pattern.
432+
- `runOpenRouterWithUsageTap`, `runLLMAsAServiceWithUsageTap`, `runAnthropicWithUsageTap`, and `runGeminiWithUsageTap` follow the same pattern.
404433
- `extractOpenAIUsage`, `extractAnthropicUsage`, and `extractGeminiUsage` are public if you want to manage UsageTap lifecycle yourself.
405434

406435
For explicit lifecycle control, use `beginUsageTapCall`, `endUsageTapCall`, or `withUsageTapCall` from `promptopskit/usagetap`. Full documentation: [docs/usagetap.md](./docs/usagetap.md).
@@ -593,7 +622,7 @@ Renders a prompt for a specific provider. Returns `{ resolved, request?, returnM
593622
|--------|------|-------------|
594623
| `path` | `string` | Prompt path (no extension), e.g. `'support/reply'` |
595624
| `source` | `string` | Inline prompt source (alternative to path) |
596-
| `provider` | `string` | `'openai'`, `'openai-responses'`, `'anthropic'`, `'gemini'`, `'openrouter'` |
625+
| `provider` | `string` | `'openai'`, `'openai-responses'`, `'anthropic'`, `'gemini'`, `'openrouter'`, `'llmasaservice'` |
597626
| `variables` | `Record<string, string>` | Template variables |
598627
| `onContextOverflow` | `(info) => string` | Optional callback to transform oversized context values before rendering |
599628
| `onHistoryCompaction` | `(info) => string \| { role, content }` | Optional callback to compact overflow history when `context.history.max_items` is exceeded |
@@ -622,16 +651,16 @@ Prompt files use YAML front matter with these fields:
622651
|-------|------|-------------|
623652
| `id` | `string` | Unique prompt identifier (required) |
624653
| `schema_version` | `number` | Schema version, currently `1` |
625-
| `provider` | `string` | `openai`, `openai-responses`, `anthropic`, `gemini` (or `google`), `openrouter`, `any` |
654+
| `provider` | `string` | `openai`, `openai-responses`, `anthropic`, `gemini` (or `google`), `openrouter`, `llmasaservice`, `any` |
626655
| `model` | `string` | Model name |
627656
| `fallback_models` | `string[]` | Fallback model list |
628657
| `reasoning` | `object` | `{ effort, budget_tokens }` |
629658
| `sampling` | `object` | `{ temperature, top_p, frequency_penalty, presence_penalty, stop, max_output_tokens }` |
630659
| `response` | `object` | `{ format, stream, schema, schema_name, schema_description, schema_strict }` |
631660
| `cache` | `object` | Provider-specific cache controls (`openai`, `anthropic`, `gemini`/`google`) |
632661
| `tools` | `array` | Tool references (string names or inline definitions) |
633-
| `provider_options` | `object` | Provider-specific non-portable options (`anthropic`, `gemini`, `openrouter`) |
634-
| `raw` | `object` | Provider-scoped request-body passthrough (`openai`, `openai-responses`, `anthropic`, `gemini`/`google`, `openrouter`) |
662+
| `provider_options` | `object` | Provider-specific non-portable options (`anthropic`, `gemini`, `openrouter`, `llmasaservice`) |
663+
| `raw` | `object` | Provider-scoped request-body passthrough (`openai`, `openai-responses`, `anthropic`, `gemini`/`google`, `openrouter`, `llmasaservice`) |
635664
| `mcp` | `object` | MCP server references |
636665
| `context` | `object` | `{ inputs, history }` — declare expected variables, with optional per-input `max_size`, `trim`, structured or literal `allow_regex`/`deny_regex`, built-in `non_empty` / `reject_secrets` validators, and `history.max_items` compaction |
637666
| `includes` | `string[]` | Paths to included prompt files |

SKILL.md

Lines changed: 64 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ description: Guidance for creating and editing promptopskit prompt files, defaul
88
This project uses **promptopskit** to manage LLM prompts as code.
99
Prompts live in markdown files with YAML front matter, are validated against
1010
a schema, and render into provider-specific request bodies (OpenAI, Anthropic,
11-
Gemini, OpenRouter, and OpenAI Responses). Follow these instructions when creating or editing prompts.
11+
Gemini, OpenRouter, LLMAsAService, and OpenAI Responses). Follow these instructions when creating or editing prompts.
1212

1313
---
1414

@@ -119,8 +119,8 @@ into a provider request body:
119119
- "generate the Anthropic call for the prompt [name]"
120120
- "render/generate/build/create/produce the request/body/call/payload/messages
121121
for prompt [name] with provider [provider]"
122-
- "turn [name] into a provider request for openai/anthropic/google/gemini/openrouter"
123-
- "wire up prompt [name] to OpenAI/Anthropic/Gemini/OpenRouter"
122+
- "turn [name] into a provider request for openai/anthropic/google/gemini/openrouter/llmasaservice"
123+
- "wire up prompt [name] to OpenAI/Anthropic/Gemini/OpenRouter/LLMAsAService"
124124
- "give me code to call [provider] with prompt [name]"
125125

126126
Provider aliases:
@@ -132,14 +132,15 @@ Provider aliases:
132132
| `anthropic`, `claude` | `anthropic` |
133133
| `google`, `gemini` | `gemini` |
134134
| `openrouter` | `openrouter` |
135+
| `llmasaservice`, `llmasaservice.io`, `llm gateway` | `llmasaservice` |
135136

136137
Behavior:
137138

138139
1. Generate code unless the user explicitly asks for only the raw rendered JSON.
139140
2. Prefer `createPromptOpsKit().renderPrompt()` for server-side app code that loads
140141
prompt source or compiled JSON by path.
141142
3. Prefer provider adapters (`openaiAdapter`, `anthropicAdapter`,
142-
`geminiAdapter`, `openrouterAdapter`) when the user asks for provider-specific
143+
`geminiAdapter`, `openrouterAdapter`, `llmasaserviceAdapter`) when the user asks for provider-specific
143144
integration code or already has a compiled asset.
144145
4. Include `variables` for every declared prompt input, using realistic placeholder
145146
values or function parameters.
@@ -256,6 +257,51 @@ if (!result.request) throw new Error('Prompt rendering did not produce an OpenRo
256257
const completion = await client.chat.completions.create(result.request.body as any);
257258
```
258259

260+
LLMAsAService example:
261+
262+
```typescript
263+
import OpenAI from 'openai';
264+
import {
265+
createLLMAsAServiceOpenAIConfig,
266+
llmasaserviceAdapter,
267+
} from 'promptopskit/llmasaservice';
268+
269+
const client = new OpenAI(createLLMAsAServiceOpenAIConfig({
270+
baseURL: process.env.LLM_GATEWAY_BASE_URL,
271+
projectId: process.env.LLM_GATEWAY_PROJECT_ID,
272+
}));
273+
274+
const result = await llmasaserviceAdapter.renderPrompt(
275+
{
276+
path: 'support/triage-summary',
277+
},
278+
{
279+
runtime: {
280+
provider_options: {
281+
llmasaservice: {
282+
project_id: process.env.LLM_GATEWAY_PROJECT_ID,
283+
customer: {
284+
customer_id: customer.id,
285+
customer_name: customer.name,
286+
customer_user_id: user.id,
287+
customer_user_email: user.email,
288+
},
289+
},
290+
},
291+
},
292+
variables: { ticket: ticketText },
293+
strict: true,
294+
},
295+
);
296+
297+
if (result.returnMessage) return result.returnMessage;
298+
if (!('body' in result)) {
299+
throw new Error('Prompt rendering did not produce an LLMAsAService request.');
300+
}
301+
302+
const completion = await client.chat.completions.create(result.body as any);
303+
```
304+
259305
If the user asks for "just the body", render with `kit.renderPrompt()` and show
260306
or return `result.request.body`, not the whole render result.
261307

@@ -330,15 +376,15 @@ the fields required by that specific file:
330376
| `id` | string | **yes** | Unique identifier for the prompt |
331377
| `schema_version` | number | yes | Always `1` |
332378
| `description` | string | no | Human-readable description |
333-
| `provider` | enum | no | `openai`, `openai-responses`, `anthropic`, `google`, `gemini`, `openrouter`, or `any` |
379+
| `provider` | enum | no | `openai`, `openai-responses`, `anthropic`, `google`, `gemini`, `openrouter`, `llmasaservice`, or `any` |
334380
| `model` | string | no | Model identifier (e.g. `gpt-5.4`, `claude-sonnet-4-20250514`) |
335381
| `fallback_models` | string[] | no | Ordered fallback model list |
336382
| `reasoning` | object | no | `{ effort: low|medium|high, budget_tokens: number }` |
337383
| `sampling` | object | no | `{ temperature, top_p, frequency_penalty, presence_penalty, stop, max_output_tokens }` |
338384
| `response` | object | no | `{ format: text|json|markdown, stream: boolean, schema?: object, schema_name?: string, schema_description?: string, schema_strict?: boolean }` |
339385
| `cache` | object | no | Provider-specific cache controls (`openai`, `anthropic`, `gemini`/`google`) |
340386
| `tools` | array | no | Tool names (strings) or inline definitions with `{ name, description, input_schema }` |
341-
| `provider_options` | object | no | Provider-specific advanced options (`anthropic`, `gemini`, `openrouter`) |
387+
| `provider_options` | object | no | Provider-specific advanced options (`anthropic`, `gemini`, `openrouter`, `llmasaservice`) |
342388
| `raw` | object | no | Provider-scoped request-body passthrough for unmodeled vendor fields |
343389
| `mcp` | object | no | `{ servers: [string | { name, config }] }` |
344390
| `context.inputs` | `Array<string | { name, max_size?, trim?, allow_regex?, deny_regex?, non_empty?, reject_secrets? }>` | no | Declared variable names used in templates, with optional size budgets and runtime hardening controls |
@@ -548,7 +594,7 @@ Prefer portable fields first:
548594
- Use `cache` for provider cache hints
549595
- Use `tools` for tool definitions
550596

551-
Treat `response.schema` as the provider-neutral JSON Schema contract. The adapters emit it through provider-specific request fields: OpenAI/OpenRouter `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
597+
Treat `response.schema` as the provider-neutral JSON Schema contract. The adapters emit it through provider-specific request fields: OpenAI/OpenRouter/LLMAsAService `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
552598

553599
Use `provider_options` for known non-portable mappings:
554600

@@ -573,8 +619,16 @@ provider_options:
573619
provider:
574620
order: ["anthropic", "openai"]
575621
transforms: ["middle-out"]
622+
llmasaservice:
623+
project_id: llm-project-id
624+
# Optional default; usually pass the real customer at render time.
625+
customer:
626+
customer_id: cust_123
627+
customer_name: Acme
576628
```
577629

630+
For LLMAsAService, prefer putting the current customer/account/user attribution in `runtime.provider_options.llmasaservice.customer` during rendering. Static prompt metadata may include a default, but runtime values should override it for real requests.
631+
578632
Use `raw` only when a vendor request-body field is important and PromptOpsKit does not model it yet:
579633

580634
```yaml
@@ -590,9 +644,11 @@ raw:
590644
openrouter:
591645
usage:
592646
include: true
647+
llmasaservice:
648+
conversationId: conv_123
593649
```
594650

595-
Raw blocks are provider-scoped (`openai`, `openai-responses`/`openai_responses`, `anthropic`, `gemini`/`google`, `openrouter`) and are shallow-merged into the final request body after normalized fields. When adding `raw`, include a short note in `# Notes` explaining why a first-class field is not being used.
651+
Raw blocks are provider-scoped (`openai`, `openai-responses`/`openai_responses`, `anthropic`, `gemini`/`google`, `openrouter`, `llmasaservice`) and are shallow-merged into the final request body after normalized fields. When adding `raw`, include a short note in `# Notes` explaining why a first-class field is not being used.
596652

597653
---
598654

docs/api-reference.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ const result = await kit.renderPrompt({
6565
|--------|------|-------------|
6666
| `path` | `string` | Prompt path (no extension), e.g. `'support/reply'` |
6767
| `source` | `string` | Inline prompt source (alternative to `path`) |
68-
| `provider` | `string` | `'openai'`, `'openai-responses'`, `'anthropic'`, `'gemini'`, `'openrouter'` (required) |
68+
| `provider` | `string` | `'openai'`, `'openai-responses'`, `'anthropic'`, `'gemini'`, `'openrouter'`, `'llmasaservice'` (required) |
6969
| `variables` | `Record<string, string>` | Template variables |
7070
| `onContextOverflow` | `(info) => string` | Optional callback to transform an oversized context value before rendering |
7171
| `onHistoryCompaction` | `(info) => string \| { role, content }` | Optional callback used when `context.history.max_items` compacts overflow history |
@@ -83,7 +83,7 @@ Either `path` or `source` must be provided.
8383
```typescript
8484
interface RenderResult {
8585
resolved: ResolvedPromptAsset; // Fully resolved asset
86-
request?: ProviderRequest; // { body, provider, model } when rendering continues
86+
request?: ProviderRequest; // { body, provider, model, baseURL?, headers? } when rendering continues
8787
returnMessage?: string; // Short-circuit message from context validation when configured
8888
warnings: string[]; // Non-fatal provider and render-time warnings
8989
}
@@ -250,6 +250,8 @@ const request = adapter.render(resolvedAsset, {
250250
});
251251
```
252252

253+
Supported adapter names are `openai`, `openai-responses`, `anthropic`, `gemini`/`google`, `openrouter`, and `llmasaservice`.
254+
253255
`RuntimeRenderOptions` for direct adapter rendering supports `environment`, `tier`, `runtime`, `variables`, `onContextOverflow`, `history`, `onHistoryCompaction`, `toolRegistry`, `strict`, and `openaiResponses`.
254256

255257
Runtime overrides can include the same overridable front matter fields as `environments` and `tiers`, including `raw` provider passthrough blocks. Raw blocks are merged into provider request bodies after normalized fields and provider-specific options.

0 commit comments

Comments
 (0)