Skip to content

Commit 78f6fc7

Browse files
FIX: Tightened regex to avoid escaping with double quoted strings. Added validation error to catch incorrect regex patterns
1 parent ee68d7c commit 78f6fc7

16 files changed

Lines changed: 384 additions & 34 deletions

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,9 +94,7 @@ context:
9494
reject_secrets: true
9595
- name: app_context
9696
max_size: 2000
97-
allow_regex:
98-
pattern: "^[A-Za-z0-9 _-]+$"
99-
flags: "i"
97+
allow_regex: /^[A-Za-z0-9 _-]+$/i
10098
includes:
10199
- ./shared/tone.md
102100
---
@@ -164,7 +162,7 @@ Supported values for `warnings.contextSize` are `auto`, `off`, `result-only`, `c
164162
- **Provider-aware input caching controls** — optional `cache` front matter maps to OpenAI prompt cache hints, Anthropic `cache_control`, and Gemini `cachedContent`
165163
- **Vendor escape hatch** — optional `raw.<provider>` blocks shallow-merge unmodeled request-body fields into the final provider payload
166164
- **Validation** — Zod schema validation, Levenshtein-based "did you mean?" for typos, variable usage checks
167-
- **Context hardening**structured regexes with flags, `/pattern/i` convenience syntax, and built-in `non_empty` / `reject_secrets` validators
165+
- **Context hardening**copyable `/pattern/i` regex literals, structured regexes with `return_message`, and built-in `non_empty` / `reject_secrets` validators
168166
- **Optional short-circuit messages** — validators can return a structured `returnMessage` instead of throwing when configured
169167
- **Context size guardrails** — optional per-input `max_size` metadata with non-blocking render-time warnings
170168
- **Warning controls** — top-level config can suppress or emit context size warnings differently in dev and prod
@@ -616,6 +614,8 @@ Prompt files use YAML front matter with these fields:
616614
| `tiers` | `object` | Named tier overrides |
617615
| `metadata` | `object` | `{ owner, tags, review_required, stable }` |
618616

617+
For `allow_regex` and `deny_regex`, prefer unquoted `/pattern/i` literal form so regex escapes such as `\s` and `\b` stay copyable from tools like regex101. If you use structured `pattern:` form, use single-quoted YAML strings or double each backslash in double-quoted strings.
618+
619619
## Website
620620

621621
The `website/` directory contains a standalone marketing website for PromptOpsKit.

SKILL.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,9 @@ Rules:
105105
- Use object-form inputs with `max_size` when a variable is likely to grow large and should trigger early warnings
106106
- Use `trim` to enforce byte budgets before interpolation when `max_size` is set
107107
- Use `allow_regex` for allowlist checks and `deny_regex` for blocklist checks on risky inputs
108-
- Prefer structured regexes like `{ pattern, flags }`; `/pattern/i` strings are also accepted and normalized internally
108+
- Prefer unquoted `/pattern/i` literals for regex validators so backslash escapes such as `\s` and `\b` stay copyable from regex tools
109+
- Use structured regexes like `{ pattern, flags, return_message }` when the validator needs a fallback message or separate flags
110+
- In structured `pattern:` YAML, use single quotes for patterns with backslashes or double each backslash in double-quoted strings
109111
- Use `non_empty: true` for required user text and `reject_secrets: true` for common secret redaction checks
110112
- When the caller should receive a structured fallback message instead of an exception, use object form with `return_message` on `allow_regex`, `deny_regex`, `non_empty`, or `reject_secrets`
111113
- Escape literal braces with `\{{` and `\}}`
@@ -127,7 +129,7 @@ At render time, callers can also pass `onContextOverflow` to transform oversized
127129

128130
If a validator declares `return_message`, `renderPrompt()` returns that message in a structured result and omits the provider request instead of throwing for that validation failure. Invalid regex definitions still fail during `validate` and `compile` as `POK013` prompt-authoring errors.
129131

130-
Malformed `allow_regex` and `deny_regex` values fail during `validate` and `compile`, not just at render time. When regex compilation fails, the error includes the prompt id, variable name, field name, and raw configured value.
132+
Malformed `allow_regex` and `deny_regex` values fail during `validate` and `compile`, not just at render time. When regex compilation fails, the error includes the prompt id, variable name, field name, and raw configured value. Double-quoted YAML regex strings with raw backslashes fail as `POK013`; use `/pattern/i`, single-quoted `pattern: '...'`, or doubled backslashes.
131133
132134
Example: this is the minimal valid shape for a prompt that references
133135
`{{ pull_request }}` even when provider/model are inherited from defaults:

docs/api-reference.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ const result = await kit.validatePrompt('support/reply');
118118
// { valid: boolean, errors: ValidationError[], warnings: ValidationError[] }
119119
```
120120

121-
`validatePrompt()` covers schema, include-graph, variable declaration issues, and context regex compilation. Render-time context size warnings are produced by `renderPrompt()`, not validation.
121+
`validatePrompt()` covers schema, include-graph, variable declaration issues, context regex compilation, and context regex YAML quoting problems. Render-time context size warnings are produced by `renderPrompt()`, not validation.
122122

123123
## `kit.clearCache()`
124124

@@ -223,7 +223,7 @@ const result = validateAsset(asset, ['id', 'schema_version', 'model'], 'hello.md
223223
// { valid: boolean, errors: ValidationError[], warnings: ValidationError[] }
224224
```
225225

226-
`validateAsset()` reports malformed `allow_regex` and `deny_regex` values before runtime, including the prompt id, variable name, field name, and raw configured value in the error message.
226+
`validateAsset()` reports malformed `allow_regex` and `deny_regex` values before runtime, including the prompt id, variable name, field name, and raw configured value in the error message. When parsing source Markdown through `parsePrompt()`, `loadPromptFile()`, or `validatePrompt()`, parser-level checks also report unsafe double-quoted YAML regex strings with raw backslashes as `POK013`; prefer unquoted `/pattern/i` literal form for copyable regex escapes.
227227

228228
### `validateAssetWithIncludes(asset, filePath, frontMatterKeys?)`
229229

docs/cli.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ Checks:
5151
- Unknown front matter keys with Levenshtein-based "did you mean?" suggestions
5252
- Variable usage — used but undeclared, declared but unused
5353
- Context regex compilation for `allow_regex` and `deny_regex`
54+
- YAML quoting problems in context regex fields, such as raw `\s` or `\b` inside double-quoted strings
5455
- Include resolution — missing files, circular includes
5556
- Folder defaults inheritance from `defaults.md` (provider, model, metadata, system instructions)
5657

@@ -82,7 +83,7 @@ promptopskit compile [sourceDir] [outputDir] [--source <dir>] [--output <dir>] [
8283

8384
Includes are resolved during compilation so compiled artifacts are self-sufficient. The output directory is cleared by default before compiling (unless `--no-clean` is set).
8485

85-
Compilation runs validation before writing artifacts. Invalid `allow_regex` or `deny_regex` definitions fail the compile step early with `POK013` instead of surfacing later during `renderPrompt()`.
86+
Compilation runs validation before writing artifacts. Invalid `allow_regex` or `deny_regex` definitions, including unsafe double-quoted YAML regex strings with raw backslashes, fail the compile step early with `POK013` instead of surfacing later during `renderPrompt()`.
8687

8788
If you omit `<out>`, the CLI chooses `./.generated-prompts/json` for `json` and `./.generated-prompts/esm` for `esm`.
8889

docs/getting-started.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ context:
4747
non_empty: true
4848
reject_secrets: true
4949
- name: app_context
50-
allow_regex: "/^[A-Za-z0-9 _-]+$/i"
50+
allow_regex: /^[A-Za-z0-9 _-]+$/i
5151
includes:
5252
- ./shared/tone.md
5353
---
@@ -121,7 +121,7 @@ Your application owns the HTTP call — PromptOpsKit produces the request body o
121121
npx promptopskit validate ./prompts
122122
```
123123

124-
This checks all `.md` files for schema errors, unknown front matter keys (with "did you mean?" suggestions), variable usage mismatches, and malformed context regex definitions.
124+
This checks all `.md` files for schema errors, unknown front matter keys (with "did you mean?" suggestions), variable usage mismatches, and malformed context regex definitions. For regex validators, prefer unquoted `/pattern/i` literals so backslash escapes stay copyable; double-quoted YAML regex strings with raw backslashes are reported as `POK013`.
125125

126126
## Compile for production
127127

docs/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Open-source developer toolkit for managing prompts, system instructions, tools,
1919
- [Schema](./schema.md) — Full YAML front matter schema reference
2020
- [Vendor Schema Gap Analysis](./vendor-schema-gap-analysis.md) — Snapshot comparison against published OpenAI, Anthropic, Gemini, and OpenRouter schema capabilities
2121
- [Testing](./testing.md) — Test helpers, mock assets, and sidecar test files
22-
- [Validation](./validation.md) — Schema validation, "did you mean?" suggestions, variable checks, and early regex validation
22+
- [Validation](./validation.md) — Schema validation, "did you mean?" suggestions, variable checks, early regex validation, and YAML regex quoting guidance
2323

2424
## Also see
2525

docs/prompt-format.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -234,8 +234,8 @@ Each entry can be either a string variable name or an object with:
234234
- `name` — the template variable name
235235
- `max_size` — optional UTF-8 byte limit for the injected value
236236
- `trim` — optional trim-to-budget (`true`/`end` keeps first bytes, `start` keeps trailing bytes) applied when `max_size` is set
237-
- `allow_regex` — optional allowlist regex; accepts `"pattern"`, `/pattern/i`, or `{ pattern, flags, return_message? }` and throws `POK031` on mismatch unless `return_message` is configured
238-
- `deny_regex` — optional blocklist regex; accepts `"pattern"`, `/pattern/i`, or `{ pattern, flags, return_message? }` and throws `POK032` on match unless `return_message` is configured
237+
- `allow_regex` — optional allowlist regex; accepts `/pattern/i`, `"pattern"`, or `{ pattern, flags, return_message? }` and throws `POK031` on mismatch unless `return_message` is configured
238+
- `deny_regex` — optional blocklist regex; accepts `/pattern/i`, `"pattern"`, or `{ pattern, flags, return_message? }` and throws `POK032` on match unless `return_message` is configured
239239
- `non_empty` — optional boolean or object validator; use `true` to throw `POK033`, or `{ return_message }` to short-circuit rendering with a structured message
240240
- `reject_secrets` — optional boolean or object validator; use `true` to throw `POK034`, or `{ return_message }` to short-circuit rendering with a structured message
241241

@@ -245,7 +245,7 @@ The validator warns about:
245245

246246
At render time, PromptOpsKit also emits a non-blocking `POK030` warning when a provided variable exceeds its declared `max_size`. In source and auto modes, the warning is also written to `console.warn` to make local development issues visible early.
247247

248-
Malformed `allow_regex` and `deny_regex` values fail during `validate` and `compile` with `POK013`, so bad patterns are caught before runtime.
248+
Malformed `allow_regex` and `deny_regex` values fail during `validate` and `compile` with `POK013`, so bad patterns are caught before runtime. Double-quoted YAML regex strings with raw backslashes are also reported as `POK013`; use unquoted `/pattern/i`, single-quoted `pattern: '...'`, or doubled backslashes in double quotes.
249249
250250
Example hardened input definition:
251251
@@ -256,16 +256,18 @@ context:
256256
trim: true
257257
max_size: 24
258258
allow_regex:
259-
pattern: "^user_[a-z0-9]+$"
260-
flags: "i"
261-
return_message: "User IDs must use the user_123 format."
259+
pattern: '^user_[a-z0-9]+$'
260+
flags: 'i'
261+
return_message: 'User IDs must use the user_123 format.'
262262
- name: pull_request_body
263263
non_empty:
264-
return_message: "Pull request content is required."
264+
return_message: 'Pull request content is required.'
265265
reject_secrets: true
266-
deny_regex: "/(ignore previous instructions|system:)/i"
266+
deny_regex: /(?:ignore|disregard|forget)\s+(?:all\s+)?(?:previous|prior|above)\s+instructions|(?:^|\b)(?:system|developer|assistant)\s*:|reveal\s+(?:your|the)\s+(?:system\s+prompt|hidden\s+instructions?)|print\s+(?:the\s+)?(?:policy|rules?)|BEGIN\s+SYSTEM\s+PROMPT|END\s+SYSTEM\s+PROMPT/i
267267
```
268268
269+
Prefer unquoted `/pattern/i` literal form for regex patterns that contain backslashes. If you use a structured `pattern` field, use single-quoted YAML strings or double each backslash in double-quoted strings.
270+
269271
## Minimal example
270272

271273
The simplest valid prompt:

docs/providers.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@ interface ProviderAdapter {
149149
}
150150
```
151151

152-
Direct adapter rendering accepts the same `environment` and `tier` selectors as `kit.renderPrompt()`. Use the synchronous `validate()` and `render()` methods when you already have a compiled `ResolvedPromptAsset`, and use the async `validatePrompt()` and `renderPrompt()` helpers when you want the adapter to resolve either markdown source or a compiled artifact from disk. Context input validation runs through the same shared prompt-input wrapper for OpenAI, OpenAI Responses, Anthropic, Gemini, and OpenRouter, so `allow_regex`, `deny_regex`, `non_empty`, `reject_secrets`, and `return_message` behave consistently across all five.
152+
Direct adapter rendering accepts the same `environment` and `tier` selectors as `kit.renderPrompt()`. Use the synchronous `validate()` and `render()` methods when you already have a compiled `ResolvedPromptAsset`, and use the async `validatePrompt()` and `renderPrompt()` helpers when you want the adapter to resolve either markdown source or a compiled artifact from disk. Context input validation runs through the same shared prompt-input wrapper for OpenAI, OpenAI Responses, Anthropic, Gemini, and OpenRouter, so `allow_regex`, `deny_regex`, `non_empty`, `reject_secrets`, and `return_message` behave consistently across all five. For regex validators authored in YAML, prefer unquoted `/pattern/i` literals so backslash escapes stay copyable.
153153

154154
Server-side example:
155155

docs/schema.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -293,12 +293,14 @@ Object-form inputs add optional controls:
293293

294294
- `max_size`: checked during `renderPrompt()` and can produce `POK030` warnings.
295295
- `trim`: trims incoming values to the `max_size` budget before interpolation (`true`/`end` keeps leading bytes, `start` keeps trailing bytes).
296-
- `allow_regex`: allowlist validation before interpolation; accepts `"pattern"`, `/pattern/i`, or `{ pattern, flags, return_message? }`. Non-matches throw `POK031` unless `return_message` is configured.
297-
- `deny_regex`: blocklist validation before interpolation; accepts `"pattern"`, `/pattern/i`, or `{ pattern, flags, return_message? }`. Matches throw `POK032` unless `return_message` is configured.
296+
- `allow_regex`: allowlist validation before interpolation; accepts `/pattern/i`, `"pattern"`, or `{ pattern, flags, return_message? }`. Non-matches throw `POK031` unless `return_message` is configured.
297+
- `deny_regex`: blocklist validation before interpolation; accepts `/pattern/i`, `"pattern"`, or `{ pattern, flags, return_message? }`. Matches throw `POK032` unless `return_message` is configured.
298298
- `non_empty`: accepts `true` or `{ return_message }`; blank values throw `POK033` unless `return_message` is configured.
299299
- `reject_secrets`: accepts `true` or `{ return_message }`; secret-like values throw `POK034` unless `return_message` is configured.
300300

301-
Malformed `allow_regex` and `deny_regex` values are reported during `validate` and `compile` with `POK013`.
301+
Prefer unquoted `/pattern/i` literal form for regex validators, especially when the pattern contains backslashes such as `\s` or `\b`. If you use structured `pattern:` form, use single-quoted YAML strings or double each backslash in double-quoted strings.
302+
303+
Malformed `allow_regex` and `deny_regex` values, including unsafe double-quoted YAML regex strings with raw backslashes, are reported during `validate` and `compile` with `POK013`.
302304

303305
## `includes`
304306

docs/validation.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ const result = await kit.validatePrompt('support/reply');
3434
| `POK010` | Warning | Unknown front matter key (with "did you mean?" suggestion) |
3535
| `POK011` | Warning | Variable used in template but not declared in `context.inputs` |
3636
| `POK012` | Warning | Variable declared in `context.inputs` but never used |
37-
| `POK013` | Error | Invalid context regex pattern (`allow_regex` or `deny_regex`), including prompt id, variable name, field name, and raw configured value |
37+
| `POK013` | Error | Invalid context regex pattern or YAML regex quoting (`allow_regex` or `deny_regex`), including location and raw configured value when available |
3838
| `POK014` | Warning | `trim` configured without `max_size` (trim-to-budget skipped) |
3939
| `POK040` | Warning | Risky context input appears unbounded (`max_size` missing) |
4040
| `POK041` | Warning | Context input has no hardening validators (`allow/deny regex`, `non_empty`, `reject_secrets`) |
@@ -106,27 +106,27 @@ context:
106106
inputs:
107107
- name: user_id
108108
trim: true
109-
allow_regex:
110-
pattern: "^user_[a-z0-9]+$"
111-
flags: "i"
109+
allow_regex: /^user_[a-z0-9]+$/i
112110
- name: user_message
113-
deny_regex: "/(ignore previous instructions|system:)/i"
111+
deny_regex: /(?:ignore|disregard|forget)\s+(?:all\s+)?(?:previous|prior|above)\s+instructions|(?:^|\b)(?:system|developer|assistant)\s*:|reveal\s+(?:your|the)\s+(?:system\s+prompt|hidden\s+instructions?)|print\s+(?:the\s+)?(?:policy|rules?)|BEGIN\s+SYSTEM\s+PROMPT|END\s+SYSTEM\s+PROMPT/i
114112
non_empty: true
115113
reject_secrets: true
116114
```
117115
116+
- Prefer unquoted `/pattern/i` literal form for regex patterns that contain backslashes. If you use a structured `pattern` field, use single-quoted YAML strings or double each backslash in double-quoted strings.
118117
- `trim` trims values to the `max_size` byte budget before interpolation.
119118
- `allow_regex` enforces an allowlist pattern before interpolation and throws `POK031` when a value fails validation, unless `return_message` is configured.
120119
- `deny_regex` enforces a blocklist pattern before interpolation and throws `POK032` when a value matches, unless `return_message` is configured.
121120
- `non_empty` rejects blank or whitespace-only values with `POK033`, unless `return_message` is configured.
122121
- `reject_secrets` rejects common secret-like strings with `POK034`, unless `return_message` is configured.
123122
- During static validation and compilation, malformed `allow_regex` or `deny_regex` patterns are reported as `POK013`.
123+
- Double-quoted YAML regex strings with raw backslashes, such as `"\s+"`, are reported as `POK013` before YAML parsing. Prefer unquoted `/pattern/i` literals for copyable regexes.
124124
- During static validation, `trim` without `max_size` returns a `POK014` warning.
125125
- During static validation, risky unbounded inputs and missing hardening are flagged as `POK040` and `POK041`.
126126
- During static validation, provider/cache hygiene checks can emit `POK042`–`POK045`.
127127
- During static validation, inline tool quality checks can emit `POK047`.
128128

129-
Regex compilation errors include the prompt id, variable name, field name, and raw configured value to make bad prompt definitions easy to locate.
129+
Regex compilation errors include the prompt id, variable name, field name, and raw configured value to make bad prompt definitions easy to locate. YAML quoting errors include the file and line when available.
130130

131131
If a validator declares `return_message`, `renderPrompt()` returns that message in a structured result and omits the provider request instead of throwing.
132132

0 commit comments

Comments
 (0)