Skip to content

Commit 583fab4

Browse files
authored
feat(core): add auto-redaction for PII protection (#271)
1 parent 79cb4a4 commit 583fab4

18 files changed

Lines changed: 1413 additions & 5 deletions

File tree

.changeset/auto-redaction-pii.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'evlog': minor
3+
---
4+
5+
Add auto-redaction (PII protection) with smart partial masking, enabled by default in production (`NODE_ENV === 'production'`). Built-in patterns (credit card, email, IPv4, phone, JWT, Bearer, IBAN) use context-preserving masks (e.g. `****1111`, `a***@***.com`) instead of flat `[REDACTED]`. Disabled in development for full debugging visibility. Fine-tune with `paths`, `patterns`, and `builtins`, or opt out with `redact: false`. Custom patterns use the configurable `replacement` string. Redaction runs before console output and before any drain sees the data.

AGENTS.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -979,6 +979,75 @@ export function maskCard(card: string): string {
979979
- [ ] PII is masked or omitted
980980
- [ ] Request bodies are selectively logged (not `log.set({ body })`)
981981

982+
### Auto-Redaction (PII Protection)
983+
984+
Built-in redaction scrubs sensitive data from wide events **before** console output and **before** any drain sees the data. **Enabled by default in production** (`NODE_ENV === 'production'`), disabled in development for full debugging visibility. Override with `redact: false` to disable or `redact: { ... }` for fine-grained control.
985+
986+
**Built-in patterns** (enabled by default with `redact: true`) use **smart partial masking** — preserving enough context for debugging while protecting PII:
987+
988+
| Name | Detects | Masked Output |
989+
|------|---------|---------------|
990+
| `creditCard` | Credit card numbers | `****1111` (last 4 digits) |
991+
| `email` | Email addresses | `a***@***.com` (first char + TLD) |
992+
| `ipv4` | IPv4 addresses (excludes 127.0.0.1, 0.0.0.0) | `***.***.***.1` (last octet) |
993+
| `phone` | International phone numbers | `+33 ****5678` (last 4 digits) |
994+
| `jwt` | JWT tokens (eyJhbG...) | `eyJ***.***` (prefix only) |
995+
| `bearer` | Bearer tokens (Bearer sk_live_...) | `Bearer ***` (type only) |
996+
| `iban` | International Bank Account Numbers | `FR76****189` (country + check + last 3) |
997+
998+
Custom `patterns` still use the flat `replacement` string (default `[REDACTED]`).
999+
1000+
```typescript
1001+
// Simplest: enable all built-in PII patterns
1002+
evlog: { redact: true }
1003+
1004+
// Add custom paths on top of built-ins
1005+
evlog: {
1006+
redact: {
1007+
paths: ['user.password', 'headers.authorization'],
1008+
}
1009+
}
1010+
1011+
// Only specific built-ins
1012+
evlog: {
1013+
redact: {
1014+
builtins: ['email', 'creditCard'],
1015+
}
1016+
}
1017+
1018+
// No built-ins, only custom
1019+
evlog: {
1020+
redact: {
1021+
builtins: false,
1022+
paths: ['user.ssn'],
1023+
patterns: [/SECRET_\w+/g],
1024+
}
1025+
}
1026+
```
1027+
1028+
```typescript
1029+
// Standalone
1030+
import { initLogger } from 'evlog'
1031+
1032+
initLogger({ redact: true })
1033+
```
1034+
1035+
**Configuration:**
1036+
1037+
| Field | Type | Default | Description |
1038+
|-------|------|---------|-------------|
1039+
| `redact` | `boolean \| RedactConfig` | `true` in production | Enabled by default in production. `false` to disable. Object for fine-grained control |
1040+
| `paths` | `string[]` | `undefined` | Dot-notation paths to redact (e.g. `user.email`) |
1041+
| `patterns` | `RegExp[]` | `undefined` | Additional regex patterns applied to all string values recursively (uses flat `replacement`) |
1042+
| `builtins` | `false \| BuiltinPatternName[]` | all enabled | `false` disables built-ins; array selects specific ones (`'creditCard'`, `'email'`, `'ipv4'`, `'phone'`, `'jwt'`, `'bearer'`, `'iban'`) |
1043+
| `replacement` | `string` | `'[REDACTED]'` | Replacement string for path-based and custom pattern redaction. Built-in patterns use smart partial masking instead |
1044+
1045+
**Architecture:** Redaction runs inside `emitWideEvent()` after the `WideEvent` is built but before console output and `globalDrain`. This is the single chokepoint for all events. Framework middleware (`BaseEvlogOptions.redact`) also applies redaction before enrich/drain as a safety net.
1046+
1047+
**Source:** `packages/evlog/src/redact.ts``redactEvent()` (path-based + masker-based + pattern-based), `resolveRedactConfig()` (resolves `true`/object into concrete config with `_maskers` for built-in smart masking), `normalizeRedactConfig()` (deserializes from JSON for Nitro env bridge).
1048+
1049+
**Serialization note:** RegExp patterns from Nuxt config are serialized to JSON via `process.env.__EVLOG_CONFIG`. The Nitro plugin uses `normalizeRedactConfig()` to reconstruct RegExp instances from `{ source, flags }` objects or plain strings.
1050+
9821051
### Client-Side Logging
9831052

9841053
The `log` API also works on the client side (auto-imported in Nuxt):

apps/docs/content/3.core-concepts/1.configuration.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ initLogger({
4747
| `stringify` | `boolean` | `true` | Emit JSON strings when `pretty` is disabled. Set to `false` for Cloudflare Workers |
4848
| `minLevel` | `'debug' \| 'info' \| 'warn' \| 'error'` | `'debug'` | Minimum severity for the global `log` API only (not `createLogger` / request wide events). Order: debug < info < warn < error |
4949
| `sampling` | `SamplingConfig` | `undefined` | Head and tail sampling configuration. See [Sampling](/core-concepts/sampling) |
50+
| `redact` | `boolean \| RedactConfig` | `true` in production | Enabled by default in production. `false` to disable. Object for fine-grained control. See [Auto-Redaction](/core-concepts/redaction) |
5051
| `drain` | `(ctx: DrainContext) => void` | `undefined` | Drain callback for sending events to external services |
5152

5253
### `minLevel` vs sampling

apps/docs/content/3.core-concepts/4.best-practices.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,22 @@ Wide events are powerful because they capture comprehensive context. However, th
2929
Logs are often accessible to your entire team and may be stored in third-party services. Treat them as semi-public.
3030
::
3131

32+
## Auto-Redaction
33+
34+
The simplest way to protect PII is to enable built-in auto-redaction:
35+
36+
```typescript [nuxt.config.ts]
37+
evlog: {
38+
redact: true,
39+
}
40+
```
41+
42+
This automatically masks credit cards (`****1111`), emails (`a***@***.com`), IPs, phone numbers, JWTs, Bearer tokens, and IBANs in all wide events — before console output and before any drain. See [Auto-Redaction](/core-concepts/redaction) for the full configuration reference.
43+
44+
::callout{icon="i-lucide-shield-check" color="success"}
45+
Auto-redaction is a safety net, not a replacement for careful logging. Always prefer explicit field selection and combine with `redact: true` for defense in depth.
46+
::
47+
3248
## Sanitization Patterns
3349

3450
### Manual Field Selection
@@ -167,6 +183,7 @@ Before deploying to production, verify:
167183

168184
### Data Security
169185

186+
- [ ] Auto-redaction is enabled (`redact: true`)
170187
- [ ] No passwords or secrets in logs
171188
- [ ] No full credit card numbers (only last 4 digits)
172189
- [ ] No API keys or tokens
@@ -243,5 +260,6 @@ Use `$production` override to keep full logging in development while sampling in
243260

244261
## Next Steps
245262

263+
- [Auto-Redaction](/core-concepts/redaction) - Built-in PII protection with smart masking
246264
- [Wide Events](/logging/wide-events) - Design effective wide events
247265
- [Structured Errors](/logging/structured-errors) - Error handling patterns
Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
---
2+
title: Auto-Redaction
3+
description: Automatically scrub PII from wide events before console output and drains. Built-in smart masking for credit cards, emails, IPs, phone numbers, JWTs, and more.
4+
navigation:
5+
icon: i-lucide-eye-off
6+
links:
7+
- label: Best Practices
8+
icon: i-lucide-shield-check
9+
to: /core-concepts/best-practices
10+
color: neutral
11+
variant: subtle
12+
- label: Configuration
13+
icon: i-lucide-settings
14+
to: /core-concepts/configuration
15+
color: neutral
16+
variant: subtle
17+
---
18+
19+
Wide events capture comprehensive context, which makes it easy to accidentally log sensitive data. Auto-redaction scrubs PII from events **before** console output and **before** any drain sees the data.
20+
21+
**Redaction is enabled by default in production** (`NODE_ENV === 'production'`). In development, it is off so you see full values for debugging. No configuration needed — just deploy.
22+
23+
## Opting Out
24+
25+
If you need to disable redaction in production:
26+
27+
::code-group
28+
```typescript [nuxt.config.ts]
29+
export default defineNuxtConfig({
30+
modules: ['evlog/nuxt'],
31+
evlog: {
32+
redact: false,
33+
},
34+
})
35+
```
36+
```typescript [lib/evlog.ts (Next.js)]
37+
import { createEvlog } from 'evlog/next'
38+
39+
export const { withEvlog, useLogger } = createEvlog({
40+
service: 'my-app',
41+
redact: false,
42+
})
43+
```
44+
```typescript [index.ts (Hono / Express / Fastify)]
45+
import { initLogger } from 'evlog'
46+
47+
initLogger({
48+
env: { service: 'my-app' },
49+
redact: false,
50+
})
51+
```
52+
::
53+
54+
You can also enable redaction explicitly in development with `redact: true`.
55+
56+
## Smart Masking
57+
58+
Built-in patterns use **partial masking** instead of flat `[REDACTED]` — preserving enough context for debugging while protecting the actual data.
59+
60+
| Pattern | Example Input | Masked Output |
61+
|---------|---------------|---------------|
62+
| `creditCard` | `4111111111111111` | `****1111` |
63+
| `email` | `alice@example.com` | `a***@***.com` |
64+
| `ipv4` | `192.168.1.100` | `***.***.***.100` |
65+
| `phone` | `+33 6 12 34 56 78` | `+33 ****5678` |
66+
| `jwt` | `eyJhbGciOiJIUzI1NiIs...` | `eyJ***.***` |
67+
| `bearer` | `Bearer sk_live_abc123...` | `Bearer ***` |
68+
| `iban` | `FR76 3000 6000 0112 ...189` | `FR76****189` |
69+
70+
::callout{icon="i-lucide-info" color="info"}
71+
`127.0.0.1` and `0.0.0.0` are excluded from IPv4 masking since they are not real client addresses.
72+
::
73+
74+
## Configuration
75+
76+
### Custom Paths
77+
78+
Add dot-notation paths to redact specific fields with `[REDACTED]`, on top of the built-in patterns:
79+
80+
```typescript
81+
evlog: {
82+
redact: {
83+
paths: ['user.password', 'headers.authorization'],
84+
}
85+
}
86+
```
87+
88+
Path-based redaction replaces the **entire value** with the `replacement` string (default `[REDACTED]`), regardless of content.
89+
90+
### Selective Built-ins
91+
92+
Pick only the patterns you need:
93+
94+
```typescript
95+
evlog: {
96+
redact: {
97+
builtins: ['email', 'creditCard'],
98+
}
99+
}
100+
```
101+
102+
### Custom Patterns
103+
104+
Add your own regex patterns. These use the flat `replacement` string, not smart masking:
105+
106+
```typescript
107+
evlog: {
108+
redact: {
109+
patterns: [/SECRET_\w+/g, /sk_live_\w+/g],
110+
replacement: '***',
111+
}
112+
}
113+
```
114+
115+
### Disable Built-ins
116+
117+
If you only want custom redaction:
118+
119+
```typescript
120+
evlog: {
121+
redact: {
122+
builtins: false,
123+
paths: ['user.ssn'],
124+
patterns: [/INTERNAL_\w+/g],
125+
}
126+
}
127+
```
128+
129+
## Configuration Reference
130+
131+
| Option | Type | Default | Description |
132+
|--------|------|---------|-------------|
133+
| `redact` | `boolean \| RedactConfig` | `true` in production | Enabled by default in production. `false` to disable. Object for fine-grained control |
134+
| `paths` | `string[]` | `undefined` | Dot-notation paths to redact entirely (e.g. `user.password`) |
135+
| `patterns` | `RegExp[]` | `undefined` | Custom regex patterns. Uses flat `replacement` string |
136+
| `builtins` | `false \| string[]` | All enabled | `false` disables built-ins. Array selects specific ones |
137+
| `replacement` | `string` | `'[REDACTED]'` | Replacement string for paths and custom patterns. Built-in patterns use smart masking instead |
138+
139+
Available built-in names: `creditCard`, `email`, `ipv4`, `phone`, `jwt`, `bearer`, `iban`.
140+
141+
## How It Works
142+
143+
Redaction runs inside the emit pipeline, after the wide event is fully built but before any output:
144+
145+
1. **Path redaction** — targeted fields replaced with `[REDACTED]`
146+
2. **Smart masking** — built-in patterns scan all string values recursively with partial masking
147+
3. **Pattern redaction** — custom regex patterns scan all string values with flat replacement
148+
4. **Console output** — masked event printed to stdout
149+
5. **Drain** — masked event sent to external services
150+
151+
::callout{icon="i-lucide-zap" color="info"}
152+
Redaction runs **after** the HTTP response is sent, so it adds zero latency to your API responses.
153+
::
154+
155+
## Production Example
156+
157+
Redaction is already on by default in production. Combine with sampling for a typical setup:
158+
159+
::code-group
160+
```typescript [nuxt.config.ts]
161+
export default defineNuxtConfig({
162+
modules: ['evlog/nuxt'],
163+
evlog: {
164+
env: { service: 'my-app' },
165+
},
166+
$production: {
167+
evlog: {
168+
sampling: {
169+
rates: { info: 10, debug: 0 },
170+
keep: [{ status: 400 }, { duration: 1000 }],
171+
},
172+
},
173+
},
174+
})
175+
```
176+
```typescript [lib/evlog.ts (Next.js)]
177+
import { createEvlog } from 'evlog/next'
178+
179+
export const { withEvlog, useLogger } = createEvlog({
180+
service: 'my-app',
181+
sampling: {
182+
rates: { info: 10, debug: 0 },
183+
keep: [{ status: 400 }, { duration: 1000 }],
184+
},
185+
})
186+
```
187+
```typescript [index.ts (Hono / Express / Fastify)]
188+
import { initLogger } from 'evlog'
189+
190+
initLogger({
191+
env: { service: 'my-app' },
192+
sampling: {
193+
rates: { info: 10, debug: 0 },
194+
keep: [{ status: 400 }, { duration: 1000 }],
195+
},
196+
})
197+
```
198+
::
199+
200+
## Before / After
201+
202+
Without redaction, sensitive data lands in your logs and drains:
203+
204+
```json
205+
{
206+
"user": { "email": "alice@example.com", "ip": "192.168.1.42" },
207+
"payment": { "card": "4111111111111111" },
208+
"auth": "Bearer sk_live_abc123def456"
209+
}
210+
```
211+
212+
With `redact: true`:
213+
214+
```json
215+
{
216+
"user": { "email": "a***@***.com", "ip": "***.***.***.42" },
217+
"payment": { "card": "****1111" },
218+
"auth": "Bearer ***"
219+
}
220+
```
221+
222+
Same debugging context, no PII in your Axiom/Datadog/Sentry.
223+
224+
## Next Steps
225+
226+
- [Best Practices](/core-concepts/best-practices) - Security guidelines and production checklist
227+
- [Sampling](/core-concepts/sampling) - Control log volume in production
228+
- [Configuration](/core-concepts/configuration) - Full configuration reference

0 commit comments

Comments
 (0)