You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
`token_classify` marks a model as a token-classification (NER) provider for the PII redactor's encoder tier (e.g. an `openai-privacy-filter` GGUF). Declare it explicitly together with `embeddings: true` (the classifier loads via TOKEN_CLS pooling). On the `llama-cpp` backend it must not be combined with `chat`/`completion` in the same config — `TokenClassify` bypasses the slot loop and would race generation, so the loader rejects that mix; split into separate model configs.
791
+
`token_classify` marks a model as a token-classification (NER) provider for the PII filter (e.g. an `openai-privacy-filter` GGUF). Declare it explicitly together with `embeddings: true` (the classifier loads via TOKEN_CLS pooling). On the `llama-cpp` backend it must not be combined with `chat`/`completion` in the same config — `TokenClassify` bypasses the slot loop and would race generation, so the loader rejects that mix; split into separate model configs.
792
+
793
+
## PII filtering
794
+
795
+
PII redaction is NER-based and runs on the **request** (input) side. It has two halves:
796
+
797
+
- **Detector models** are `token_classify` models that carry the detection *policy* in a top-level `pii_detection:` block. The policy is defined once, on the model itself:
798
+
799
+
```yaml
800
+
name: privacy-filter-multilingual
801
+
backend: llama-cpp
802
+
embeddings: true
803
+
known_usecases:
804
+
- token_classify
805
+
pii_detection:
806
+
min_score: 0.5 # drop detections below this confidence
807
+
default_action: mask # mask | block | allow — applied to any detected
808
+
# group with no explicit entry (empty = mask)
809
+
entity_actions: # which PII to block vs mask vs allow-log
810
+
PASSWORD: block
811
+
CREDITCARD: block
812
+
EMAIL: mask
813
+
```
814
+
815
+
- **Consuming models** opt in and reference one or more detectors by name — no per-consumer policy:
816
+
817
+
```yaml
818
+
name: my-assistant
819
+
pii:
820
+
enabled: true # default: off for local backends, on for cloud-proxy
821
+
detectors:
822
+
- privacy-filter-multilingual
823
+
```
824
+
825
+
Multiple detectors union their detections; overlapping spans resolve to the strongest action (`block` > `mask` > `allow`). A configured detector that can't be loaded fails the request closed (HTTP 503) rather than silently skipping the check. Detections are audited at `/api/pii/events` (hash-prefix only, never the raw value).
826
+
827
+
> The earlier regex pattern tier (`pii.patterns`, the global pattern catalogue, `--pii-config`, and the `/api/pii/patterns` admin endpoints) has been removed, along with response/streaming-side redaction. Those keys now no-op with a startup warning; migrate to `pii.detectors` + a detector's `pii_detection` block.
action: block # already the default, made explicit for audit
97
-
- id: email
98
-
action: allow
94
+
enabled: true # default-on for cloud-proxy; explicit for audit
95
+
detectors:
96
+
- privacy-filter-multilingual
99
97
```
100
98
101
-
The regex itself stays global — only the action is settable per-model.
102
-
Adding new patterns is a build-time concern (extend `patternRegexps` in
103
-
`core/services/routing/pii/patterns.go`).
104
-
105
-
### NER tier (optional)
106
-
107
-
The regex matcher covers high-precision patterns. For natural-language
108
-
PII (proper names, addresses, organization names) LocalAI carries an
109
-
**encoder NER tier** that runs after the regex pass. It expects a
110
-
transformers token-classification model wired through the `TokenClassify`
111
-
gRPC primitive (e.g. `dslim/bert-base-NER`). The detector annotates
112
-
spans with an entity group (`PER`, `LOC`, `ORG`, `MISC`); per-group
113
-
actions are configurable through the same `pii:` block.
114
-
115
-
The NER tier ships as a contract (`NERDetector`, `NERConfig` in
116
-
`core/services/routing/pii/ner.go`); an operator-facing knob to load and
117
-
attach a detector is not plumbed yet. When no detector is configured the
118
-
regex tier still runs.
119
-
120
-
### Streaming PII filter
121
-
122
-
Buffered (`/v1/chat/completions` without `"stream": true`) responses are
123
-
forwarded verbatim today — only the request-side scan runs. Streaming
124
-
responses run through `pii.StreamFilter` which buffers SSE chunks until
125
-
either a full pattern matches or the buffer's max length is reached,
126
-
then emits the safe prefix. The streaming filter is what makes the
127
-
cloud-proxy backend and the MITM proxy safe to expose to clients that
128
-
issue streaming requests.
129
-
130
-
The streaming filter is wired automatically for any model with `pii.enabled`
131
-
true — there is no separate streaming toggle.
99
+
Multiple detectors **union** their detections; overlapping spans resolve to
100
+
the strongest action (`block` > `mask` > `allow`). A configured detector
101
+
that can't be loaded **fails the request closed** (HTTP 503,
102
+
`error.type=pii_ner_unavailable`) rather than silently skipping the check.
103
+
The same NER path runs on the [MITM proxy]({{< relref "mitm-proxy.md" >}})
104
+
request body for intercepted hosts. Response/output redaction is out of
105
+
scope for now.
132
106
133
107
### Admin page
134
108
135
109
The `/app/middleware` page (admin role only) has four tabs — **Filtering**,
136
110
**Routing**, **MITM Proxy** (see the [MITM doc]({{< relref "mitm-proxy.md" >}})),
137
-
and **Events**. The Filtering tab shows:
138
-
139
-
- The pattern catalogue with live action dropdowns. Changing an action via
140
-
the UI calls `PUT /api/pii/patterns/:id` and updates the live redactor
141
-
in-process. Click **Persist** in the action header to write the current
142
-
state into `runtime_settings.json` so the next process start re-applies it.
143
-
- A per-model resolved-state table — each model row reports `enabled`,
144
-
the per-pattern overrides, and which patterns are effectively active.
145
-
- A live test panel that posts sample text to `/api/pii/test` and
146
-
highlights matches with their resolved actions, without storing the
147
-
text in the event log.
111
+
and **Events**. The Filtering tab shows a per-model table: each row reports
112
+
`enabled`and the detector model(s) it references, plus the recent event
113
+
count. Detection policy is edited on each detector model's config (Models →
114
+
edit → PII), not globally.
148
115
149
116
### REST surface
150
117
151
118
| Method | Path | Auth | Purpose |
152
119
|---|---|---|---|
153
-
| GET | `/api/pii/patterns` | any | Live pattern list with current actions. Used by the UI catalogue. |
154
-
| POST | `/api/pii/test` | any | Dry-run the redactor on `{"text":"..."}`. Returns hits and the would-be-rewritten body. Does not write to the event log. |
| PUT | `/api/pii/patterns/:id` | admin | Update a pattern in-process. Body accepts `{"action":"mask"\|"block"\|"allow"}` and/or `{"disabled":true\|false}`. Transient — reverts on restart unless persisted. |
157
-
| POST | `/api/pii/patterns/persist` | admin | Snapshot the live per-pattern (action, disabled) state into `runtime_settings.json`. |
158
-
| GET | `/api/middleware/status` | admin | Aggregated dashboard data: patterns + per-model resolved state + router status + MITM status + admission status. One round-trip for the UI. |
| GET | `/api/middleware/status` | admin | Aggregated dashboard data: per-model PII state + detectors + router status + MITM status + admission status. One round-trip for the UI. |
159
122
160
123
### MCP tools
161
124
162
-
The same surface is mirrored through the LocalAI Assistant MCP server so
163
-
the in-process and stdio assistants can manage the filter conversationally:
125
+
The same surface is mirrored through the LocalAI Assistant MCP server:
164
126
165
127
| Tool | Read/Write | Purpose |
166
128
|---|---|---|
167
-
| `list_pii_patterns` | read | Returns the live pattern list. |
0 commit comments