Skip to content

Commit 9800b1e

Browse files
committed
Document queue memory considerations
1 parent 50cd81b commit 9800b1e

1 file changed

Lines changed: 29 additions & 0 deletions

File tree

README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,35 @@ $entities = PrivacyFilter::entities(
148148
);
149149
```
150150

151+
## Queueing And Memory
152+
153+
Privacy Filter runs the native `privacy-filter.cpp` binary when classifying text. The GGUF model is loaded into memory by the native process while classification is running. Larger models require more memory.
154+
155+
For production workloads, you should avoid running many classifications concurrently from normal web requests. Instead, dispatch classification work to a dedicated queue and limit the number of workers assigned to that queue based on the memory available on your server.
156+
157+
For example, if your selected model uses approximately 3 GB of memory while classifying, running four concurrent workers may require approximately 12 GB of memory, plus memory used by PHP, Redis, your database, and the rest of your application.
158+
159+
A typical Horizon configuration may dedicate a small worker pool to privacy filtering:
160+
161+
```php
162+
'privacy-filter' => [
163+
'connection' => 'redis',
164+
'queue' => ['privacy-filter'],
165+
'balance' => 'simple',
166+
'processes' => 2,
167+
'tries' => 1,
168+
'timeout' => 300,
169+
],
170+
```
171+
172+
You may then dispatch redaction or classification work to the dedicated queue:
173+
174+
```php
175+
RedactTranscript::dispatch($transcript)->onQueue('privacy-filter');
176+
```
177+
178+
Tune the `processes` value according to your server memory and model size. Each concurrent classification may load its own copy of the model into memory.
179+
151180
## Entity Types
152181

153182
The raw entity type is available through the entity's `type` property:

0 commit comments

Comments
 (0)