Skip to content

Commit 9a86ad1

Browse files
feat: add llm_tag documentation
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
1 parent b405125 commit 9a86ad1

4 files changed

Lines changed: 170 additions & 1 deletion

File tree

docs/agent/features/index.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,7 @@ This section documents advanced features and capabilities of Fluent Bit.
1010

1111
- **[Record Deduplication](record-deduplication.md)** - Remove duplicate log records based on configurable keys
1212
- **[Log Sampling](log-sampling.md)** - Sample log data to reduce volume while maintaining statistical accuracy
13+
14+
## AI
15+
16+
- **[LLM auto-tagging](llm-tagging.md)** - Automatically tag records using an LLM prompt to easily route data

docs/agent/features/llm-tagging.md

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# LLM Tag Filter Plugin
2+
3+
The `llm_tag` filter plugin uses a Large Language Model (LLM) to **classify log records** and **route them by rewriting their tag** based on natural-language conditions. Instead of writing complex regex rules or static matchers, you describe what you're looking for in plain English, and the LLM decides which records match.
4+
5+
It extracts the log payload (looking for `log` or `message` keys) and securely queries an OpenAI-compatible API using natural language prompts. If the LLM determines that a record matches a specific prompt classification, the plugin emits a copy of the record with a newly assigned tag.
6+
7+
This is highly useful for semantic log routing, such as identifying security anomalies, analyzing sentiment, or catching obscure errors without writing complex regular expressions.
8+
9+
This plugin is OpenAI API-compatible and works with OpenAI, Azure OpenAI, vLLM, Ollama (with the OpenAI-compatible endpoint), LM Studio, and any other server that exposes an OpenAI-compatible `/chat/completions` endpoint.
10+
11+
## How It Works
12+
13+
1. For each incoming record, the filter extracts the `log` or `message` field.
14+
2. It builds a **batch prompt** that includes every configured rule's prompt, and sends a single request to the configured LLM.
15+
3. The LLM responds with one `yes`/`no` decision per rule (e.g. `1: yes\n2: no\n3: yes`).
16+
4. For each rule that matched, the record is re-emitted with the rule's target tag through a shared internal emitter input.
17+
5. Depending on `keep_record` and `tags_match_mode`, the original record is either dropped, kept, or evaluation stops at the first match.
18+
19+
> **Note:** Because the plugin re-emits records through an internal `emitter` input, downstream `[OUTPUT]` sections can match the new tags directly with the `Match` directive.
20+
21+
<details>
22+
<summary><strong>Architecture details (click to expand)</strong></summary>
23+
24+
- A **single shared emitter** is created at init time (named `emitter_for_<filter_name>` by default), used for all re-emissions. This avoids creating one emitter per rule.
25+
- Records are batched into **one LLM call per log record** (not one per rule), which dramatically reduces latency and cost when you have many rules.
26+
- The plugin protects against **infinite loops** by skipping records that come from its own emitter, and it pauses cleanly when the emitter buffer is full or during shutdown (records are passed through untouched).
27+
- On any LLM failure (timeout, network error, parse failure), the original record is **always preserved** to prevent data loss.
28+
29+
</details>
30+
31+
## Configuration Parameters
32+
33+
| Key | Description | Default |
34+
|---|---|---|
35+
| `model_endpoint` | **Required.** The LLM HTTP endpoint URL, e.g. `https://api.openai.com/v1/chat/completions`. | _(none)_ |
36+
| `model_id` | **Required.** The LLM model identifier sent in the request body (e.g. `gpt-4o-mini`, `llama3.1:8b`). | _(none)_ |
37+
| `model_api_key` | API key for authentication. If relying on a local/unauthenticated model, this can be omitted. | _(none)_ |
38+
| `model_timeout` | HTTP request timeout in milliseconds for the LLM API call. | `1000` |
39+
| `tags` | **Required.** The classification rules array. Each item must be an object containing `tag` (the new tag to apply) and `prompt` (the natural language condition). | _(none)_ |
40+
| `tags_match_mode` | `first` to stop at the first matching rule, or `all` to evaluate every rule and emit one record per match. | `first` |
41+
| `keep_record` | When `true`, also keep the original record (with the original tag) after re-emission. When `false`, the original is dropped if any rule matched. | `false` |
42+
43+
### Rule structure
44+
45+
Each entry in the `tags` array is an object with two string fields:
46+
47+
| Field | Description |
48+
|---|---|
49+
| `tag` | The new tag to assign when this rule matches. |
50+
| `prompt` | A natural-language condition. The LLM will answer **yes** or **no** for this rule for every log line. |
51+
52+
## Examples
53+
54+
### Example 1: Route errors and security events with OpenAI
55+
56+
This pipeline reads from a JSON file, asks the LLM to flag error and security-related lines, and routes them to different outputs.
57+
58+
```yaml
59+
service:
60+
log_level: info
61+
62+
pipeline:
63+
inputs:
64+
- name: tail
65+
path: /var/log/app.log
66+
tag: app.raw
67+
parser: json
68+
69+
filters:
70+
- name: llm_tag
71+
match: app.raw
72+
model_endpoint: https://api.openai.com/v1/chat/completions
73+
model_id: gpt-4o-mini
74+
model_api_key: ${OPENAI_API_KEY}
75+
model_timeout: 5000
76+
tags_match_mode: all
77+
keep_record: false
78+
tags:
79+
- tag: app.errors
80+
prompt: "Does this log indicate an application error, exception, or stack trace?"
81+
- tag: app.security
82+
prompt: "Does this log describe a security event such as failed login, unauthorized access, or suspicious activity?"
83+
- tag: app.slow
84+
prompt: "Does this log indicate slow performance, high latency, or a timeout?"
85+
86+
outputs:
87+
- name: stdout
88+
match: app.errors
89+
- name: http
90+
match: app.security
91+
host: siem.internal
92+
port: 8080
93+
- name: stdout
94+
match: app.slow
95+
```
96+
97+
### Example 2: Local Ollama
98+
99+
```yaml
100+
filters:
101+
- name: llm_tag
102+
match: logs
103+
tags_match_mode: all
104+
model_endpoint: http://127.0.0.1:11434
105+
model_id: phi3:mini
106+
model_timeout: 10000
107+
model_api_key: ""
108+
keep_record: true
109+
tags:
110+
- tag: security
111+
prompt: "This log indicates a security incident or authentication failure"
112+
- tag: phishing
113+
prompt: "This log contains phishing attempt or credential request"
114+
```
115+
116+
In this example `keep_record true` means the original record under `sys.raw` is also preserved, so you'll see each matched record twice: once with its new tag and once with the original.
117+
118+
### Example 3: Match modes compared
119+
120+
Given two rules and a log that matches both:
121+
122+
| `tags_match_mode` | `keep_record` | Result |
123+
|---|---|---|
124+
| `first` | `false` | 1 record emitted with the **first** matching tag; original dropped. |
125+
| `first` | `true` | 1 record emitted with the first matching tag; original kept. |
126+
| `all` | `false` | 1 record emitted **per matching rule**; original dropped. |
127+
| `all` | `true` | 1 record emitted **per matching rule**; original kept. |
128+
129+
If **no** rule matches, the original record is always passed through unchanged regardless of these settings.
130+
131+
## Observability
132+
133+
The plugin tracks four internal counters, logged at shutdown:
134+
135+
- `requests_total` — total LLM requests issued
136+
- `requests_failed` — failed LLM requests (timeouts, errors, etc.)
137+
- `records_emitted` — records re-emitted with a new tag
138+
- `records_dropped` — original records dropped after a successful match
139+
140+
Each LLM call also logs its latency at `info` level, e.g.:
141+
142+
```
143+
[ info] [filter:llm_tag:llm_tag.0] LLM API request completed in 412.55 ms
144+
```
145+
146+
Enable `log_level: debug` on the service to see the raw and parsed LLM responses, which is useful when tuning prompts.
147+
148+
## Tips and Caveats
149+
150+
- **Latency.** Every record incurs an LLM round-trip. For high-volume streams, place `llm_tag` after a sampling or rate-limiting filter, or use a small, fast local model.
151+
- **Cost.** With a hosted API, every record is a billable request. Batch mode keeps it to **1 request per record**, regardless of how many rules you define.
152+
- **Prompt design.** Phrase prompts as **yes/no questions**. Ambiguous prompts produce inconsistent classifications. Include short, unambiguous criteria.
153+
- **Field names.** The plugin looks for the log content in either a `log` or `message` field. Records without either field are passed through unmodified.
154+
- **Failure mode.** If the LLM call fails or returns an unparseable response, the original record is preserved — you will not silently lose data.
155+
- **Output matching.** Ensure to add outputs that match the new tags you define under `tags[].tag`, otherwise re-emitted records have nowhere to go.
156+
- **Recursion.** The plugin skips records from its own emitter to avoid infinite loops.
157+
158+
## Security considerations
159+
160+
Logs are sent to the configured LLM endpoint. Do not send sensitive data to third-party services unless permitted by your security policy.
161+
162+
* Use environment variables or secrets management for `model_api_key`.
163+
* Consider redacting secrets before this filter if logs may contain credentials, tokens, personal data, or regulated information.
164+
* For sensitive environments, use a local OpenAI-compatible model endpoint.

docs/agent/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ The Telemetry Forge Agent is an **enterprise-hardened distribution of Fluent Bit
1717
- [Performant log deduplication at source](./features/record-deduplication.md)
1818
- [Log sampling processor](./features/log-sampling.md)
1919
- [Git configuration auto-reload](./features/git-config-auto-reload.md)
20-
- AI-based filtering and routing
20+
- [AI-based filtering and routing](./features/llm-tagging.md)
2121
- Tail sampling and OTTL-style logic
2222
- Efficient filesystem storage buffer
2323
- Dedicated integration and regression testing

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ nav:
1010
- Log Sampling: agent/features/log-sampling.md
1111
- Git Configuration Auto-Reload: agent/features/git-config-auto-reload.md
1212
- Hardening and Optimisation: agent/build-optimisations.md
13+
- AI filtering and routing: agent/features/llm-tagging.md
1314
- Security:
1415
- Overview: agent/security.md
1516
- CVE Information: agent/security/cves.md

0 commit comments

Comments
 (0)