Skip to content

Commit ba79741

Browse files
authored
Merge pull request #149 from fluent-plugins-nursery/docs
docs: clarify timeout handling to prevent silent log loss
2 parents 602a134 + d933bdc commit ba79741

1 file changed

Lines changed: 34 additions & 34 deletions

File tree

README.md

Lines changed: 34 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -124,40 +124,6 @@ Specify first line of multiline by regular expression.
124124
</filter>
125125
```
126126

127-
You can handle timeout events and remaining buffers on shutdown this plugin.
128-
129-
```aconf
130-
<label @ERROR>
131-
<match docker.log>
132-
@type file
133-
path /path/to/error.log
134-
</match>
135-
</label>
136-
```
137-
138-
Handle timeout log lines the same as normal logs.
139-
140-
```aconf
141-
<filter **>
142-
@type concat
143-
key message
144-
multiline_start_regexp /^Start/
145-
flush_interval 5
146-
timeout_label @NORMAL
147-
</filter>
148-
149-
<match **>
150-
@type relabel
151-
@label @NORMAL
152-
</match>
153-
154-
<label @NORMAL>
155-
<match **>
156-
@type stdout
157-
</match>
158-
</label>
159-
```
160-
161127
Handle single line JSON from Docker containers.
162128

163129
```aconf
@@ -252,6 +218,40 @@ Handle containerd/cri in Kubernetes.
252218
</label>
253219
```
254220

221+
### Timeout Handling and Preventing Log Loss
222+
223+
When a multiline log is incomplete and reaches the `flush_interval`, the plugin flushes the remaining buffer. **These timeout events are routed to the `@ERROR` label by default**.
224+
225+
**If you do not explicitly handle these timeout events, your last log lines will be silently dropped.**
226+
227+
To prevent data loss and handle timeout logs the same as normal logs, you **must** configure `timeout_label` and set up a corresponding `<label>` block to catch and route them properly.
228+
229+
#### Example: Safely rescuing timeout logs
230+
231+
```aconf
232+
<filter **>
233+
@type concat
234+
key message
235+
multiline_start_regexp /^Start/
236+
flush_interval 5
237+
# 1. Route timeout events to a specific label
238+
timeout_label @NORMAL
239+
</filter>
240+
241+
# 2. Route normal events to the same label
242+
<match **>
243+
@type relabel
244+
@label @NORMAL
245+
</match>
246+
247+
# 3. Handle both normal and timeout events together
248+
<label @NORMAL>
249+
<match **>
250+
@type stdout
251+
</match>
252+
</label>
253+
```
254+
255255
## Contributing
256256

257257
1. Fork it

0 commit comments

Comments
 (0)