You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+52Lines changed: 52 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -243,6 +243,58 @@ Especially, if the `in_tail` uses [follow_inodes](https://docs.fluentd.org/input
243
243
244
244
Please consider updating Fluentd if you are using any version older than Fluentd v1.16.2 (td-agent v4.5.1).
245
245
246
+
## Feature
247
+
248
+
`tailcheck` performs the following checks on the specified pos file.
249
+
250
+
* duplication check
251
+
* collection ratio check
252
+
253
+
### Duplication check
254
+
255
+
`tailcheck` checks whether there is any dulication in the specified pos files.
256
+
257
+
The keys of watching pos entries in one pos file must be unique.
258
+
259
+
By default, the key is the path of the target log file.
260
+
When using [follow_inodes](https://docs.fluentd.org/input/tail#follow_inodes), the key is the inode.
261
+
262
+
If duplication occurs, it means some anomalies occurs in that `in_tail`.
263
+
Especially, the following log missing issue causes this duplication.
264
+
265
+
> *https://github.com/fluent/fluentd/issues/3614
266
+
> * In case `follow_inodes false` (default setting), collection of a file may stop and continue to stop after log rotation.
267
+
> * Fixed since Fluentd v1.16.3 (fluent-package v5.0.2, td-agent v4.5.2).
268
+
269
+
### Collection ratio check
270
+
271
+
`tailcheck` checks collection ratio of each watching pos entry.
272
+
273
+
By default, `tailcheck` detects log files that have not been collected up to 50% of the filesize.
274
+
You can change this threshold by `--ratio` option.
275
+
Please see `Usage` section for details.
276
+
277
+
If this it too low, collection of those files may not be keeping up.
278
+
Or it may have stopped with some anomalies.
279
+
280
+
Especially, if the `in_tail` uses [follow_inodes](https://docs.fluentd.org/input/tail#follow_inodes), there is a known log missing issue that was fixed in Fluentd v1.16.2.
281
+
282
+
> *https://github.com/fluent/fluentd/issues/4190
283
+
> * In case `follow_inodes true`, collection of a file may stop and continue to stop after log rotation.
284
+
> * Fixed since Fluentd v1.16.2 (fluent-package v5.0.0, td-agent v4.5.1).
285
+
286
+
#### Limitation
287
+
288
+
`tailcheck` checks the sizes of logfiles based on the paths recorded in the pos files (except already unwatched pos entries).
289
+
290
+
However, if using `--follow_inodes`, there is a restriction on which files can be checked.
291
+
If using `--follow_inodes`, it is possible that `tailcheck` can not check the already rotated logfiles even if they are recorded in the pos files and are not unwatched yet.
292
+
Since the path recorded in the pos file is not updated after log rotation, the current path and inode may differ.
293
+
`tailcheck` checks only log files whose path and inode in the pos file match.
294
+
At least, it can check the current log files.
295
+
296
+
If not using `--follow_inodes`, this limitation does not exist.
0 commit comments