Skip to content

Commit 405f39c

Browse files
committed
README: add feature and limitation
Signed-off-by: Daijiro Fukuda <fukuda@clear-code.com>
1 parent 300635e commit 405f39c

1 file changed

Lines changed: 52 additions & 0 deletions

File tree

README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,58 @@ Especially, if the `in_tail` uses [follow_inodes](https://docs.fluentd.org/input
243243
244244
Please consider updating Fluentd if you are using any version older than Fluentd v1.16.2 (td-agent v4.5.1).
245245

246+
## Feature
247+
248+
`tailcheck` performs the following checks on the specified pos file.
249+
250+
* duplication check
251+
* collection ratio check
252+
253+
### Duplication check
254+
255+
`tailcheck` checks whether there is any dulication in the specified pos files.
256+
257+
The keys of watching pos entries in one pos file must be unique.
258+
259+
By default, the key is the path of the target log file.
260+
When using [follow_inodes](https://docs.fluentd.org/input/tail#follow_inodes), the key is the inode.
261+
262+
If duplication occurs, it means some anomalies occurs in that `in_tail`.
263+
Especially, the following log missing issue causes this duplication.
264+
265+
> * https://github.com/fluent/fluentd/issues/3614
266+
> * In case `follow_inodes false` (default setting), collection of a file may stop and continue to stop after log rotation.
267+
> * Fixed since Fluentd v1.16.3 (fluent-package v5.0.2, td-agent v4.5.2).
268+
269+
### Collection ratio check
270+
271+
`tailcheck` checks collection ratio of each watching pos entry.
272+
273+
By default, `tailcheck` detects log files that have not been collected up to 50% of the filesize.
274+
You can change this threshold by `--ratio` option.
275+
Please see `Usage` section for details.
276+
277+
If this it too low, collection of those files may not be keeping up.
278+
Or it may have stopped with some anomalies.
279+
280+
Especially, if the `in_tail` uses [follow_inodes](https://docs.fluentd.org/input/tail#follow_inodes), there is a known log missing issue that was fixed in Fluentd v1.16.2.
281+
282+
> * https://github.com/fluent/fluentd/issues/4190
283+
> * In case `follow_inodes true`, collection of a file may stop and continue to stop after log rotation.
284+
> * Fixed since Fluentd v1.16.2 (fluent-package v5.0.0, td-agent v4.5.1).
285+
286+
#### Limitation
287+
288+
`tailcheck` checks the sizes of logfiles based on the paths recorded in the pos files (except already unwatched pos entries).
289+
290+
However, if using `--follow_inodes`, there is a restriction on which files can be checked.
291+
If using `--follow_inodes`, it is possible that `tailcheck` can not check the already rotated logfiles even if they are recorded in the pos files and are not unwatched yet.
292+
Since the path recorded in the pos file is not updated after log rotation, the current path and inode may differ.
293+
`tailcheck` checks only log files whose path and inode in the pos file match.
294+
At least, it can check the current log files.
295+
296+
If not using `--follow_inodes`, this limitation does not exist.
297+
246298
## Development
247299

248300
### Quick start

0 commit comments

Comments
 (0)