Problem
When a colon (:) is in the tag, and a timeout flush occurs, then everything past the colon is lost.
Steps to replicate
-
On windows, configure a in_tail, to tail a file like C:\temp\foo.log and set "tag *", this will cause the tag to be like "C:\temp\foo.log".
-
Configure the concat plugin in a way which would cause a timeout flush
result: the timeout flush corrupts the tag...it eliminates everything after the colon, so it just says "C:"
Provide example config and message
-
Create an empty C:\temp\foo.log file
-
Create fluent.conf
########
# Source
########
<source>
@type tail
path C:/Temp/foo.log
pos_file C:/Temp/foo.log.pos
<parse>
@type none
</parse>
tag *
</source>
################################
# Concatenate multiline messages
################################
<filter **>
@type concat
key message
multiline_start_regexp /^(INFO)/
continuous_line_regexp /[^(INFO)]/
flush_interval 1
use_first_timestamp true
timeout_label @TIMEOUT_OUTPUT
</filter>
<label @TIMEOUT_OUTPUT>
<match **>
@type stdout
</match>
</label>
<match **>
@type stdout
</match>
-
run 'fluentd -c fluent.conf'
-
add the lines to foo.log and save it
INFO - check the tag some other text which should have been concatenated.
Result:
The tag is corrupted:
2020-02-24 10:38:09.599785900 -0500 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
2020-02-24 10:38:32.600000600 -0500 C:.Temp.foo.log: {"message":""}
2020-02-24 10:38:32.600013400 -0500 C: {"message":"INFO - check the tag"}
2020-02-24 10:38:33 -0500 [info]: #0 Timeout flush: C:.Temp.foo.log:default
2020-02-24 10:38:33.599202100 -0500 fluent.info: {"message":"Timeout flush: C:.Temp.foo.log:default"}
Expected Behavior
The tag should remain in-tact...it should say C:.temp.foo.log
Your environment
windows (this also occurs in docker)
-
OS version
wntx64, but also occurs in linux/docker.
-
paste result of fluentd --version or td-agent --version
fluentd 1.9.1
-
plugin version
- paste boot log of fluentd or td-agent
- paste result of
fluent-gem list, td-agent-gem list or your Gemfile.lock
fluent-plugin-concat (2.4.0)
Problem
When a colon (:) is in the tag, and a timeout flush occurs, then everything past the colon is lost.
Steps to replicate
On windows, configure a in_tail, to tail a file like C:\temp\foo.log and set "tag *", this will cause the tag to be like "C:\temp\foo.log".
Configure the concat plugin in a way which would cause a timeout flush
result: the timeout flush corrupts the tag...it eliminates everything after the colon, so it just says "C:"
Provide example config and message
Create an empty C:\temp\foo.log file
Create fluent.conf
run 'fluentd -c fluent.conf'
add the lines to foo.log and save it
INFO - check the tag some other text which should have been concatenated.Result:
The tag is corrupted:
2020-02-24 10:38:09.599785900 -0500 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
2020-02-24 10:38:32.600000600 -0500 C:.Temp.foo.log: {"message":""}
2020-02-24 10:38:32.600013400 -0500 C: {"message":"INFO - check the tag"}
2020-02-24 10:38:33 -0500 [info]: #0 Timeout flush: C:.Temp.foo.log:default
2020-02-24 10:38:33.599202100 -0500 fluent.info: {"message":"Timeout flush: C:.Temp.foo.log:default"}
Expected Behavior
The tag should remain in-tact...it should say C:.temp.foo.log
Your environment
windows (this also occurs in docker)
OS version
wntx64, but also occurs in linux/docker.
paste result of
fluentd --versionortd-agent --versionfluentd 1.9.1
plugin version
fluent-gem list,td-agent-gem listor your Gemfile.lockfluent-plugin-concat (2.4.0)