Skip to content

timeout flush corrupts tags which contain colon #86

Description

@shpatel2

Problem

When a colon (:) is in the tag, and a timeout flush occurs, then everything past the colon is lost.

Steps to replicate

  1. On windows, configure a in_tail, to tail a file like C:\temp\foo.log and set "tag *", this will cause the tag to be like "C:\temp\foo.log".

  2. Configure the concat plugin in a way which would cause a timeout flush

result: the timeout flush corrupts the tag...it eliminates everything after the colon, so it just says "C:"

Provide example config and message

  1. Create an empty C:\temp\foo.log file

  2. Create fluent.conf

########
# Source
########
<source>
  @type tail
  path C:/Temp/foo.log
  pos_file C:/Temp/foo.log.pos
  <parse>
	@type none
  </parse>
  tag *
</source>

################################
# Concatenate multiline messages
################################
<filter **>
  @type concat
  key message
  multiline_start_regexp /^(INFO)/
  continuous_line_regexp /[^(INFO)]/
  flush_interval 1
  use_first_timestamp true
  timeout_label @TIMEOUT_OUTPUT
</filter>

<label @TIMEOUT_OUTPUT>
    <match **>
        @type stdout
    </match>  
</label>

<match **>
	@type stdout
</match> 
  1. run 'fluentd -c fluent.conf'

  2. add the lines to foo.log and save it
    INFO - check the tag some other text which should have been concatenated.

Result:
The tag is corrupted:
2020-02-24 10:38:09.599785900 -0500 fluent.info: {"worker":0,"message":"fluentd worker is now running worker=0"}
2020-02-24 10:38:32.600000600 -0500 C:.Temp.foo.log: {"message":""}
2020-02-24 10:38:32.600013400 -0500 C: {"message":"INFO - check the tag"}
2020-02-24 10:38:33 -0500 [info]: #0 Timeout flush: C:.Temp.foo.log:default
2020-02-24 10:38:33.599202100 -0500 fluent.info: {"message":"Timeout flush: C:.Temp.foo.log:default"}

Expected Behavior

The tag should remain in-tact...it should say C:.temp.foo.log

Your environment

windows (this also occurs in docker)

  • OS version
    wntx64, but also occurs in linux/docker.

  • paste result of fluentd --version or td-agent --version
    fluentd 1.9.1

  • plugin version

    • paste boot log of fluentd or td-agent
    • paste result of fluent-gem list, td-agent-gem list or your Gemfile.lock
      fluent-plugin-concat (2.4.0)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions