Skip to content

syslog.md

Latest commit

 

History

History
231 lines (193 loc) · 8.09 KB

syslog.md

File metadata and controls

231 lines (193 loc) · 8.09 KB

Syslog

The Syslog output plugin lets you deliver messages to Syslog servers. It supports RFC3164 and RFC5424 formats through different transports such as UDP, TCP, or TLS.

Configuration parameters

Key Description Default
allow_longer_sd_id If true, Fluent Bit allows SD-ID values longer than 32 characters. SD-ID values that exceed 32 characters violate RFC5424 standards. false
host Domain or IP address of the remote Syslog server. 127.0.0.1
mode Desired transport type. Available options are tcp and udp. To use a TLS secure channel, set this to tcp and enable the tls option separately. Datagram Transport Layer Security (DTLS) over UDP isn't supported. udp
port TCP or UDP port of the remote Syslog server. 514
syslog_appname_key Optional. The key name from the original record that contains the application name that generated the message. none
syslog_appname_preset Optional. The preset application name. It will be overwritten if syslog_appname_key is set and a key of a record is matched. none
syslog_facility_key Optional. The key name from the original record that contains the Syslog facility number. none
syslog_facility_preset Optional. The preset facility number. It will be overwritten if syslog_facility_key is set and a key of a record is matched. 1
syslog_format The Syslog protocol format to use. Available options are rfc3164 and rfc5424. rfc5424
syslog_hostname_key Optional. The key name from the original record that contains the hostname that generated the message. none
syslog_hostname_preset Optional. The preset hostname. It will be overwritten if syslog_hostname_key is set and a key of a record is matched. none
syslog_maxsize The maximum size allowed per message. The value must be an integer representing the number of bytes allowed. If no value is provided, the default size is set depending on the protocol version specified by syslog_format. The value rfc3164 sets max size to 1024 bytes, and rfc5424 sets the size to 2048 bytes. 0
syslog_message_key Required. The key name from the original record that contains the message to deliver. none
syslog_msgid_key Optional. The key name from the original record that contains the Message ID associated to the message. none
syslog_msgid_preset Optional. The preset message ID. It will be overwritten if syslog_msgid_key is set and a key of a record is matched. none
syslog_procid_key Optional. The key name from the original record that contains the Process ID that generated the message. none
syslog_procid_preset Optional. The preset process ID. It will be overwritten if syslog_procid_key is set and a key of a record is matched. none
syslog_sd_key Optional. The key name from the original record that contains a map of key/value pairs to use as Structured Data (SD) content. The key name is included in the resulting SD field as shown in the examples in this doc. none
syslog_severity_key Optional. The key name from the original record that contains the Syslog severity number. none
syslog_severity_preset Optional. The preset severity number. It will be overwritten if syslog_severity_key is set and a key of a record is matched. 6
workers The number of workers to perform flush operations for this output. 0

TLS / SSL

The Syslog output plugin supports TLS/SSL. For more details about the properties available and general configuration, see TLS/SSL.

Examples

Configuration file

Get started quickly with this configuration file:

{% tabs %} {% tab title="fluent-bit.yaml" %}

pipeline:

  outputs:
    - name: syslog
      match: "*"
      host: syslog.yourserver.com
      port: 514
      mode: udp
      syslog_format: rfc5424
      syslog_maxsize: 2048
      syslog_severity_key: severity
      syslog_facility_key: facility
      syslog_hostname_key: hostname
      syslog_appname_key: appname
      syslog_procid_key: procid
      syslog_msgid_key: msgid
      syslog_sd_key: sd
      syslog_message_key: message

{% endtab %} {% tab title="fluent-bit.conf" %}

[OUTPUT]
  Name                 syslog
  Match                *
  Host                 syslog.yourserver.com
  Port                 514
  Mode                 udp
  Syslog_Format        rfc5424
  Syslog_Maxsize       2048
  Syslog_Severity_Key  severity
  Syslog_Facility_Key  facility
  Syslog_Hostname_Key  hostname
  Syslog_Appname_Key   appname
  Syslog_Procid_Key    procid
  Syslog_Msgid_Key     msgid
  Syslog_Sd_Key        sd
  Syslog_Message_Key   message

{% endtab %} {% endtabs %}

Structured data

The following is an example of how to configure the syslog_sd_key to send Structured Data to the remote Syslog server.

Example log:

{
  "hostname": "myhost",
  "appname": "myapp",
  "procid": "1234",
  "msgid": "ID98",
  "uls@0": {
    "logtype": "access",
    "clustername": "mycluster",
    "namespace": "mynamespace"
  },
  "log": "Sample app log message."
}

Example configuration file:

{% tabs %} {% tab title="fluent-bit.yaml" %}

pipeline:

  outputs:
    - name: syslog
      match: "*"
      host: syslog.yourserver.com
      port: 514
      mode: udp
      syslog_format: rfc5424
      syslog_maxsize: 2048
      syslog_hostname_key: hostname
      syslog_appname_key: appname
      syslog_procid_key: procid
      syslog_msgid_key: msgid
      syslog_sd_key: uls@0
      syslog_message_key: log

{% endtab %} {% tab title="fluent-bit.conf" %}

[OUTPUT]
  Name                 syslog
  Match                *
  Host                 syslog.yourserver.com
  Port                 514
  Mode                 udp
  Syslog_Format        rfc5424
  Syslog_Maxsize       2048
  Syslog_Hostname_Key  hostname
  Syslog_Appname_Key   appname
  Syslog_Procid_Key    procid
  Syslog_Msgid_Key     msgid
  Syslog_Sd_Key        uls@0
  Syslog_Message_Key   log

{% endtab %} {% endtabs %}

Example output:

...
<14>1 2021-07-12T14:37:35.569848Z myhost myapp 1234 ID98 [uls@0 logtype="access" clustername="mycluster" namespace="mynamespace"] Sample app log message.
...

Add structured data authentication token

Some services use the structured data field to pass authentication tokens (for example, [<token>@41018]), which would need to be added to each log message dynamically. However, this requires setting the token as a key rather than as a value.

Here's an example of how that might be achieved, using AUTH_TOKEN as a variable:

{% tabs %} {% tab title="fluent-bit.yaml" %}

pipeline:

  filters:
    - name:  lua
      match: "*"
      call:  append_token
      code:  |
        function append_token(tag, timestamp, record)
            record["${AUTH_TOKEN}"] = {}
            return 2, timestamp, record
        end

  outputs:
    - name: syslog
      match: "*"
      host: syslog.yourserver.com
      port: 514
      mode: tcp
      syslog_format: rfc5424
      syslog_hostname_preset: myhost
      syslog_appname_preset: myapp
      syslog_message_key: log
      allow_longer_sd_id: true
      syslog_sd_key: ${AUTH_TOKEN}
      tls: on
      tls.crt_file: /path/to/my.crt

{% endtab %} {% tab title="fluent-bit.conf" %}

[FILTER]
  Name  lua
  Match *
  Call  append_token
  Code  function append_token(tag, timestamp, record) record["${AUTH_TOKEN}"] = {} return 2, timestamp, record end

[OUTPUT]
  Name                    syslog
  Match                   *
  Host                    syslog.yourserver.com
  Port                    514
  Mode                    tcp
  Syslog_Format           rfc5424
  Syslog_Hostname_Preset  my-hostname
  Syslog_Appname_Preset   my-appname
  Syslog_Message_Key      log
  Allow_Longer_Sd_Id      true
  Syslog_Sd_Key           ${AUTH_TOKEN}
  Tls                     on
  Tls.crt_file            /path/to/my.crt

{% endtab %} {% endtabs %}