Skip to content

Nested JSON output #256

Description

@atze234

Problem

Hello,

i'm parsing my logs using Grok Patterns with the fluent-plugin-grok-parser.
Is it possible somehow that i can put nested JSON directly to cloudwatch?

...

Steps to replicate

Having such a pattern in filter for example:

<filter matcher>
  @type parser
  key_name message
  reserve_data true
<grok>
  pattern "^%{IP:[http][request][client][ip]} %{DATA}"
</grok>
</filter>

This will create the Field "[http][request][client][ip]" in cloudwatch logs.

Expected Behavior or What you need to ask

What i expect in Cloudwatch Logs:

{
   
    "host": "myhostname",
    "message": "192.168.0.1 message-from-ip",
    "http": {
        "request": {
          "client": {
            "ip" : "192.168.0.1"
          }
        }
      }
}

But i got in cloudwatch:

{
   
    "host": "myhostname",
    "message": "192.168.0.1 message-from-ip",
    "[http][request][client][ip]": "192.168.0.1"
}

Using Fluentd and CloudWatchLogs plugin versions

  • Debian 11
  • Bare Metal
  • fluent-package
  • Dependent gem versions
    • fluent-plugin-cloudwatch-logs (0.14.3)
    • fluent-plugin-grok-parser (2.6.2)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions