Skip to content

[Feature Request] Support for native out-of-band Labels and Namespace mapping in google-chronicle destination plugin #11833

@agup006

Description

@agup006

Is your feature request related to a problem? Please describe.

To properly utilize Google SecOps, logs often need to be enriched with custom Labels and assigned to a specific SecOps Namespace. (In Google SecOps, Namespaces are critical for asset resolution, handling overlapping IP spaces, and multi-tenant isolation).

Currently, injecting these metadata key-value pairs directly into the raw log payload (e.g., appending them to a Syslog string) frequently breaks default Google SecOps parsers. These out-of-the-box parsers rely on strict regex, grok patterns, or fixed JSON schemas designed strictly around the original raw log format.

The Google SecOps unstructuredlogentries:batchCreate API natively supports labels and namespace as out-of-band body parameters to solve this exact issue. However, the google-chronicle destination plugin currently lacks the configuration keys to map nested Fluent Bit metadata into these specific API fields. This forces users to choose between breaking out-of-the-box SecOps parsers (by mutating the log) or losing critical multi-tenant visibility and accurate asset resolution (by dropping the namespace and labels).

Describe the solution you'd like

Update the google-chronicle destination plugin to support out-of-band metadata injection.

For the Namespace, we request a configuration key that supports standard Fluent Bit Record Accessors to dynamically extract the value. For Labels, we request a configuration format identical to how Header is defined in the standard Fluent Bit http output plugin, allowing for both static values and dynamic record accessor mappings.

Proposed Plugin Additions (YAML Format):

outputs:
  - name: google-chronicle
    match: '*'
    customer_id: YOUR_CUSTOMER_ID
    log_type: YOUR_LOG_TYPE
    # Proposed Feature Additions:
    namespace_key: $tenant_namespace
    # Labels formatted similarly to HTTP output plugin headers:
    labels:
      env: production                  # Static value example
      cluster_name: $cluster['name']   # Dynamic record accessor example

Describe alternatives you've considered
The current workaround involves completely bypassing the native google-chronicle plugin. It requires writing a Lua filter to manually reshape the JSON payload into the exact Google SecOps API specification and sending it via the generic http output plugin to a Webhook URL.

Why this alternative is insufficient:
While this Lua + HTTP workaround solves the parser integrity gap, it breaks native batching. The batchCreate API expects a batched array inside the entries: [] key, meaning the Lua filter effectively forces a 1:1 log-to-API request ratio. A native update to the destination plugin is the only way to achieve both parser integrity and high-throughput batching.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions