Skip to content

Add JSON body expansion to log attributes with configurable threshold#64

Closed
RaphaelManke wants to merge 1 commit into
masterfrom
claude/amazing-pasteur-zWMVU
Closed

Add JSON body expansion to log attributes with configurable threshold#64
RaphaelManke wants to merge 1 commit into
masterfrom
claude/amazing-pasteur-zWMVU

Conversation

@RaphaelManke

Copy link
Copy Markdown
Contributor

Summary

This PR adds automatic expansion of JSON-formatted log bodies into individual log attributes, with a configurable threshold to prevent attribute explosion. When a log message body is valid JSON object, its key-value pairs are extracted and added as separate attributes to the OTLP log record.

Key Changes

  • New helper functions in src/otlp/log_mutations.rs:

    • json_value_to_any_value(): Converts serde_json values to OpenTelemetry AnyValue types, handling strings, booleans, integers, floats, nulls, and complex types (objects/arrays as stringified JSON)
    • try_expand_json_body_to_attributes(): Attempts to parse log body as JSON object and convert to KeyValue attributes, respecting a configurable maximum attribute count
  • Integration in map_logs_to_otlp():

    • JSON expansion is applied only to non-platform logs (user function logs)
    • Respects a configurable threshold via DASH0_LOG_JSON_BODY_MAX_ATTRIBUTES environment variable
    • Original log body is preserved unchanged
  • New configuration in src/config/user.rs:

    • log_json_body_max_attributes(): Reads DASH0_LOG_JSON_BODY_MAX_ATTRIBUTES environment variable with a default of 25
    • Returns 0 to disable expansion entirely
  • Comprehensive test coverage:

    • Unit tests for json_value_to_any_value() covering all JSON types
    • Unit tests for try_expand_json_body_to_attributes() covering edge cases (empty objects, threshold boundaries, invalid JSON)
    • Integration tests for map_logs_to_otlp() covering various scenarios: JSON expansion, non-JSON bodies, platform logs, Node.js prefix handling, and threshold enforcement
    • Configuration tests for the new environment variable

Implementation Details

  • JSON expansion only occurs for function logs (not platform logs like platform.start)
  • Objects exceeding the configured attribute limit are not expanded (body remains as-is)
  • Complex JSON types (nested objects, arrays) are stringified as attribute values
  • Null values are converted to the string "null"
  • The feature is disabled by default when DASH0_LOG_JSON_BODY_MAX_ATTRIBUTES=0

https://claude.ai/code/session_01QyWWPfMHcJ4S4wWJNUAwj6

When a function log body is a valid JSON object, each top-level key is
added as a structured OTLP log attribute, enabling server-side filtering
and querying on individual fields. The original JSON string is kept in
the log body for human readability.

Expansion is skipped when the JSON object has more keys than a
configurable threshold (DASH0_LOG_JSON_BODY_MAX_ATTRIBUTES, default 25),
preventing attribute bloat on verbose payloads. Setting the threshold to
0 disables JSON expansion entirely.

https://claude.ai/code/session_01QyWWPfMHcJ4S4wWJNUAwj6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants