Add JSON body expansion to log attributes with configurable threshold#64
Closed
RaphaelManke wants to merge 1 commit into
Closed
Add JSON body expansion to log attributes with configurable threshold#64RaphaelManke wants to merge 1 commit into
RaphaelManke wants to merge 1 commit into
Conversation
When a function log body is a valid JSON object, each top-level key is added as a structured OTLP log attribute, enabling server-side filtering and querying on individual fields. The original JSON string is kept in the log body for human readability. Expansion is skipped when the JSON object has more keys than a configurable threshold (DASH0_LOG_JSON_BODY_MAX_ATTRIBUTES, default 25), preventing attribute bloat on verbose payloads. Setting the threshold to 0 disables JSON expansion entirely. https://claude.ai/code/session_01QyWWPfMHcJ4S4wWJNUAwj6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds automatic expansion of JSON-formatted log bodies into individual log attributes, with a configurable threshold to prevent attribute explosion. When a log message body is valid JSON object, its key-value pairs are extracted and added as separate attributes to the OTLP log record.
Key Changes
New helper functions in
src/otlp/log_mutations.rs:json_value_to_any_value(): Converts serde_json values to OpenTelemetry AnyValue types, handling strings, booleans, integers, floats, nulls, and complex types (objects/arrays as stringified JSON)try_expand_json_body_to_attributes(): Attempts to parse log body as JSON object and convert to KeyValue attributes, respecting a configurable maximum attribute countIntegration in
map_logs_to_otlp():DASH0_LOG_JSON_BODY_MAX_ATTRIBUTESenvironment variableNew configuration in
src/config/user.rs:log_json_body_max_attributes(): ReadsDASH0_LOG_JSON_BODY_MAX_ATTRIBUTESenvironment variable with a default of 25Comprehensive test coverage:
json_value_to_any_value()covering all JSON typestry_expand_json_body_to_attributes()covering edge cases (empty objects, threshold boundaries, invalid JSON)map_logs_to_otlp()covering various scenarios: JSON expansion, non-JSON bodies, platform logs, Node.js prefix handling, and threshold enforcementImplementation Details
platform.start)"null"DASH0_LOG_JSON_BODY_MAX_ATTRIBUTES=0https://claude.ai/code/session_01QyWWPfMHcJ4S4wWJNUAwj6