Skip to content

Support excluding MDC attributes from capture-all#18912

Open
philsttr wants to merge 11 commits into
open-telemetry:mainfrom
philsttr:exclude_mdc_entries
Open

Support excluding MDC attributes from capture-all#18912
philsttr wants to merge 11 commits into
open-telemetry:mainfrom
philsttr:exclude_mdc_entries

Conversation

@philsttr

@philsttr philsttr commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Problem

I need the ability to exclude specific attributes when capturing MDC/context-data. Previously, the logging instrumentations supported including all attributes (e.g. *) or specific attributes (e.g. foo,bar). But there was no way to say "include everything except x). This PR adds that capability.

Closes #14925

Change Summary

Added the ability to exclude specific attributes when capturing all MDC/context-data attributes with the * wildcard, across the jboss-logmanager, log4j 1.2, log4j 2.17, and logback log appenders. Prefix an attribute name with ! to exclude it.

*           → capture all MDC attributes (unchanged)
*,!foo      → capture all MDC attributes except "foo"
*,!foo,!bar → capture all except "foo" and "bar"
!foo        → capture only key named "!foo"  (literally, preserves backwards compatibility)

Behavior

  • The * check changed from "list is exactly [*]" to "list contains *", so * can be combined with !-prefixed exclusions.
  • Exclusions only apply in the capture-all path; explicit attribute lists behave as before.
  • otel.event.name remains special-cased (mapped to the log event name, never emitted as an attribute).
  • These are all experimental config options, so no stable API is affected.

Changes by area

Mappers (core logic. identical small change in each)

Module File
log4j 1.2 LogEventMapper.java
log4j 2.17 LogEventMapper.java (context data)
logback LoggingEventMapper.java
jboss-logmanager LoggingEventMapper.java

Each parses !-prefixed entries into an exclusion Set<String> and skips those keys when iterating the MDC map.

Tests

  • testAllWithExclusion added to the two library modules with directly testable mappers (log4j 2.17 and logback).
  • javaagent-only mappers (log4j 1.2, jboss) share the identical logic; covered by the library unit tests.

Documentation

  • 4 metadata.yaml descriptions + a *,!foo example.
  • 5 README.md settings tables (realigned to the repo's table style).

When capturing all MDC/context-data attributes with the `*` wildcard in the
jboss-logmanager, log4j (1.2 and 2.17), and logback log appenders, attributes
can now be excluded by prefixing them with `!`. For example, `*,!foo` captures
all MDC attributes except `foo`.
Copilot AI review requested due to automatic review settings June 5, 2026 17:59
@philsttr philsttr requested a review from a team as a code owner June 5, 2026 17:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds support for excluding specific MDC/context-data keys when wildcard capture (*) is enabled across multiple logging appenders/instrumentations.

Changes:

  • Implement exclusion syntax (!key) when wildcard capture is enabled in Logback / Log4j / JBoss Log Manager mappers.
  • Add unit tests for exclusion behavior in Logback and Log4j 2.17.
  • Update metadata and READMEs to document *,!foo configuration.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
instrumentation/logback/logback-appender-1.0/metadata.yaml Documents !key exclusions alongside * wildcard capture.
instrumentation/logback/logback-appender-1.0/library/src/test/java/io/opentelemetry/instrumentation/logback/appender/v1_0/internal/LoggingEventMapperTest.java Adds test coverage for wildcard capture with exclusions.
instrumentation/logback/logback-appender-1.0/library/src/main/java/io/opentelemetry/instrumentation/logback/appender/v1_0/internal/LoggingEventMapper.java Implements exclusion set for MDC capture when using *.
instrumentation/logback/logback-appender-1.0/javaagent/README.md Documents *,!foo exclusion syntax for MDC capture.
instrumentation/log4j/log4j-appender-2.17/metadata.yaml Documents !key exclusions alongside * wildcard capture.
instrumentation/log4j/log4j-appender-2.17/library/src/test/java/io/opentelemetry/instrumentation/log4j/appender/v2_17/internal/LogEventMapperTest.java Adds test coverage for wildcard capture with exclusions.
instrumentation/log4j/log4j-appender-2.17/library/src/main/java/io/opentelemetry/instrumentation/log4j/appender/v2_17/internal/LogEventMapper.java Implements exclusion set for context data capture when using *.
instrumentation/log4j/log4j-appender-2.17/library/README.md Documents *,!foo exclusion syntax for context data capture.
instrumentation/log4j/log4j-appender-2.17/javaagent/README.md Documents *,!foo exclusion syntax for context data capture.
instrumentation/log4j/log4j-appender-1.2/metadata.yaml Documents !key exclusions alongside * wildcard capture.
instrumentation/log4j/log4j-appender-1.2/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/log4j/appender/v1_2/LogEventMapper.java Implements exclusion set for MDC capture when using *.
instrumentation/log4j/log4j-appender-1.2/javaagent/README.md Documents *,!foo exclusion syntax for MDC capture.
instrumentation/jboss-logmanager/jboss-logmanager-appender-1.1/metadata.yaml Documents !key exclusions alongside * wildcard capture.
instrumentation/jboss-logmanager/jboss-logmanager-appender-1.1/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/jbosslogmanager/appender/v1_1/LoggingEventMapper.java Implements exclusion set for MDC capture when using *.
instrumentation/jboss-logmanager/README.md Documents *,!foo exclusion syntax for MDC capture.
docs/instrumentation-list.yaml Propagates exclusion syntax into generated instrumentation docs.
Comments suppressed due to low confidence (1)

instrumentation/logback/logback-appender-1.0/library/src/main/java/io/opentelemetry/instrumentation/logback/appender/v1_0/internal/LoggingEventMapper.java:1

  • When * is not present, entries like "!key2" are currently treated as a literal attribute key (because only "*" is filtered out). Since docs define !key as meaningful only with *, it would be better to ignore !… entries (or fail fast / warn) in the non-wildcard branch to avoid silently doing something unexpected.
/*

@philsttr philsttr requested a review from Copilot June 5, 2026 18:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

@philsttr philsttr requested a review from Copilot June 5, 2026 19:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

@philsttr philsttr requested a review from Copilot June 5, 2026 19:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.

Comment thread instrumentation/logback/logback-appender-1.0/metadata.yaml Outdated
@laurit

laurit commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Is there prior art for this in otel ecosystem? There is a moratorium in otel spec for introducing new environment variables with complex encoding. The moratorium was originally in place to incentivize development of declarative config, which is done now. While we are not strictly bound by this moratorium the question is still whether there could be a better solution for this that does not involve complex encodings. Perhaps some sort of include/exclude pair would work better?
cc @trask

@philsttr

Copy link
Copy Markdown
Contributor Author

I'm happy to refactor if needed. Let me know your concrete suggestions. (e.g. new option names, and what to do with the existing option names)

philsttr added 5 commits June 12, 2026 10:50
…for MDC capture

Replace the !-prefix exclusion syntax with a dedicated exclude-mdc-attributes
property and reuse the OpenTelemetry SDK's IncludeExcludePredicate (glob matching)
across the jboss-logmanager, log4j (1.2 and 2.17), and logback appenders.

- capture-mdc-attributes is the include list; exclude-mdc-attributes is the new
  exclude list; both support glob wildcards (* and ?)
- empty includes captures nothing (preserving the default); excludes only take
  effect alongside a non-empty include list
- wire the new property through the javaagent config, the standalone library
  appenders, and the Spring Boot starter (non-agent path)
@philsttr

philsttr commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

I've updated the code to have both include and exclude properties as suggested. The existing capture-mdc-attributes properties are the include properties (to keep backwards compatibility). And I've added new exclude-mdc-attributes properties to each instrumentation to handle excludes.

The instrumentations now use IncludeExcludePredicate from opentelemetry-sdk-common. Is this allowed?
I had to add the dependency, which I think is the source of the muzzle problems, but it's not obvious to me how to fix it.
I could use some guidance on the muzzle failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow excluding Logback MDC attributes from being copied

4 participants