feat(cloudwatch-logs): Add Entity config support to sink#6864
Merged
dlvenable merged 5 commits intoMay 27, 2026
Merged
Conversation
bagmarnikhil
requested review from
KarstenSchnitter,
Zhangxunmt,
dinujoh,
divbok,
dlvenable,
graytaylor0,
kkondaka,
oeyh,
san81,
sb2k16 and
srikanthjg
as code owners
✅ License Header Check PassedAll newly added files have proper license headers. Great work! 🎉 |
1 task
Add an optional 'entity' configuration block on the CloudWatch Logs sink that attaches CloudWatch Entity metadata (key_attributes + attributes) to every PutLogEvents request, enabling entity-based correlation in CloudWatch. When 'entity' is omitted the sink behaves identically to before. When configured, an SDK Entity is built and passed to the dispatcher via the builder; the dispatcher conditionally sets it on each PLE request. If CloudWatch rejects the entity, the request is still considered successful (events are released), a WARN log records the RejectedEntityInfo error type, and a new 'cloudWatchLogsEntityRejected' counter is incremented as the primary alarmable signal. Validation is intentionally minimal — @notempty on key_attributes only. AWS-owned limits (max entries, allowed key names, length caps) are enforced server-side and surfaced via the rejection metric, avoiding client-side drift when the Entity API contract changes. Tests follow TDD: each test was written before the code that makes it pass. New unit tests cover EntityConfig defaults, deserialization, and @notempty validation; @Valid cascade from CloudWatchLogsSinkConfig; the new counter; dispatcher entity-on-request, no-entity, and rejection paths; and sink wiring of EntityConfig -> Entity -> dispatcher. TestSinkOperationWithEntity integration test exercises the SDK end-to-end. Acceptance criteria from the spec are met: backward compatibility, entity attached when configured, non-fatal rejection with metric and log, all existing tests green, new tests cover new code paths, license headers on new files. Refs: data-prepper-plugins/cloudwatch-logs/specs/entity-config.md Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
- Remove duplicate @test annotations from auto-merge - Add Hamcrest imports for entity assertion matchers - Update EXPECTED_DISPATCHER_ARITY to 11 (entity field added) Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
bagmarnikhil
force-pushed
the
feature/cwl-sink-entity-config
branch
from
f18af61 to
60dc039
Compare
Himtanaya
approved these changes
May 21, 2026
srikanthpadakanti
requested changes
May 21, 2026
…rejection check - Default keyAttributes and attributes to Collections.emptyMap() - Return Collections.unmodifiableMap() from both getters - Move entity rejection metric before releaseEventHandles - Add unmodifiable map and createResources exception tests Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
dlvenable
previously approved these changes
May 26, 2026
dlvenable
left a comment
dlvenable
left a comment
Member
There was a problem hiding this comment.
Thanks @bagmarnikhil for this contribution!
srikanthpadakanti
requested changes
May 26, 2026
| @NotEmpty | ||
| private Map<String, String> keyAttributes = Collections.emptyMap(); | ||
|
|
||
| @JsonProperty("attributes") |
Collaborator
Himtanaya
approved these changes
May 26, 2026
… log, remove duplicate test - Add @NotNull on attributes field to prevent NPE from explicit null in YAML - Use NOISY marker on entity rejection WARN to rate-limit under load - Remove redundant empty-key-attributes test (default is already emptyMap) - Add test for @NotNull constraint violation on null attributes Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
srikanthpadakanti
approved these changes
May 27, 2026
Collaborator
|
Thank you @bagmarnikhil for this contribution. Approved the changes |
kkondaka
pushed a commit
to kkondaka/kk-data-prepper-f2
that referenced
this pull request
Jul 1, 2026
Add Entity config support to sink Add an optional 'entity' configuration block on the CloudWatch Logs sink that attaches CloudWatch Entity metadata (key_attributes + attributes) to every PutLogEvents request, enabling entity-based correlation in CloudWatch. When 'entity' is omitted the sink behaves identically to before. When configured, an SDK Entity is built and passed to the dispatcher via the builder; the dispatcher conditionally sets it on each PLE request. If CloudWatch rejects the entity, the request is still considered successful (events are released), a WARN log records the RejectedEntityInfo error type, and a new 'cloudWatchLogsEntityRejected' counter is incremented as the primary alarmable signal. Validation is intentionally minimal — @notempty on key_attributes only. AWS-owned limits (max entries, allowed key names, length caps) are enforced server-side and surfaced via the rejection metric, avoiding client-side drift when the Entity API contract changes. Tests follow TDD: each test was written before the code that makes it pass. New unit tests cover EntityConfig defaults, deserialization, and @notempty validation; @Valid cascade from CloudWatchLogsSinkConfig; the new counter; dispatcher entity-on-request, no-entity, and rejection paths; and sink wiring of EntityConfig -> Entity -> dispatcher. TestSinkOperationWithEntity integration test exercises the SDK end-to-end. Acceptance criteria from the spec are met: backward compatibility, entity attached when configured, non-fatal rejection with metric and log, all existing tests green, new tests cover new code paths, license headers on new files. Add full license headers to EntityConfig and EntityConfigTest Signed-off-by: Nikhil Bagmar <nikhilbagmar73@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add an optional 'entity' configuration block on the CloudWatch Logs sink that attaches CloudWatch Entity metadata (key_attributes + attributes) to every PutLogEvents request, enabling entity-based correlation in CloudWatch.
When 'entity' is omitted the sink behaves identically to before. When configured, an SDK Entity is built and passed to the dispatcher via the builder; the dispatcher conditionally sets it on each PLE request.
If CloudWatch rejects the entity, the request is still considered successful (events are released), a WARN log records the RejectedEntityInfo error type, and a new 'cloudWatchLogsEntityRejected' counter is incremented as the primary alarmable signal.
Validation is intentionally minimal — @notempty on key_attributes only. AWS-owned limits (max entries, allowed key names, length caps) are enforced server-side and surfaced via the rejection metric, avoiding client-side drift when the Entity API contract changes.
Tests follow TDD: each test was written before the code that makes it pass. New unit tests cover EntityConfig defaults, deserialization, and @notempty validation; @Valid cascade from CloudWatchLogsSinkConfig; the new counter; dispatcher entity-on-request, no-entity, and rejection paths; and sink wiring of EntityConfig -> Entity -> dispatcher. TestSinkOperationWithEntity integration test exercises the SDK end-to-end.
Acceptance criteria from the spec are met: backward compatibility, entity attached when configured, non-fatal rejection with metric and log, all existing tests green, new tests cover new code paths, license headers on new files.
Refs: data-prepper-plugins/cloudwatch-logs/specs/entity-config.md
Description
[Describe what this change achieves]
Issues Resolved
Resolves #6860
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.