perf(forwarder): create scrubber, matcher, and batcher once in __init__#1081
Conversation
| serialized = dump_event(log) | ||
| evaluated_log = log.get("message") or serialized | ||
| else: | ||
| serialized = dump_event(log) | ||
| evaluated_log = log |
There was a problem hiding this comment.
💬 suggestion: We'll dump event to JSON even if we don't forward them with this change. Some event, especially cloudtrail, can be very large and that's why we wanted to dump only if required.
There was a problem hiding this comment.
👍 I've updated the code to defer json.dumps until after matching. For dicts with a message field (CloudTrail), it now matches on the message string directly and only serialize if the event passes the filter. This avoids wasted serialization for excluded events!
There was a problem hiding this comment.
Great catch on the serialization concern! That said, this is actually what the code did before your change — you've essentially circled back to the previous behavior, just with the matching logic split across three branches now. The original version intentionally serialized once and matched once in a single path, which I'd argue was easier to follow. 🤖
…der.__init__ Move DatadogScrubber, DatadogMatcher, and DatadogBatcher instantiation from per-call in _forward_logs to __init__, avoiding repeated object creation on every invocation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0374eca to
1444f4c
Compare
Summary
_forward_logs()created a newDatadogScrubber(compiles regex per scrubbing rule),DatadogMatcher(compiles include/exclude regex), andDatadogBatcheron every callForwarder.__init__()asself._scrubber,self._matcher, andself._batcherForwarderis a container-global singleton, this also eliminates re-compilation across warm Lambda invocations_forward_logsand uses idiomatic emptiness checksTest plan
test_forwarder.pytests pass🤖 Generated with Claude Code