Skip to content

perf(go-forwarder): benchmarking decoder vs unmarshal on parsing#1156

Merged
ndakkoune merged 8 commits into
nabil.dakkoune/go-forwarderfrom
nabil.dakkoune/go-forwarder-parsing-benchmark
Jun 18, 2026
Merged

perf(go-forwarder): benchmarking decoder vs unmarshal on parsing#1156
ndakkoune merged 8 commits into
nabil.dakkoune/go-forwarderfrom
nabil.dakkoune/go-forwarder-parsing-benchmark

Conversation

@ndakkoune

Copy link
Copy Markdown
Contributor

I needed to propagate information from SQS events in the parsing package and ended up benchmarking the whole package against an unmarshal approach which, besides being way easier to understand, increased performances of all event sources (but slightly decrease for CW).


goos: darwin
goarch: arm64
cpu: Apple M4 Max
                     │   decoder   │              unmarshal              │
                     │   sec/op    │   sec/op     vs base                │
Parse_CloudWatch-16    283.2n ± 1%   293.6n ± 1%   +3.65% (p=0.000 n=10)
Parse_S3-16            679.6n ± 1%   641.4n ± 1%   -5.63% (p=0.000 n=10)
Parse_SNS_S3-16        3.653µ ± 1%   2.473µ ± 1%  -32.31% (p=0.000 n=10)
Parse_SQS_S3-16        18.41µ ± 1%   15.73µ ± 1%  -14.52% (p=0.000 n=10)
Parse_SQS_SNS_S3-16    28.46µ ± 1%   26.25µ ± 0%   -7.76% (p=0.000 n=10)
Parse_EventBridge-16   4.446µ ± 1%   1.537µ ± 1%  -65.43% (p=0.000 n=10)
geomean                3.433µ        2.581µ       -24.82%

                     │   decoder    │              unmarshal               │
                     │     B/op     │     B/op      vs base                │
Parse_CloudWatch-16     1064.0 ± 0%     400.0 ± 0%  -62.41% (p=0.000 n=10)
Parse_S3-16             1368.0 ± 0%     528.0 ± 0%  -61.40% (p=0.000 n=10)
Parse_SNS_S3-16        5.703Ki ± 0%   1.727Ki ± 0%  -69.73% (p=0.000 n=10)
Parse_SQS_S3-16        34.47Ki ± 0%   14.79Ki ± 0%  -57.09% (p=0.000 n=10)
Parse_SQS_SNS_S3-16    41.58Ki ± 0%   21.51Ki ± 0%  -48.27% (p=0.000 n=10)
Parse_EventBridge-16    5866.0 ± 0%    1000.0 ± 0%  -82.95% (p=0.000 n=10)
geomean                6.341Ki        2.182Ki       -65.58%

                     │   decoder   │             unmarshal              │
                     │  allocs/op  │ allocs/op   vs base                │
Parse_CloudWatch-16    12.000 ± 0%   8.000 ± 0%  -33.33% (p=0.000 n=10)
Parse_S3-16             28.00 ± 0%   12.00 ± 0%  -57.14% (p=0.000 n=10)
Parse_SNS_S3-16        119.00 ± 0%   35.00 ± 0%  -70.59% (p=0.000 n=10)
Parse_SQS_S3-16         656.0 ± 0%   171.0 ± 0%  -73.93% (p=0.000 n=10)
Parse_SQS_SNS_S3-16     846.0 ± 0%   261.0 ± 0%  -69.15% (p=0.000 n=10)
Parse_EventBridge-16   137.00 ± 0%   19.00 ± 0%  -86.13% (p=0.000 n=10)
geomean                 120.4        37.65       -68.72%

@github-actions github-actions Bot added the aws label Jun 16, 2026
@ndakkoune ndakkoune requested a review from ViBiOh June 16, 2026 16:55
@ndakkoune ndakkoune marked this pull request as ready for review June 16, 2026 16:55
@ndakkoune ndakkoune requested a review from a team as a code owner June 16, 2026 16:55
@ndakkoune ndakkoune force-pushed the nabil.dakkoune/go-forwarder-parsing-benchmark branch from e102658 to 328b204 Compare June 17, 2026 13:18
@datadog-datadog-prod-us1-2

This comment has been minimized.

@ViBiOh ViBiOh self-assigned this Jun 18, 2026
Comment thread aws/logs_monitoring_go/internal/handling/cloudtrail.go Outdated
@ndakkoune ndakkoune merged commit c1f4f8e into nabil.dakkoune/go-forwarder Jun 18, 2026
10 checks passed
@ndakkoune ndakkoune deleted the nabil.dakkoune/go-forwarder-parsing-benchmark branch June 18, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants