Skip to content

Commit bacfad2

Browse files
lym953claude
andauthored
fix(traces): update payload size after dropping traces (#1091)
## This PR After dropping sampled-out traces, recompute `body_size`. This ensures the `TraceAggregator`'s 3.2 MB batch limit is evaluated against only the data that will actually be sent to the backend. Upgrades the version of some external crates because those are the versions used in libdatadog. ## Testing - Unit tests ## Note The recomputed size uses the protobuf encoded payload size, different from the original size, computed from: (from Claude) > - Datadog tracer path: src/traces/trace_agent.rs:506 — body_size is the first element of the tuple returned by get_traces_from_request_body() / get_v05_traces_from_request_body(), which is the raw HTTP request body byte count (msgpack-encoded bytes). > - OTLP path: src/otlp/agent.rs:167 — let body_size = size_of_val(&traces) where traces is a TracerPayloadCollection, giving only the stack size of the Rust enum (~32 bytes). The new approach be more accurate because it's computed at a later point for the size of protobuf data, which is what's finally sent to Datadog. By the way, the size doesn't need to be accurate. An approximate size is good enough. See: https://github.com/DataDog/libdatadog/blob/3f3efefb2ff45d7a5491b770480396d001b87631/libdd-trace-utils/src/send_data/mod.rs#L148 Also here are some sample logs (logging node not included in the PR) showing the different sizes computed in different ways: > body_size (original): 582, body_size (encoded_len): 1355 > body_size (original): 373, body_size (encoded_len): 1269 > body_size (original): 336, body_size (encoded_len): 899 --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 4c089b1 commit bacfad2

File tree

4 files changed

+207
-26
lines changed

4 files changed

+207
-26
lines changed

bottlecap/Cargo.lock

Lines changed: 75 additions & 21 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

bottlecap/Cargo.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,13 +44,13 @@ rustls-pemfile = { version = "2.0", default-features = false, features = ["std"]
4444
rustls-pki-types = { version = "1.0", default-features = false }
4545
hyper-rustls = { version = "0.27.7", default-features = false }
4646
rand = { version = "0.8", default-features = false }
47-
prost = { version = "0.13", default-features = false }
48-
tonic-types = { version = "0.13", default-features = false }
47+
prost = { version = "0.14", default-features = false }
48+
tonic-types = { version = "0.14", default-features = false }
4949
zstd = { version = "0.13.3", default-features = false }
5050
futures = { version = "0.3.31", default-features = false }
5151
serde-aux = { version = "4.7", default-features = false }
5252
serde_html_form = { version = "0.2", default-features = false }
53-
opentelemetry-proto = { version = "0.30.0", features = ["trace", "with-serde", "gen-tonic"] }
53+
opentelemetry-proto = { version = "0.31.0", features = ["trace", "with-serde", "gen-tonic"] }
5454
opentelemetry-semantic-conventions = { version = "0.30", features = ["semconv_experimental"] }
5555
# Pinned to <0.8.3: version 0.8.3 upgraded to openssl-probe 0.2.x which scans all cert
5656
# directories and parses ~200 individual cert files on Lambda instead of loading a single

bottlecap/LICENSE-3rdparty.csv

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ aws-lc-sys,https://github.com/aws/aws-lc-rs,ISC AND (Apache-2.0 OR ISC) AND Open
1212
axum,https://github.com/tokio-rs/axum,MIT,The axum Authors
1313
axum-core,https://github.com/tokio-rs/axum,MIT,The axum-core Authors
1414
base64,https://github.com/marshallpierce/rust-base64,MIT OR Apache-2.0,Marshall Pierce <marshall@mpierce.org>
15+
bit-set,https://github.com/contain-rs/bit-set,Apache-2.0 OR MIT,Alexis Beingessner <a.beingessner@gmail.com>
16+
bit-vec,https://github.com/contain-rs/bit-vec,Apache-2.0 OR MIT,Alexis Beingessner <a.beingessner@gmail.com>
1517
bitflags,https://github.com/bitflags/bitflags,MIT OR Apache-2.0,The Rust Project Developers
1618
block-buffer,https://github.com/RustCrypto/utils,MIT OR Apache-2.0,RustCrypto Developers
1719
buf_redux,https://github.com/abonander/buf_redux,MIT OR Apache-2.0,Austin Bonander <austin.bonander@gmail.com>
@@ -22,6 +24,7 @@ bytes,https://github.com/tokio-rs/bytes,MIT,"Carl Lerche <me@carllerche.com>, Se
2224
cc,https://github.com/rust-lang/cc-rs,MIT OR Apache-2.0,Alex Crichton <alex@alexcrichton.com>
2325
cfg-if,https://github.com/rust-lang/cfg-if,MIT OR Apache-2.0,Alex Crichton <alex@alexcrichton.com>
2426
chrono,https://github.com/chronotope/chrono,MIT OR Apache-2.0,The chrono Authors
27+
const-hex,https://github.com/danipopes/const-hex,MIT OR Apache-2.0,DaniPopes <57450786+DaniPopes@users.noreply.github.com>
2528
const_format,https://github.com/rodrimati1992/const_format_crates,Zlib,rodrimati1992 <rodrimatt1985@gmail.com>
2629
const_format_proc_macros,https://github.com/rodrimati1992/const_format_crates,Zlib,rodrimati1992 <rodrimatt1985@gmail.com>
2730
cookie,https://github.com/SergioBenitez/cookie-rs,MIT OR Apache-2.0,"Sergio Benitez <sb@sergio.bz>, Alex Crichton <alex@alexcrichton.com>"
@@ -144,6 +147,7 @@ powerfmt,https://github.com/jhpratt/powerfmt,MIT OR Apache-2.0,Jacob Pratt <jaco
144147
ppv-lite86,https://github.com/cryptocorrosion/cryptocorrosion,MIT OR Apache-2.0,The CryptoCorrosion Contributors
145148
proc-macro2,https://github.com/dtolnay/proc-macro2,MIT OR Apache-2.0,"David Tolnay <dtolnay@gmail.com>, Alex Crichton <alex@alexcrichton.com>"
146149
proc-macro2-diagnostics,https://github.com/SergioBenitez/proc-macro2-diagnostics,MIT OR Apache-2.0,Sergio Benitez <sb@sergio.bz>
150+
proptest,https://github.com/proptest-rs/proptest,MIT OR Apache-2.0,Jason Lingle
147151
prost,https://github.com/tokio-rs/prost,Apache-2.0,"Dan Burkert <dan@danburkert.com>, Lucio Franco <luciofranco14@gmail.com>, Casper Meijn <casper@meijn.net>, Tokio Contributors <team@tokio.rs>"
148152
prost-derive,https://github.com/tokio-rs/prost,Apache-2.0,"Dan Burkert <dan@danburkert.com>, Lucio Franco <luciofranco14@gmail.com>, Casper Meijn <casper@meijn.net>, Tokio Contributors <team@tokio.rs>"
149153
prost-types,https://github.com/tokio-rs/prost,Apache-2.0,"Dan Burkert <dan@danburkert.com>, Lucio Franco <luciofranco14@gmail.com>, Casper Meijn <casper@meijn.net>, Tokio Contributors <team@tokio.rs>"
@@ -158,6 +162,7 @@ r-efi,https://github.com/r-efi/r-efi,MIT OR Apache-2.0 OR LGPL-2.1-or-later,The
158162
rand,https://github.com/rust-random/rand,MIT OR Apache-2.0,"The Rand Project Developers, The Rust Project Developers"
159163
rand_chacha,https://github.com/rust-random/rand,MIT OR Apache-2.0,"The Rand Project Developers, The Rust Project Developers, The CryptoCorrosion Contributors"
160164
rand_core,https://github.com/rust-random/rand,MIT OR Apache-2.0,"The Rand Project Developers, The Rust Project Developers"
165+
rand_xorshift,https://github.com/rust-random/rngs,MIT OR Apache-2.0,"The Rand Project Developers, The Rust Project Developers"
161166
redox_syscall,https://gitlab.redox-os.org/redox-os/syscall,MIT,Jeremy Soller <jackpot51@gmail.com>
162167
regex,https://github.com/rust-lang/regex,MIT OR Apache-2.0,"The Rust Project Developers, Andrew Gallant <jamslam@gmail.com>"
163168
regex-automata,https://github.com/rust-lang/regex,MIT OR Apache-2.0,"The Rust Project Developers, Andrew Gallant <jamslam@gmail.com>"
@@ -174,6 +179,7 @@ rustls-native-certs,https://github.com/rustls/rustls-native-certs,Apache-2.0 OR
174179
rustls-pemfile,https://github.com/rustls/pemfile,Apache-2.0 OR ISC OR MIT,The rustls-pemfile Authors
175180
rustls-pki-types,https://github.com/rustls/pki-types,MIT OR Apache-2.0,The rustls-pki-types Authors
176181
rustls-webpki,https://github.com/rustls/webpki,ISC,The rustls-webpki Authors
182+
rusty-fork,https://github.com/altsysrq/rusty-fork,MIT OR Apache-2.0,Jason Lingle
177183
ryu,https://github.com/dtolnay/ryu,Apache-2.0 OR BSL-1.0,David Tolnay <dtolnay@gmail.com>
178184
safemem,https://github.com/abonander/safemem,MIT OR Apache-2.0,Austin Bonander <austin.bonander@gmail.com>
179185
schannel,https://github.com/steffengy/schannel-rs,MIT,"Steven Fackler <sfackler@gmail.com>, Steffen Butzer <steffen.butzer@outlook.com>"
@@ -224,6 +230,7 @@ tokio-rustls,https://github.com/rustls/tokio-rustls,MIT OR Apache-2.0,The tokio-
224230
tokio-stream,https://github.com/tokio-rs/tokio,MIT,Tokio Contributors <team@tokio.rs>
225231
tokio-util,https://github.com/tokio-rs/tokio,MIT,Tokio Contributors <team@tokio.rs>
226232
tonic,https://github.com/hyperium/tonic,MIT,Lucio Franco <luciofranco14@gmail.com>
233+
tonic-prost,https://github.com/hyperium/tonic,MIT,Lucio Franco <luciofranco14@gmail.com>
227234
tonic-types,https://github.com/hyperium/tonic,MIT,"Lucio Franco <luciofranco14@gmail.com>, Rafael Lemos <flemos.rafael.dev@gmail.com>"
228235
tower,https://github.com/tower-rs/tower,MIT,Tower Maintainers <team@tower-rs.com>
229236
tower-http,https://github.com/tower-rs/tower-http,MIT,Tower Maintainers <team@tower-rs.com>
@@ -237,6 +244,7 @@ tracing-subscriber,https://github.com/tokio-rs/tracing,MIT,"Eliza Weisman <eliza
237244
try-lock,https://github.com/seanmonstar/try-lock,MIT,Sean McArthur <sean@seanmonstar.com>
238245
twoway,https://github.com/bluss/twoway,MIT OR Apache-2.0,bluss
239246
typenum,https://github.com/paholg/typenum,MIT OR Apache-2.0,"Paho Lurie-Gregg <paho@paholg.com>, Andre Bogus <bogusandre@gmail.com>"
247+
unarray,https://github.com/cameron1024/unarray,MIT OR Apache-2.0,The unarray Authors
240248
uncased,https://github.com/SergioBenitez/uncased,MIT OR Apache-2.0,Sergio Benitez <sb@sergio.bz>
241249
unicase,https://github.com/seanmonstar/unicase,MIT OR Apache-2.0,Sean McArthur <sean@seanmonstar.com>
242250
unicode-ident,https://github.com/dtolnay/unicode-ident,(MIT OR Apache-2.0) AND Unicode-3.0,David Tolnay <dtolnay@gmail.com>
@@ -248,6 +256,7 @@ ustr,https://github.com/anderslanglands/ustr,BSD-2-Clause-Patent,Anders Langland
248256
utf8_iter,https://github.com/hsivonen/utf8_iter,Apache-2.0 OR MIT,Henri Sivonen <hsivonen@hsivonen.fi>
249257
valuable,https://github.com/tokio-rs/valuable,MIT,The valuable Authors
250258
value-bag,https://github.com/sval-rs/value-bag,Apache-2.0 OR MIT,Ashley Mannix <ashleymannix@live.com.au>
259+
wait-timeout,https://github.com/alexcrichton/wait-timeout,MIT OR Apache-2.0,Alex Crichton <alex@alexcrichton.com>
251260
want,https://github.com/seanmonstar/want,MIT,Sean McArthur <sean@seanmonstar.com>
252261
wasi,https://github.com/bytecodealliance/wasi,Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT,The Cranelift Project Developers
253262
wasip2,https://github.com/bytecodealliance/wasi-rs,Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT,The wasip2 Authors

0 commit comments

Comments
 (0)