Skip to content

Commit f7d471d

Browse files
authored
feat(trace-utils)!: introduce VecMap datastructure (#2022)
# What does this PR do? This PR is the first of a series that backports the change buffer implementation for dd-trace-js from experimentation to production in libdatadog. This PR introduces a replacement for `HashMap`: `VecMap`. It's not used anywhere else; this is left for follow-up PRs, but the idea is to use it for the `meta`, `metrics`, and `meta_struct` fields of v04 spans. # Motivation The change-buffer module (introduced in subsequent PRs) experimentations showed that we spend quite some time expanding hashmaps and thus re-hashing during span reconstruction. Introducing a map-like data structure that is optimized for construction/insertion (which is the vast majority of operations) showed to be beneficial. Even if not optimized for reads, we expect that given the small cardinality of meta and metrics in a span (< 20 entries in general), linear scan is actually going to perform well because of cache locality. Removal is one operation that is made quite more expensive (although still linear), but can be mitigated later with tombstones (also still linear, but doesn't require shifting). Tombstones aren't implemented for this first version. # Additional Notes Deduplication is performed at serialization time. We initially set up for sort + deduplication to avoid allocation, but this would require larger changes in libdatadog by requiring Span values to be `Ord`. For now, deduplication is based on a `HashSet` since `SpanData::Text` is already `Hash`, but this incurs one allocation. I think it's reasonable because the serialization path is not hot anyway, and we can later request `Ord` on values to go the in-place `sort_dedup` route instead if desired. # How to test the change? Run tests. Co-authored-by: yann.hamdaoui <yann.hamdaoui@datadoghq.com>
1 parent 8ac7358 commit f7d471d

2 files changed

Lines changed: 469 additions & 0 deletions

File tree

libdd-trace-utils/src/span/mod.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
pub mod trace_utils;
55
pub mod v04;
66
pub mod v05;
7+
pub mod vec_map;
78

89
use crate::msgpack_decoder::decode::buffer::read_string_ref_nomut;
910
use crate::msgpack_decoder::decode::error::DecodeError;

0 commit comments

Comments
 (0)