metadata: improved performance

[ktalg]

Addresses: #8860

Description

This PR introduces an experimental v2 implementation for outgoing metadata append handling.

The goal here is not to replace the current implementation yet, but to validate an alternative internal structure and measure its performance characteristics.

The main idea is to avoid copying the entire append history on every AppendToOutgoingContext call.

---

Background

The current implementation stores appended kv pairs in a [][]string slice. Every time we append:

added := make([][]string, len(md.added)+1)
copy(added, md.added)

So each append copies the full append history.

This means:

• Append cost grows with depth
• Allocation size grows with depth
• Deep append chains become increasingly expensive

---

What v2 does

In v2, I changed the append storage to a persistent linked structure:

type rawMDAddedV2 struct {
    added []string
    prev  *rawMDAddedV2
}

Each append:

• Allocates one node
• Links it to the previous tail
• Does NOT copy previous append history

So append becomes effectively O(1) per call instead of O(n) with respect to append depth.

---

Complexity comparison

Operation	Old Implementation	v2 Implementation
Append (single call)	O(n) (copy full append history)	O(1)
Append (depth = N)	O(N²) total cost	O(N) total cost
FromOutgoingContext	O(M + N)	O(M + N)

Where:

• M = number of base metadata entries
• N = total number of appended kv pairs

Important note:

FromOutgoingContext still builds a new MD and copies values, so its asymptotic complexity remains unchanged in v2.

---

Benchmark Results

cpu: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
BenchmarkOutgoingContextAPIComparison/append_depth_1/append_old-8                9436165               126.1 ns/op           168 B/op          4 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_1/append_v2-8                 9535257               131.4 ns/op           168 B/op          4 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_1/from_old-8                  3840616               315.0 ns/op           464 B/op          5 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_1/from_v2-8                   4016422               302.0 ns/op           464 B/op          5 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_4/append_old-8                1946552               593.2 ns/op           824 B/op         16 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_4/append_v2-8                 2278744               531.9 ns/op           672 B/op         16 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_4/from_old-8                  1705720               730.7 ns/op          1048 B/op         11 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_4/from_v2-8                   1746504               672.7 ns/op          1048 B/op         11 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_16/append_old-8                406848              2976 ns/op            5672 B/op         64 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_16/append_v2-8                 558073              2102 ns/op            2688 B/op         64 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_16/from_old-8                  630596              1666 ns/op            3800 B/op         15 allocs/op
BenchmarkOutgoingContextAPIComparison/append_depth_16/from_v2-8                   718364              1648 ns/op            3800 B/op         15 allocs/op

---

Why append improves but from does not

Append improves because:

• v2 removes repeated copying of append history.
• Each append only allocates one node instead of copying the entire slice of previous append entries.

From does not significantly improve because:

• It still constructs a fresh MD
• It still deep-copies values
• It still merges appended entries into the map

So the dominant cost of FromOutgoingContext remains unchanged.

---

Notes

This is currently experimental and meant for performance validation.

If we want to significantly improve FromOutgoingContext, we would likely need:

• A zero-copy view API
• Or a cached flattened representation
• Or a different metadata representation entirely

Happy to iterate based on feedback.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.33%. Comparing base (8f47d36) to head (daadc74).
⚠️ Report is 51 commits behind head on master.

Files with missing lines	Patch %	Lines
metadata/metadata.go	0.00%	38 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8942      +/-   ##
==========================================
+ Coverage   83.27%   83.33%   +0.05%     
==========================================
  Files         417      417              
  Lines       32986    33024      +38     
==========================================
+ Hits        27468    27519      +51     
- Misses       4096     4101       +5     
+ Partials     1422     1404      -18     

Files with missing lines	Coverage Δ
metadata/metadata.go	68.49% <0.00%> (-24.10%)	⬇️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ktalg]

@easwars

From the benchmarks, v2 clearly improves AppendToOutgoingContext for deeper append chains, but doesn’t significantly change the cost of FromOutgoingContext.

Do you think this approach is worth merging as-is?

If it makes sense, I can consolidate the current implementation into a single internal structure (just changing the underlying layout), or explore shaping this into part of a future metadata v2 API.

[easwars]

Thanks for the PR, @ktalg! We’re currently focusing our resources on other priorities and try to avoid keeping drafts open in the interim. I'll close this for now, but we appreciate the effort. Let's revisit this down the road when we have more cycles to review!