Skip to content

Latest commit

 

History

History
59 lines (46 loc) · 2.65 KB

File metadata and controls

59 lines (46 loc) · 2.65 KB

Benchmark Dashboard

Interactive visualization of performance measurements for the a2a-protocol-sdk Rust implementation. All data is auto-generated from Criterion.rs benchmark results by CI.

Open the interactive dashboard: Benchmark Dashboard →

What the dashboard shows

Tab Contents
Overview Key performance highlights, payload scaling, event queue throughput
Transport HTTP round-trip latency, JSON-RPC vs REST, connection reuse, error paths
Serde Per-type serialization cost, batch scaling, SerBuffer vs to_vec, interceptor overhead
Concurrency Sends, streams, and store operations at 1-64 concurrent connections
Data Volume Store operations at 1K-100K tasks, history depth scaling
Backpressure Stream volume, slow consumer impact, concurrent streams, timer calibration
Enterprise Multi-tenant isolation, CORS, rate limiting, eviction, large histories, pagination
Production Agent burst scaling, E2E orchestration, cold start, push config CRUD, cross-language
Memory Allocation timing under counting allocator, bytes per payload, history scaling
All Results Searchable table of all 267 individual benchmark measurements

Methodology

All benchmarks use Criterion.rs with median ± MAD (Median Absolute Deviation) as the robust central tendency measure. Each benchmark runs 100 samples with warm-up iterations to avoid cold-start artifacts.

  • Environment: CI runners (ubuntu-latest) — use for relative comparisons and regression detection, not absolute performance guarantees
  • Transport: All HTTP benchmarks use loopback (127.0.0.1) to isolate SDK overhead from network latency
  • Determinism: Fixed task IDs and payloads inside measurement loops

Reproducing locally

# Run all benchmarks and generate the dashboard
cargo bench -p a2a-benchmarks
./benches/scripts/generate_dashboard.sh

# Open the dashboard
open book/src/reference/benchmark-dashboard.html

See also