Skip to content

feat: Add P95 latency metrics alongside P99 system-wide#564

Closed
akhilpathivada wants to merge 8 commits intozilliztech:mainfrom
akhilpathivada:feat/add-p95-latency-metrics
Closed

feat: Add P95 latency metrics alongside P99 system-wide#564
akhilpathivada wants to merge 8 commits intozilliztech:mainfrom
akhilpathivada:feat/add-p95-latency-metrics

Conversation

@akhilpathivada
Copy link
Copy Markdown
Contributor

📋 Summary

Adds P95 (95th percentile) latency metrics alongside existing P99 metrics across all VectorDBBench components for more comprehensive performance analysis.

🎯 Motivation

  • P95 provides a middle ground between P99 (slowest 1%) and average latency
  • More actionable for optimization compared to P99 alone
  • Industry standard metric for performance monitoring

🔧 Key Changes

  • Backend: Added P95 calculation in serial, concurrent, and streaming runners
  • Frontend: Added P95 selection option in streaming and concurrent pages
  • Results: Added P95 column to results table and exports
  • Backward Compatibility: Existing result files show 0.0 for P95 metrics

🎮 Usage

# Streaming/Concurrent pages now offer P95 option
latency_type = st.radio("Latency Type", options=["latency_p99", "latency_p95", "latency_avg"])

Results Table:

latency(p99) latency(p95) recall
12.5ms 8.3ms 0.95

🧪 Testing

  • All files compile successfully
  • Backward compatibility with existing result files
  • No breaking changes to existing APIs

📊 Impact

Generic support across all database clients and test cases. Users can now choose between P99 and P95 latency metrics based on their analysis needs.

@sre-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: akhilpathivada
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@akhilpathivada akhilpathivada marked this pull request as draft July 14, 2025 22:29
@akhilpathivada akhilpathivada marked this pull request as ready for review July 15, 2025 11:10
@akhilpathivada
Copy link
Copy Markdown
Contributor Author

akhilpathivada commented Jul 15, 2025

/assign @XuanYang-cn @alwayslove2013

@alwayslove2013
Copy link
Copy Markdown
Contributor

@akhilpathivada plz fix the lint issues~

pip install black ruff --upgrade
make format
make lint

akhilpathivada and others added 6 commits July 25, 2025 00:49
* Add OSS OpenSearch Client

* Update cli.py with OSSOpenSearch

* add ossopensearch cli command

* rollback num_per_batch to 100
Signed-off-by: min.tian <min.tian.cn@gmail.com>
Signed-off-by: min.tian <min.tian.cn@gmail.com>
Signed-off-by: min.tian <min.tian.cn@gmail.com>
- add new int filter to run tests with multiple filter rate values like[0.001,0.02,0.3,...,0.7,0.98,0.999] instead of just 0.01 and 0.99
- achieve compatibility for int filter case between new and old versions
- add int filter performance page
@akhilpathivada
Copy link
Copy Markdown
Contributor Author

Please review the new PR instead due to commit history issues: #573

Same functionality, cleaner Git history. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants