Skip to content

fix(oss-opensearch): Resolve streaming crashes and improve code reliability#568

Merged
alwayslove2013 merged 7 commits intozilliztech:mainfrom
akhilpathivada:fix/oss-opensearch-retry-labels-bug
Jul 25, 2025
Merged

fix(oss-opensearch): Resolve streaming crashes and improve code reliability#568
alwayslove2013 merged 7 commits intozilliztech:mainfrom
akhilpathivada:fix/oss-opensearch-retry-labels-bug

Conversation

@akhilpathivada
Copy link
Copy Markdown
Contributor

@akhilpathivada akhilpathivada commented Jul 17, 2025

Summary

This PR enhances the OSS OpenSearch client with critical bug fixes, improved configurability, performance optimizations, and comprehensive code refactoring for better maintainability.

Key Changes

🐛 Bug Fixes

  • Fix streaming insertion retry crash: Resolved crash in streaming tests during bulk insertion retries
  • Fix _id field extraction: Corrected document ID extraction for accurate recall calculation
  • Add cosine normalization: Proper vector normalization for cosine similarity metrics

⚙️ Configuration Enhancements

  • Configurable refresh interval: Made refresh interval configurable through UI for better performance tuning

🏗️ Code Architecture Improvements

  • Extract OpenSearchSettingsManager: Centralized cluster and index settings management
  • Extract BulkInsertManager: Isolated bulk insertion logic with enhanced validation and error handling
  • Extract SearchQueryBuilder: Clean separation of query construction logic
  • Enhanced type safety: Comprehensive type hints and improved error handling
  • Removed dead code: Eliminated unused get_index_setting method
  • Simplified exception handling: Streamlined OpenSearchException class

📊 Impact

  • Reliability: Eliminates crashes in streaming scenarios and improves error recovery
  • Performance: Better configuration control and optimized bulk operations
  • Maintainability: Clean architecture following SOLID principles
  • Developer Experience: Better type safety and clearer code organization

Files Changed

  • vectordb_bench/backend/clients/oss_opensearch/oss_opensearch.py - Core client improvements
  • vectordb_bench/frontend/config/dbCaseConfigs.py - UI configuration enhancement

Testing

  • Streaming insertion scenarios
  • Recall calculation accuracy
  • Configuration UI functionality
  • Bulk insertion with various client counts

@akhilpathivada akhilpathivada changed the title fix: OSS OpenSearch insertion retry missing labels_data parameter fix(oss-opensearch): bug fixes and make refresh interval configurable in UI Jul 17, 2025
@akhilpathivada akhilpathivada changed the title fix(oss-opensearch): bug fixes and make refresh interval configurable in UI fix(oss-opensearch): Resolve streaming crashes and improve code reliability Jul 21, 2025
@akhilpathivada akhilpathivada marked this pull request as ready for review July 21, 2025 19:37
@akhilpathivada
Copy link
Copy Markdown
Contributor Author

/assign @XuanYang-cn @alwayslove2013

@alwayslove2013
Copy link
Copy Markdown
Contributor

@akhilpathivada plz fix the lint issues~

pip install black ruff --upgrade
make format
make lint

@sre-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: akhilpathivada, alwayslove2013
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@alwayslove2013 alwayslove2013 merged commit d09ea1e into zilliztech:main Jul 25, 2025
4 checks passed
euphoria0-0 pushed a commit to CryptoLabInc/VectorDBBench that referenced this pull request Nov 21, 2025
…bility (zilliztech#568)

* Fix OSS OpenSearch insertion retry crash in streaming tests

* make refresh interval configurable in UI

* normalize vectors for cosine function

* fix: correct OS _id field extraction for recall calculation

* refactor(oss-opensearch): extract helper classes and improve code organization

* fix: lint issues
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants