Skip to content

feat: add support for PolarDB#737

Merged
XuanYang-cn merged 1 commit intozilliztech:mainfrom
nanlongyu:main
Apr 1, 2026
Merged

feat: add support for PolarDB#737
XuanYang-cn merged 1 commit intozilliztech:mainfrom
nanlongyu:main

Conversation

@nanlongyu
Copy link
Copy Markdown
Contributor

Description

This PR introduces support for PolarDB, an Alibaba Cloud cloud-native relational database service, as a new database engine in VectorDBBench. PolarDB features IMCI (In-Memory Column Index) based vector search capabilities, leveraging FAISS as its underlying vector engine. It is compatible with the MySQL protocol, offering high-performance and cost-effective vector retrieval on top of a fully-managed cloud-native architecture with features like storage-compute separation, read-write splitting, and elastic scaling — making it well-suited for enterprise-level AI and similarity search scenarios.

Supported Index Types:

  • HNSW:High-performance graph-based indexing with full-precision vectors, providing the best recall quality.
  • HNSWPQ:Combines HNSW graph navigation with Product Quantization for memory-efficient retrieval at scale.
  • HNSWSQ:Combines HNSW graph navigation with Scalar Quantization, balancing speed, memory usage, and recall.

@nanlongyu
Copy link
Copy Markdown
Contributor Author

/assign @XuanYang-cn

@XuanYang-cn XuanYang-cn self-assigned this Mar 24, 2026
Copy link
Copy Markdown
Collaborator

@XuanYang-cn XuanYang-cn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! The overall structure follows the VectorDBBench client pattern well. A few items to address:

CI blocker — lint errors (12 total)

  • cli.py:123: line too long (132 > 120 chars)
  • cli.py, polardb.py: 8 missing type annotations (_build_db_config(parameters), _create_db_table(cursor, ...), vector_to_hex(v), _insert_batch(...))
  • polardb.py: 4 SQL injection warnings — add # noqa: S608 (standard pattern used by AliSQL, MariaDB, TiDB, OceanBase)

DROP DATABASE is too destructive (polardb.py:599)

cursor.execute(f"DROP DATABASE IF EXISTS {self.db_config['database']}")

Other MySQL-based clients (AliSQL, MariaDB) only drop the table, not the entire database. This could wipe out unrelated data. Please change to DROP TABLE IF EXISTS.

raise e from None hides traceback (polardb.py:656)

raise e from None suppresses the original exception chain. Use raise instead to preserve the full traceback for debugging.

Redundant COMMIT with autocommit=True

The connection is created with autocommit=True, but there are several explicit cursor.execute("COMMIT") calls throughout. Pick one approach — either use autocommit or manage transactions manually.

Config field duplication (config.py)

PolarDBHNSWFlatConfig, PolarDBHNSWPQConfig, and PolarDBHNSWSQConfig duplicate M, ef_construction, ef_search, insert_workers, post_load_index. Consider extracting a shared PolarDBHNSWBaseConfig (similar to how PolarDBHNSWTypedDict already works for the CLI side).

- add PolarDB vector search client with FAISS_HNSW_FLAT, FAISS_HNSW_PQ, and FAISS_HNSW_SQ index types
- add CLI integration with hnswflat, hnswpq, and hnswsq benchmark commands
- add frontend (Streamlit) UI support with index type selection, HNSW/PQ/SQ parameter configuration
@nanlongyu
Copy link
Copy Markdown
Contributor Author

hi @XuanYang-cn , thanks for your comments. All issues have been fixed and it seems that all checks have passed. Please review again.

@sre-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nanlongyu, XuanYang-cn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@XuanYang-cn XuanYang-cn merged commit 6f7a153 into zilliztech:main Apr 1, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants