feat: add support for PolarDB#737
Conversation
|
/assign @XuanYang-cn |
XuanYang-cn
left a comment
There was a problem hiding this comment.
Thanks for the contribution! The overall structure follows the VectorDBBench client pattern well. A few items to address:
CI blocker — lint errors (12 total)
cli.py:123: line too long (132 > 120 chars)cli.py,polardb.py: 8 missing type annotations (_build_db_config(parameters),_create_db_table(cursor, ...),vector_to_hex(v),_insert_batch(...))polardb.py: 4 SQL injection warnings — add# noqa: S608(standard pattern used by AliSQL, MariaDB, TiDB, OceanBase)
DROP DATABASE is too destructive (polardb.py:599)
cursor.execute(f"DROP DATABASE IF EXISTS {self.db_config['database']}")Other MySQL-based clients (AliSQL, MariaDB) only drop the table, not the entire database. This could wipe out unrelated data. Please change to DROP TABLE IF EXISTS.
raise e from None hides traceback (polardb.py:656)
raise e from None suppresses the original exception chain. Use raise instead to preserve the full traceback for debugging.
Redundant COMMIT with autocommit=True
The connection is created with autocommit=True, but there are several explicit cursor.execute("COMMIT") calls throughout. Pick one approach — either use autocommit or manage transactions manually.
Config field duplication (config.py)
PolarDBHNSWFlatConfig, PolarDBHNSWPQConfig, and PolarDBHNSWSQConfig duplicate M, ef_construction, ef_search, insert_workers, post_load_index. Consider extracting a shared PolarDBHNSWBaseConfig (similar to how PolarDBHNSWTypedDict already works for the CLI side).
- add PolarDB vector search client with FAISS_HNSW_FLAT, FAISS_HNSW_PQ, and FAISS_HNSW_SQ index types - add CLI integration with hnswflat, hnswpq, and hnswsq benchmark commands - add frontend (Streamlit) UI support with index type selection, HNSW/PQ/SQ parameter configuration
|
hi @XuanYang-cn , thanks for your comments. All issues have been fixed and it seems that all checks have passed. Please review again. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: nanlongyu, XuanYang-cn The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Description
This PR introduces support for PolarDB, an Alibaba Cloud cloud-native relational database service, as a new database engine in VectorDBBench. PolarDB features IMCI (In-Memory Column Index) based vector search capabilities, leveraging FAISS as its underlying vector engine. It is compatible with the MySQL protocol, offering high-performance and cost-effective vector retrieval on top of a fully-managed cloud-native architecture with features like storage-compute separation, read-write splitting, and elastic scaling — making it well-suited for enterprise-level AI and similarity search scenarios.
Supported Index Types: