use iterator-based rangesearch if support ann_iterator (faiss_hnsw)

[alwayslove2013]

No description provided.

[mergify]

@alwayslove2013 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

1. If you're fixing a bug, label it as kind/bug.
2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

[alwayslove2013]

/kind improvement

[alexanderguzhva]

@alwayslove2013 Do I get it correct that we're deprecating the range search completely?

[alwayslove2013]

@alexanderguzhva In knowhere, there are two approaches to range_search. One is an internally implemented range_search within a specific index (override, like ivf / faiss_hnsw), while the other is a unified method from the parent class (index_node) that utilizes an iterator. This iterator continuously calls next() to collect the best results that meet the specified range.

knowhere/include/knowhere/index/index_node.h

Lines 155 to 171 in 8a705a0

	/**
	* @brief Performs a range search operation on the index.
	*
	* This method provides a default implementation of range search based on the `AnnIterator`, assuming the iterator
	* will buffer an expanded range and return the closest elements on each Next() call. It can be overridden by
	* derived classes for more efficient implementations.
	*
	* @param dataset Query vectors.
	* @param cfg
	* @param bitset A BitsetView object for filtering results.
	* @return An expected<> object containing the range search results or an error.
	* @note Since the config object needs to be held in a future or lambda function, a smart pointer is required to
	* delay its release.
	*/
	virtual expected<DataSetPtr>
	RangeSearch(const DataSetPtr dataset, std::unique_ptr<Config> cfg,
	const BitsetView& bitset) const { // TODO: @alwayslove2013 test with mock AnnIterator after we

It's a historical misalignment between milvus and knowhere. Previously, it was defined that knowhere would return all results that fit within the range. However, Milvus does not require all results and operates under a k limit (which we can name range_search_k). If the specified range is too large while range_search_k is relatively small, it can lead to significant inefficiencies. The iterator-based range_search can appropriately halt when sufficient results are collected.

[chasingegg]

/lgtm

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alwayslove2013, chasingegg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [chasingegg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment