Skip to content

feat(valkey): add Valkey indexer and retriever components#883

Open
MatthiasHowellYopp wants to merge 1 commit into
cloudwego:mainfrom
MatthiasHowellYopp:feat/valkey-indexer-retriever
Open

feat(valkey): add Valkey indexer and retriever components#883
MatthiasHowellYopp wants to merge 1 commit into
cloudwego:mainfrom
MatthiasHowellYopp:feat/valkey-indexer-retriever

Conversation

@MatthiasHowellYopp

Copy link
Copy Markdown

What type of PR is this?

feat: A new feature

Check the PR title.

  • This PR title match the format: <type>(optional scope): <description>
  • The description of this PR title is user-oriented and clear enough for others to understand.
  • Attach the PR updating the user documentation if the current PR requires user awareness at the usage level.

(Optional) Translate the PR title into Chinese.

feat(valkey): 添加 Valkey 索引器和检索器组件

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:
Add native Valkey Retriever and Indexer components using the valkey-glide-go client.

Components:

  • components/retriever/valkey/ — implements retriever.Retriever
  • components/indexer/valkey/ — implements indexer.Indexer

Retriever capabilities:

  • KNN vector queries via Valkey Search (FT.SEARCH)
  • Hybrid search with filter expressions
  • Configurable vector field, return fields, dialect
  • Embedding integration for automatic query vectorization
  • Full Eino callback integration (OnStart, OnEnd, OnError)

Indexer capabilities:

  • Pipeline batch document storage using HSET and JSON.SET
  • Automatic embedding generation with configurable batch size
  • Support for both Hash and JSON document types
  • Configurable key prefix and field mapping via DocumentToHashes/DocumentToJSON
  • Full Eino callback integration

Dependencies:

  • Client: github.com/valkey-io/valkey-glide/go/v2 v2.4.1
  • Backend: Valkey 9.1+ with Search module
  • Framework: github.com/cloudwego/eino v0.6.0

Testing:

  • Unit tests for both components (constructor validation, embedding errors, batch logic, document conversion)
  • Integration tests (gated with //go:build integration) verified against Valkey 9.1.0

zh(optional): 使用 valkey-glide-go 客户端添加原生 Valkey 检索器和索引器组件。

(Optional) Which issue(s) this PR fixes:

Fixes #879

(optional) The PR that updates user documentation:

N/A

@CLAassistant

CLAassistant commented Jun 10, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@MatthiasHowellYopp MatthiasHowellYopp force-pushed the feat/valkey-indexer-retriever branch 3 times, most recently from 72057b3 to 1dfd987 Compare June 19, 2026 13:45
Add native Valkey Retriever and Indexer components using the valkey-glide-go
client for use with Valkey 9.1+ and the Valkey Search module.

Retriever capabilities:
- KNN vector queries via FT.SEARCH
- Hybrid search with filter expressions
- Configurable vector field, return fields, dialect
- Embedding integration for automatic query vectorization
- Full Eino callback integration (OnStart, OnEnd, OnError)

Indexer capabilities:
- Pipeline batch document storage (HSET and JSON.SET)
- Automatic embedding generation with configurable batch size
- Support for both Hash and JSON document types
- Configurable key prefix and field mapping
- Full Eino callback integration

Closes cloudwego#879

Signed-off-by: Matthias Howell <matthias.howell@improving.com>
@MatthiasHowellYopp MatthiasHowellYopp force-pushed the feat/valkey-indexer-retriever branch from 1dfd987 to 191ee28 Compare July 2, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

feat(valkey): add Valkey indexer and retriever components

2 participants