|
7 | 7 | --- |
8 | 8 | # Benchmark Text Embedding Models for RecSys in 2026 |
9 | 9 |
|
10 | | -In the 2025 post [Text Embedding Benchmark for Recommender Systems](./embedding-benchmark.md), we benchmarked the performance of text embedding models in similarity-based recommendations. Within six months of that post's publication, Alibaba Cloud and Google launched a new generation of open-source text embedding models: [qwen3-embedding](https://github.com/QwenLM/Qwen3-Embedding) by Alibaba Cloud and [embeddinggemma](https://ai.google.dev/gemma/docs/embeddinggemma) by Google. Recently, the [gorse-cli](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-cli) also added a benchmarking feature for text embedding models. This post will use [gorse-cli](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-cli) and the playground dataset to conduct a comprehensive benchmark of popular open-source text embedding models. |
| 10 | +In the 2025 post [Text Embedding Benchmark for Recommender Systems](./embedding-benchmark.md), we benchmarked the performance of text embedding models in similarity-based recommendations. Within six months of that post's publication, Alibaba Cloud and Google launched a new generation of open-source text embedding models: [qwen3-embedding](https://github.com/QwenLM/Qwen3-Embedding) by Alibaba Cloud and [embeddinggemma](https://ai.google.dev/gemma/docs/embeddinggemma) by Google. Recently, the [gorse-bench](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-bench) also added a benchmarking feature for text embedding models. This post will use [gorse-bench](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-bench) and the playground dataset to conduct a comprehensive benchmark of popular open-source text embedding models. |
11 | 11 |
|
12 | 12 | ## Evaluation: 1-shot Similarity-based Recommendation |
13 | 13 |
|
@@ -39,10 +39,10 @@ OPENAI_BASE_URL="https://integrate.api.nvidia.com/v1" |
39 | 39 | OPENAI_AUTH_TOKEN="NVIDIA_API_KEY" |
40 | 40 | ``` |
41 | 41 |
|
42 | | -Compile [gorse-cli](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-cli) from Gorse repository and run the following command to evaluate the performance of the text embedding model: |
| 42 | +Compile [gorse-bench](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-bench) from Gorse repository and run the following command to evaluate the performance of the text embedding model: |
43 | 43 |
|
44 | 44 | ```bash |
45 | | -./gorse-cli bench-embedding --config ./config/config.toml \ |
| 45 | +./gorse-bench embedding --config ./config/config.toml \ |
46 | 46 | --text-column item.Comment \ |
47 | 47 | --embedding-model qwen3-embedding:0.6b \ |
48 | 48 | --embedding-dimensions 1024 \ |
@@ -146,4 +146,4 @@ For text embedding models for recommender systems in 2026, we offer the followin |
146 | 146 | - **Cost-Efficiency/Private Deployment**: [qwen3-embedding:4b](https://huggingface.co/Qwen/Qwen3-Embedding-4B) is the current king of cost-efficiency. It achieves recommendation accuracy comparable to commercial models with fewer parameters. |
147 | 147 | - **Low Latency/Edge**: [qwen3-embedding:0.6b](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) with 64 or 128-dimension is the best lightweight solution. |
148 | 148 |
|
149 | | -While this post provides some guidance, it is recommended to use [gorse-cli](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-cli) to evaluate on your own dataset to choose the text embedding model that best fits your specific business scenario. |
| 149 | +While this post provides some guidance, it is recommended to use [gorse-bench](https://github.com/gorse-io/gorse/tree/master/cmd/gorse-bench) to evaluate on your own dataset to choose the text embedding model that best fits your specific business scenario. |
0 commit comments