1-
21+++
32disableToc = false
43title = " 🧠 Embeddings"
@@ -14,6 +13,28 @@ For the API documentation you can refer to the OpenAI docs: https://platform.ope
1413
1514The embedding endpoint is compatible with ` llama.cpp ` models, ` bert.cpp ` models and sentence-transformers models available in huggingface.
1615
16+ ## Using Gallery Models
17+
18+ LocalAI provides a model gallery with pre-configured embedding models. To use a gallery model:
19+
20+ 1 . Ensure the model is available in the gallery (check [ Model Gallery] ({{%relref "features/model-gallery" %}}))
21+ 2 . Use the model name directly in your API calls
22+
23+ Example gallery models:
24+ - ` qwen3-embedding-4b ` - Qwen3 Embedding 4B model
25+ - ` qwen3-embedding-8b ` - Qwen3 Embedding 8B model
26+ - ` qwen3-embedding-0.6b ` - Qwen3 Embedding 0.6B model
27+
28+ ### Example: Using Qwen3-Embedding-4B from Gallery
29+
30+ ``` bash
31+ curl http://localhost:8080/embeddings -X POST -H " Content-Type: application/json" -d ' {
32+ "input": "My text to embed",
33+ "model": "qwen3-embedding-4b",
34+ "dimensions": 2560
35+ }'
36+ ```
37+
1738## Manual Setup
1839
1940Create a ` YAML ` config file in the ` models ` directory. Specify the ` backend ` and the model file.
@@ -73,4 +94,78 @@ curl http://localhost:8080/embeddings -X POST -H "Content-Type: application/json
7394
7495# # 💡 Examples
7596
76- - Example that uses LLamaIndex and LocalAI as embedding : [here](https://github.com/mudler/LocalAI-examples/tree/main/query_data).
97+ - Example that uses LLamaIndex and LocalAI as embedding : [here](https://github.com/mudler/LocalAI-examples/tree/main/query_data).
98+
99+ # # ⚠️ Common Issues and Troubleshooting
100+
101+ # ## Issue: Embedding model not returning correct results
102+
103+ **Symptoms:**
104+ - Model returns empty or incorrect embeddings
105+ - API returns errors when calling embedding endpoint
106+
107+ **Common Causes:**
108+
109+ 1. **Incorrect model filename** : Ensure you're using the correct filename from the gallery or your model file location.
110+ - Gallery models use specific filenames (e.g., `Qwen3-Embedding-4B-Q4_K_M.gguf`)
111+ - Check the [Model Gallery]({{%relref "features/model-gallery" %}}) for correct filenames
112+
113+ 2. **Context size mismatch** : Ensure your `context_size` setting doesn't exceed the model's maximum context length.
114+ - Qwen3-Embedding-4B : max 32k (32768) context
115+ - Qwen3-Embedding-8B : max 32k (32768) context
116+ - Qwen3-Embedding-0.6B : max 32k (32768) context
117+
118+ 3. **Missing `embeddings : true` flag**: The model configuration must have `embeddings: true` set.
119+
120+ **Correct Configuration Example:**
121+
122+ ` ` ` yaml
123+ name: qwen3-embedding-4b
124+ backend: llama-cpp
125+ embeddings: true
126+ context_size: 32768
127+ parameters:
128+ model: Qwen3-Embedding-4B-Q4_K_M.gguf
129+ ` ` `
130+
131+ # ## Issue: Dimension mismatch
132+
133+ **Symptoms:**
134+ - Returned embedding dimensions don't match expected dimensions
135+
136+ **Solution:**
137+ - Use the `dimensions` parameter in your API request to specify the output dimension
138+ - Qwen3-Embedding models support dimensions from 32 to 2560 (4B) or 4096 (8B)
139+
140+ ` ` ` bash
141+ curl http://localhost:8080/embeddings -X POST -H "Content-Type: application/json" -d '{
142+ "input": "My text",
143+ "model": "qwen3-embedding-4b",
144+ "dimensions": 1024
145+ }'
146+ ` ` `
147+
148+ # ## Issue: Model not found
149+
150+ **Symptoms:**
151+ - API returns 404 or "model not found" error
152+
153+ **Solution:**
154+ - Ensure the model is properly configured in the models directory
155+ - Check that the model name in your API request matches the `name` field in the configuration
156+ - For gallery models, ensure the gallery is properly loaded
157+
158+ # # Qwen3 Embedding Models Specifics
159+
160+ The Qwen3 Embedding series models have these characteristics :
161+
162+ | Model | Parameters | Max Context | Max Dimensions | Supported Languages |
163+ |-------|------------|-------------|----------------|---------------------|
164+ | qwen3-embedding-0.6b | 0.6B | 32k | 1024 | 100+ |
165+ | qwen3-embedding-4b | 4B | 32k | 2560 | 100+ |
166+ | qwen3-embedding-8b | 8B | 32k | 4096 | 100+ |
167+
168+ All models support :
169+ - User-defined output dimensions (32 to max dimensions)
170+ - Multilingual text embedding (100+ languages)
171+ - Instruction-tuned embedding with custom instructions
0 commit comments