Skip to content

Commit bbe9067

Browse files
localai-botlocalai-bot
andauthored
docs: Add troubleshooting guide for embedding models (fixes #9064) (#9065)
docs: Add troubleshooting guide for embedding models (#9064) - Add section on using gallery models for embeddings - Document common issues with embedding model configuration - Add troubleshooting guide for Qwen3 embedding models - Include correct configuration examples for Qwen3-Embedding-4B - Document context size limits and dimension parameters - Add table of Qwen3 embedding model specifications Fixes #9064 Signed-off-by: localai-bot <localai-bot@localai.io> Co-authored-by: localai-bot <localai-bot@localai.io>
1 parent 9a9da06 commit bbe9067

1 file changed

Lines changed: 97 additions & 2 deletions

File tree

docs/content/features/embeddings.md

Lines changed: 97 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
21
+++
32
disableToc = false
43
title = "🧠 Embeddings"
@@ -14,6 +13,28 @@ For the API documentation you can refer to the OpenAI docs: https://platform.ope
1413

1514
The embedding endpoint is compatible with `llama.cpp` models, `bert.cpp` models and sentence-transformers models available in huggingface.
1615

16+
## Using Gallery Models
17+
18+
LocalAI provides a model gallery with pre-configured embedding models. To use a gallery model:
19+
20+
1. Ensure the model is available in the gallery (check [Model Gallery]({{%relref "features/model-gallery" %}}))
21+
2. Use the model name directly in your API calls
22+
23+
Example gallery models:
24+
- `qwen3-embedding-4b` - Qwen3 Embedding 4B model
25+
- `qwen3-embedding-8b` - Qwen3 Embedding 8B model
26+
- `qwen3-embedding-0.6b` - Qwen3 Embedding 0.6B model
27+
28+
### Example: Using Qwen3-Embedding-4B from Gallery
29+
30+
```bash
31+
curl http://localhost:8080/embeddings -X POST -H "Content-Type: application/json" -d '{
32+
"input": "My text to embed",
33+
"model": "qwen3-embedding-4b",
34+
"dimensions": 2560
35+
}'
36+
```
37+
1738
## Manual Setup
1839

1940
Create a `YAML` config file in the `models` directory. Specify the `backend` and the model file.
@@ -73,4 +94,78 @@ curl http://localhost:8080/embeddings -X POST -H "Content-Type: application/json
7394

7495
## 💡 Examples
7596

76-
- Example that uses LLamaIndex and LocalAI as embedding: [here](https://github.com/mudler/LocalAI-examples/tree/main/query_data).
97+
- Example that uses LLamaIndex and LocalAI as embedding: [here](https://github.com/mudler/LocalAI-examples/tree/main/query_data).
98+
99+
## ⚠️ Common Issues and Troubleshooting
100+
101+
### Issue: Embedding model not returning correct results
102+
103+
**Symptoms:**
104+
- Model returns empty or incorrect embeddings
105+
- API returns errors when calling embedding endpoint
106+
107+
**Common Causes:**
108+
109+
1. **Incorrect model filename**: Ensure you're using the correct filename from the gallery or your model file location.
110+
- Gallery models use specific filenames (e.g., `Qwen3-Embedding-4B-Q4_K_M.gguf`)
111+
- Check the [Model Gallery]({{%relref "features/model-gallery" %}}) for correct filenames
112+
113+
2. **Context size mismatch**: Ensure your `context_size` setting doesn't exceed the model's maximum context length.
114+
- Qwen3-Embedding-4B: max 32k (32768) context
115+
- Qwen3-Embedding-8B: max 32k (32768) context
116+
- Qwen3-Embedding-0.6B: max 32k (32768) context
117+
118+
3. **Missing `embeddings: true` flag**: The model configuration must have `embeddings: true` set.
119+
120+
**Correct Configuration Example:**
121+
122+
```yaml
123+
name: qwen3-embedding-4b
124+
backend: llama-cpp
125+
embeddings: true
126+
context_size: 32768
127+
parameters:
128+
model: Qwen3-Embedding-4B-Q4_K_M.gguf
129+
```
130+
131+
### Issue: Dimension mismatch
132+
133+
**Symptoms:**
134+
- Returned embedding dimensions don't match expected dimensions
135+
136+
**Solution:**
137+
- Use the `dimensions` parameter in your API request to specify the output dimension
138+
- Qwen3-Embedding models support dimensions from 32 to 2560 (4B) or 4096 (8B)
139+
140+
```bash
141+
curl http://localhost:8080/embeddings -X POST -H "Content-Type: application/json" -d '{
142+
"input": "My text",
143+
"model": "qwen3-embedding-4b",
144+
"dimensions": 1024
145+
}'
146+
```
147+
148+
### Issue: Model not found
149+
150+
**Symptoms:**
151+
- API returns 404 or "model not found" error
152+
153+
**Solution:**
154+
- Ensure the model is properly configured in the models directory
155+
- Check that the model name in your API request matches the `name` field in the configuration
156+
- For gallery models, ensure the gallery is properly loaded
157+
158+
## Qwen3 Embedding Models Specifics
159+
160+
The Qwen3 Embedding series models have these characteristics:
161+
162+
| Model | Parameters | Max Context | Max Dimensions | Supported Languages |
163+
|-------|------------|-------------|----------------|---------------------|
164+
| qwen3-embedding-0.6b | 0.6B | 32k | 1024 | 100+ |
165+
| qwen3-embedding-4b | 4B | 32k | 2560 | 100+ |
166+
| qwen3-embedding-8b | 8B | 32k | 4096 | 100+ |
167+
168+
All models support:
169+
- User-defined output dimensions (32 to max dimensions)
170+
- Multilingual text embedding (100+ languages)
171+
- Instruction-tuned embedding with custom instructions

0 commit comments

Comments
 (0)