Skip to content

Commit f1c6cec

Browse files
are-cesclaude
andauthored
LCORE-1037: address review nits in BYOK and RAG guides (#1842)
- Replace pdm with uv for embedding model download command - Remove unnecessary OpenAI API key note - Fix Ollama section: clarify no tool RAG but inline RAG supported - Remove empty References section from RAG guide - Fix incorrect embedding dimensions for all-mpnet-base-v2 (768, not 1024/384) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 388ea35 commit f1c6cec

3 files changed

Lines changed: 4 additions & 16 deletions

File tree

docs/byok_guide.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ You can use the embedding generation step mentioned in the rag-content repo:
161161

162162
```bash
163163
mkdir ./embeddings_model
164-
pdm run python ./scripts/download_embeddings_model.py -l ./embeddings_model/ -r sentence-transformers/all-mpnet-base-v2
164+
uv run python ./scripts/download_embeddings_model.py -l ./embeddings_model/ -r sentence-transformers/all-mpnet-base-v2
165165
```
166166

167167
#### Option 2: Manual Download and Configuration
@@ -340,10 +340,6 @@ rag:
340340
- company-docs
341341
```
342342

343-
> [!NOTE]
344-
> Your LLM inference provider (e.g., OpenAI, vLLM) must also be configured in your `run.yaml`.
345-
> For OpenAI, set the `OPENAI_API_KEY` environment variable.
346-
347343
### Example 2: Multiple Knowledge Sources with pgvector
348344

349345
A configuration combining a local FAISS store (via `byok_rag`) with a remote pgvector store (configured directly in the Llama Stack configuration file):

docs/rag_guide.md

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -223,11 +223,7 @@ Not yet supported.
223223

224224
### Ollama
225225

226-
The `remote::ollama` provider can be used for inference. However, it does not support tool calling, including RAG.
227-
While Ollama also exposes an OpenAI compatible endpoint that supports tool calling, it cannot currently be used due to limitations in the `remote::openai` provider.
228-
229-
Tool calling with Ollama is not yet supported.
230-
Currently, tool calling is not supported out of the box. Some experimental patches exist (including internal workarounds), but these are not officially released.
226+
The `remote::ollama` provider does not support tool calling, so RAG as a tool is not available. However, inline RAG is supported.
231227

232228
### vLLM Mistral
233229

@@ -386,7 +382,3 @@ You are a helpful assistant with access to a 'knowledge_search' tool. When users
386382

387383
The top-level `vector_stores` block in [`run.yaml`](../examples/run.yaml) may include `annotation_prompt_params` to control whether extra RAG annotation instructions are injected into the model prompt (for example, citation-style markers). The default configuration sets `enable_annotations: false` under that block to avoid unwanted annotations.
388384

389-
---
390-
391-
# References
392-

examples/lightspeed-stack-byok-okp-rag.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,14 +38,14 @@ byok_rag:
3838
- rag_id: ocp-docs # referenced in rag.inline / rag.tool
3939
rag_type: inline::faiss
4040
embedding_model: sentence-transformers/all-mpnet-base-v2
41-
embedding_dimension: 1024
41+
embedding_dimension: 768
4242
vector_db_id: vs_123 # Vector store ID (from index generation)
4343
db_path: /tmp/ocp.faiss
4444
score_multiplier: 1.0 # Weight for this vector store's results (Inline RAG only)
4545
- rag_id: knowledge-base # referenced in rag.inline / rag.tool
4646
rag_type: inline::faiss
4747
embedding_model: sentence-transformers/all-mpnet-base-v2
48-
embedding_dimension: 384
48+
embedding_dimension: 768
4949
vector_db_id: vs_456 # Vector store ID (from index generation)
5050
db_path: /tmp/kb.faiss
5151
score_multiplier: 1.2 # Weight for this vector store's results (Inline RAG only)

0 commit comments

Comments
 (0)