Skip to content

Commit 82ddcf0

Browse files
are-cesclaude
andcommitted
LCORE-1037: remove all llama-stack references from BYOK and RAG guides
Replace remaining Llama Stack mentions with generic terms — users should not need to know about the underlying Llama Stack layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 8435561 commit 82ddcf0

4 files changed

Lines changed: 102 additions & 104 deletions

File tree

docs/byok_guide.md

Lines changed: 52 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -148,10 +148,8 @@ class CustomMetadataProcessor(MetadataProcessor):
148148
```
149149

150150
**Important Notes:**
151-
- The vector database must be compatible with Llama Stack
152151
- Supported formats:
153-
- Llama-Stack Faiss Vector-IO
154-
- Llama-Stack SQLite-vec Vector-IO
152+
- Faiss Vector-IO
155153
- The same embedding model must be used for both creation and querying
156154

157155
### Step 3: Configure Embedding Model
@@ -178,15 +176,15 @@ The embedding model is specified per knowledge source in the `byok_rag` section
178176

179177
### Step 4: Configure BYOK Knowledge Sources
180178

181-
Declare your knowledge sources in the `byok_rag` section of your `lightspeed-stack.yaml`. The Lightspeed Stack service automatically generates the required Llama Stack configuration at startup.
179+
Declare your knowledge sources in the `byok_rag` section of your `lightspeed-stack.yaml`. The required configuration is automatically generated at startup when using `make run`, `make run-stack`, `docker-compose`, or library mode.
182180

183181
```yaml
184182
byok_rag:
185183
- rag_id: my-docs # Unique identifier for this knowledge source
186184
rag_type: inline::faiss # Vector store type (default: inline::faiss)
187185
embedding_model: sentence-transformers/all-mpnet-base-v2 # Embedding model (default)
188186
embedding_dimension: 768 # Must match your embedding model's output
189-
vector_db_id: your-index-id # Llama Stack vector store ID (from index generation)
187+
vector_db_id: vs_8c94967b-81cc-4028-a294-9cfac6fd9ae2 # Generated by rag-content during index creation
190188
db_path: /path/to/vector_db/faiss_store.db # Path to the vector database file
191189
score_multiplier: 1.0 # Weight for Inline RAG result ranking (default: 1.0)
192190
```
@@ -199,7 +197,7 @@ byok_rag:
199197
| `rag_type` | No | `inline::faiss` | Vector store provider type |
200198
| `embedding_model` | No | `sentence-transformers/all-mpnet-base-v2` | Embedding model identifier or path |
201199
| `embedding_dimension` | No | `768` | Embedding vector dimensionality |
202-
| `vector_db_id` | Yes | — | Vector store ID (must match the ID from index generation) |
200+
| `vector_db_id` | Yes | — | Vector store ID generated by rag-content (e.g. `vs_8c94967b-81cc-4028-a294-9cfac6fd9ae2`) |
203201
| `db_path` | Yes | — | Path to the vector database file |
204202
| `score_multiplier` | No | `1.0` | Weight for Inline RAG ranking (values > 1.0 boost; < 1.0 reduce) |
205203

@@ -213,20 +211,20 @@ byok_rag:
213211
rag_type: inline::faiss
214212
embedding_model: sentence-transformers/all-mpnet-base-v2
215213
embedding_dimension: 768
216-
vector_db_id: ocp-index
214+
vector_db_id: vs_3a7f9b2e-45dc-4e1a-b8f2-1c9d0e3f5a6b
217215
db_path: /data/vector_dbs/ocp_docs/faiss_store.db
218216
score_multiplier: 1.0
219217
220218
- rag_id: internal-kb
221219
rag_type: inline::faiss
222220
embedding_model: sentence-transformers/all-mpnet-base-v2
223221
embedding_dimension: 768
224-
vector_db_id: kb-index
222+
vector_db_id: vs_d4c8e1f0-92ab-4d3c-a5e7-6b8f0c2d1e3a
225223
db_path: /data/vector_dbs/internal_kb/faiss_store.db
226224
score_multiplier: 1.2 # Boost results from this store
227225
```
228226

229-
**⚠️ Important**: The `vector_db_id` value must exactly match the ID you provided when creating the vector database using the rag-content tool. This identifier links your configuration to the specific vector database index you created.
227+
**⚠️ Important**: The `vector_db_id` value must exactly match the ID generated by the rag-content tool during index creation (e.g. `vs_8c94967b-81cc-4028-a294-9cfac6fd9ae2`). This identifier links your configuration to the specific vector database index.
230228

231229
### Step 5: Configure RAG Strategy
232230

@@ -278,32 +276,35 @@ byok_rag:
278276
rag_type: inline::faiss
279277
embedding_model: sentence-transformers/all-mpnet-base-v2
280278
embedding_dimension: 768
281-
vector_db_id: your-index-id
279+
vector_db_id: vs_8c94967b-81cc-4028-a294-9cfac6fd9ae2
282280
db_path: /path/to/faiss_store.db
283281
```
284282

285283
### 2. pgvector (PostgreSQL)
286284
- **Type**: PostgreSQL with pgvector extension
287285
- **Best for**: Large-scale deployments, shared knowledge bases
288-
- **Configuration**: `rag_type: remote::pgvector`
286+
- **Configuration**: `remote::pgvector`
289287
- **Requirements**: PostgreSQL with pgvector extension
290288

289+
> [!NOTE]
290+
> pgvector is not yet supported via `byok_rag` in `lightspeed-stack.yaml` (see [LCORE-2437](https://redhat.atlassian.net/browse/LCORE-2437)).
291+
> It must be configured directly in the Llama Stack configuration file.
292+
291293
```yaml
292-
byok_rag:
293-
- rag_id: pgvector-knowledge
294-
rag_type: remote::pgvector
295-
embedding_model: sentence-transformers/all-mpnet-base-v2
296-
embedding_dimension: 768
297-
vector_db_id: enterprise-docs
298-
db_path: .llama/distributions/pgvector/registry.db
294+
vector_io:
295+
- provider_id: pgvector-knowledge
296+
provider_type: remote::pgvector
297+
config:
298+
host: localhost
299+
port: 5432
300+
db: knowledge_db
301+
user: lightspeed_user
302+
password: ${env.POSTGRES_PASSWORD}
303+
kvstore:
304+
type: sqlite
305+
db_path: .llama/distributions/pgvector/registry.db
299306
```
300307

301-
> [!NOTE]
302-
> For pgvector, `db_path` points to the local registry database used by Llama Stack to track the vector store metadata.
303-
> The PostgreSQL connection details (host, port, database, user, password) are configured
304-
> in the Llama Stack provider configuration. Use environment variables for credentials
305-
> (e.g., `POSTGRES_PASSWORD`).
306-
307308
**pgvector Table Schema:**
308309
- `id` (text): UUID identifier of the chunk
309310
- `document` (jsonb): JSON containing content and metadata
@@ -329,7 +330,7 @@ byok_rag:
329330
rag_type: inline::faiss
330331
embedding_model: sentence-transformers/all-mpnet-base-v2
331332
embedding_dimension: 768
332-
vector_db_id: company-knowledge-index
333+
vector_db_id: vs_f1a2b3c4-56de-4f78-90ab-cdef12345678
333334
db_path: /home/user/vector_dbs/company_docs/faiss_store.db
334335
335336
rag:
@@ -340,12 +341,18 @@ rag:
340341
```
341342

342343
> [!NOTE]
343-
> Your LLM inference provider (e.g., OpenAI, vLLM) must also be configured.
344+
> Your LLM inference provider (e.g., OpenAI, vLLM) must also be configured in your `run.yaml`.
344345
> For OpenAI, set the `OPENAI_API_KEY` environment variable.
345346

346347
### Example 2: Multiple Knowledge Sources with pgvector
347348

348-
A configuration combining a local FAISS store with a remote pgvector store:
349+
A configuration combining a local FAISS store (via `byok_rag`) with a remote pgvector store (configured directly in the Llama Stack configuration file):
350+
351+
> [!NOTE]
352+
> pgvector is not yet supported via `byok_rag` in `lightspeed-stack.yaml` (see [LCORE-2437](https://redhat.atlassian.net/browse/LCORE-2437)).
353+
> The pgvector provider must be configured directly in the Llama Stack configuration file.
354+
355+
**`lightspeed-stack.yaml`** — FAISS store and RAG strategy:
349356

350357
```yaml
351358
name: Lightspeed Core Service (LCS)
@@ -359,25 +366,32 @@ byok_rag:
359366
rag_type: inline::faiss
360367
embedding_model: sentence-transformers/all-mpnet-base-v2
361368
embedding_dimension: 768
362-
vector_db_id: local-index
369+
vector_db_id: vs_e9d8c7b6-43af-4b2d-8e1f-0a9b8c7d6e5f
363370
db_path: /data/vector_dbs/local/faiss_store.db
364371
score_multiplier: 1.0
365372
366-
- rag_id: enterprise-kb
367-
rag_type: remote::pgvector
368-
embedding_model: sentence-transformers/all-mpnet-base-v2
369-
embedding_dimension: 768
370-
vector_db_id: enterprise-docs
371-
db_path: .llama/distributions/pgvector/registry.db
372-
score_multiplier: 1.2
373-
374373
rag:
375374
inline:
376375
- local-docs
377-
- enterprise-kb
378376
tool:
379377
- local-docs
380-
- enterprise-kb
378+
```
379+
380+
**Llama Stack configuration file** — pgvector provider:
381+
382+
```yaml
383+
vector_io:
384+
- provider_id: enterprise-kb
385+
provider_type: remote::pgvector
386+
config:
387+
host: localhost
388+
port: 5432
389+
db: knowledge_db
390+
user: lightspeed_user
391+
password: ${env.POSTGRES_PASSWORD}
392+
kvstore:
393+
type: sqlite
394+
db_path: .llama/distributions/pgvector/registry.db
381395
```
382396

383397
> [!NOTE]
@@ -396,7 +410,6 @@ The BYOK (Bring Your Own Knowledge) feature in Lightspeed Core provides powerful
396410

397411
For additional support and advanced configurations, refer to:
398412
- [RAG Configuration Guide](rag_guide.md)
399-
- [Llama Stack Documentation](https://llama-stack.readthedocs.io/)
400413
- [rag-content Tool Repository](https://github.com/lightspeed-core/rag-content)
401414

402415
Remember to regularly update your knowledge sources and monitor system performance to maintain optimal BYOK functionality.

0 commit comments

Comments
 (0)