Skip to content

Commit df3f80a

Browse files
are-cesare-ces
committed
Update config
1 parent b9be0e0 commit df3f80a

19 files changed

Lines changed: 589 additions & 668 deletions

‎docs/byok_guide.md‎

Lines changed: 29 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,9 @@ BYOK (Bring Your Own Knowledge) is Lightspeed Core's implementation of Retrieval
3636

3737
BYOK knowledge sources can be queried in two complementary modes, configured independently:
3838

39-
### Inline RAG (pre-query injection)
39+
### Inline RAG
4040

41-
Context is fetched from your BYOK vector stores and/or OKP **before** the LLM generates a response, and injected into every query automatically. No tool calls are required.
41+
Context is fetched from your BYOK vector stores and/or OKP and injected before the LLM request. No tool calls are required.
4242

4343
```mermaid
4444
graph TD
@@ -54,7 +54,7 @@ graph TD
5454

5555
### Tool RAG (on-demand retrieval)
5656

57-
The LLM can call the `file_search` tool during generation when it decides external knowledge is needed. Only BYOK vector stores are supported in Tool RAG mode.
57+
The LLM can call the `file_search` tool during generation when it decides external knowledge is needed. Both BYOK vector stores and OKP are supported in Tool RAG mode.
5858

5959
```mermaid
6060
graph TD
@@ -67,7 +67,7 @@ graph TD
6767
B --> C{Need External Knowledge?}
6868
C -->|Yes| D[file_search Tool]
6969
C -->|No| E[Generate Response]
70-
D --> F[BYOK Vector Stores]
70+
D --> F[BYOK / OKP Vector Stores]
7171
F --> G[Retrieve Relevant Context]
7272
G --> B
7373
E --> H[Response to User]
@@ -78,7 +78,13 @@ Both modes rely on:
7878
- **Embedding Model**: Converts queries and documents into vector representations for similarity matching
7979

8080
Inline RAG additionally supports:
81-
- **Score Multiplier**: Optional weight applied per BYOK vector store when mixing multiple sources. Allows custom prioritization of content.
81+
- **Score Multiplier**: Optional weight applied per BYOK vector store when mixing multiple sources. Allows custom prioritization of content.
82+
83+
> [!NOTE]
84+
> OKP and BYOK scores are not directly comparable (different scoring systems), so
85+
> `score_multiplier` does not apply to OKP results. To control the amount of retrieved
86+
> context, set the `BYOK_RAG_MAX_CHUNKS` and `OKP_RAG_MAX_CHUNKS` constants in `src/constants.py`
87+
> (defaults: 10 and 5 respectively). For Tool RAG, use `TOOL_RAG_MAX_CHUNKS` (default: 10).
8288
8389
---
8490

@@ -290,29 +296,37 @@ registered_resources:
290296

291297
### Step 5: Configure RAG Strategy
292298

293-
Add a `rag` section to your `lightspeed-stack.yaml` to choose how BYOK knowledge is used:
299+
Add a `rag` section to your `lightspeed-stack.yaml` to choose how BYOK knowledge is used.
300+
Each list entry is a `rag_id` from `byok_rag`, or the special value `okp-rag` for OKP.
294301

295302
```yaml
296303
rag:
297-
# Inline RAG: inject context before every LLM response (no tool calls needed)
304+
# Inline RAG: inject context before the LLM request (no tool calls needed)
298305
inline:
299-
byok:
300-
enabled: true # fetch and inject BYOK vector store context pre-query
301-
okp:
302-
enabled: true # fetch and inject OKP context pre-query
306+
- my-docs # rag_id from byok_rag
307+
- okp-rag # include OKP context inline
303308
304309
# Tool RAG: the LLM can call file_search to retrieve context on demand
310+
# Omit to use all registered BYOK stores (backward compatibility)
305311
tool:
306-
byok:
307-
enabled: true # expose BYOK vector stores as the file_search tool
312+
- my-docs # expose this BYOK store as the file_search tool
313+
- okp-rag # expose OKP as the file_search tool
314+
315+
# OKP provider settings (only relevant when okp-rag is listed above)
316+
okp:
317+
offline: true # true = use parent_id for source URLs, false = use reference_url
308318
```
309319

310320
Both modes can be enabled simultaneously. Choose based on your latency and control preferences:
311321

312322
| Mode | When context is fetched | Tool call needed | Supported sources | score_multiplier |
313323
|------|------------------------|------------------|-------------------|-----------------|
314-
| Inline RAG | Before every query | No | BYOK + OKP | Yes (BYOK only) |
315-
| Tool RAG | On LLM demand | Yes | BYOK only | No |
324+
| Inline RAG | With every query | No | BYOK + OKP | Yes (BYOK only) |
325+
| Tool RAG | On LLM demand | Yes | BYOK + OKP | No |
326+
327+
> [!TIP]
328+
> A ready-to-use example combining BYOK and OKP is available at
329+
> [`examples/lightspeed-stack-byok-okp-rag.yaml`](../examples/lightspeed-stack-byok-okp-rag.yaml).
316330

317331
---
318332

‎docs/config.md‎

Lines changed: 47 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,22 @@ Microsoft Entra ID authentication attributes for Azure.
110110

111111
BYOK (Bring Your Own Knowledge) RAG configuration.
112112

113+
Each entry registers a local vector store with the service. The `rag_id` is the
114+
identifier used in `rag.inline` and `rag.tool` to select which stores to use.
115+
116+
Example:
117+
118+
```yaml
119+
byok_rag:
120+
- rag_id: my-docs # referenced in rag.inline / rag.tool
121+
rag_type: inline::faiss
122+
embedding_model: sentence-transformers/all-MiniLM-L6-v2
123+
embedding_dimension: 384
124+
vector_db_id: vs_abc123
125+
db_path: /path/to/faiss_store.db
126+
score_multiplier: 1.0
127+
```
128+
113129
114130
| Field | Type | Description |
115131
|-------|------|-------------|
@@ -525,68 +541,55 @@ the service can handle requests concurrently.
525541

526542
Top-level RAG strategy configuration. Controls two complementary retrieval modes:
527543

528-
- **Inline RAG**: context is fetched from OKP and/or BYOK vector stores and injected
529-
into every query before the LLM responds.
544+
- **Inline RAG**: context is fetched from the listed sources and injected before the
545+
LLM request.
530546
- **Tool RAG**: the LLM can call the `file_search` tool during generation to retrieve
531-
context on demand from BYOK vector stores.
532-
533-
534-
| Field | Type | Description |
535-
|-------|------|-------------|
536-
| inline | | Pre-query RAG from OKP and BYOK. See InlineRagConfiguration. |
537-
| tool | | Tool-based RAG that the LLM can invoke. See ToolRagConfiguration. |
538-
539-
540-
## InlineRagConfiguration
541-
542-
543-
Pre-query RAG configuration that injects context before the LLM generates a response.
544-
545-
Both OKP and BYOK sources can be enabled independently. When enabled, retrieved
546-
chunks are added as context on every query.
547+
context on demand from the listed vector stores. Supports both BYOK and OKP.
547548

549+
Each strategy is configured as a list of RAG IDs referencing entries in `byok_rag`.
550+
The special ID `okp-rag` activates the OKP provider (no `byok_rag` entry needed).
548551

549-
| Field | Type | Description |
550-
|-------|------|-------------|
551-
| okp | | OKP RAG configuration for pre-query context injection. |
552-
| byok | | BYOK RAG configuration for pre-query context injection. |
553-
554-
555-
## OkpRagConfiguration
556-
557-
558-
OKP configuration for Inline RAG (pre-query context injection).
559-
560-
Controls whether to use offline or online mode when building document URLs
561-
from vector search results, and enables/disables OKP vector IO functionality.
562-
563-
564-
| Field | Type | Description |
565-
|-------|------|-------------|
566-
| enabled | boolean | When True, enables OKP vector IO functionality for vector search queries. When False, disables OKP vector search processing. |
567-
| offline | boolean | When True, use parent_id for chunk source URLs. When False, use reference_url for chunk source URLs. |
568-
552+
**Backward compatibility**: omitting `tool` uses all registered BYOK vector stores
553+
(equivalent to the old `tool.byok.enabled = True`). Omitting `inline` means no
554+
context is injected before the LLM request.
569555

570-
## ByokRagConfiguration
556+
Example:
571557

558+
```yaml
559+
rag:
560+
inline:
561+
- my-docs # inject context from my-docs before the LLM request
562+
tool:
563+
- okp-rag # LLM can search OKP as a tool
564+
- my-docs # LLM can also search my-docs as a tool
572565
573-
Configuration to enable or disable BYOK RAG retrieval.
566+
okp:
567+
offline: true # use parent_id for OKP URL construction
568+
```
574569

575570

576571
| Field | Type | Description |
577572
|-------|------|-------------|
578-
| enabled | boolean | When True, queries BYOK vector stores for RAG context. Default: False. |
573+
| inline | list[string] | RAG IDs whose content is injected before the LLM request. Use `okp-rag` for OKP. Empty by default (no inline RAG). |
574+
| tool | list[string] or null | RAG IDs exposed as a `file_search` tool the LLM can invoke. Use `okp-rag` to include OKP. When omitted, all registered BYOK vector stores are used (backward compatibility). |
579575

580576

581-
## ToolRagConfiguration
577+
## OkpConfiguration
582578

579+
OKP (Offline Knowledge Portal) provider settings. Only used when `okp-rag` is listed in `rag.inline` or `rag.tool`.
583580

584-
Configuration for exposing RAG as a tool the LLM can call during generation.
581+
Example:
585582

583+
```yaml
584+
okp:
585+
offline: true # use parent_id for OKP URL construction
586+
chunk_filter_query: "is_chunk:true"
587+
```
586588

587589
| Field | Type | Description |
588590
|-------|------|-------------|
589-
| byok | | BYOK RAG configuration for tool-based retrieval. Default: enabled. |
591+
| offline | boolean | When `true` (default), use `parent_id` for OKP chunk source URLs. When `false`, use `reference_url`. |
592+
| chunk_filter_query | string | OKP filter query (`fq`) applied to every OKP search request. Defaults to `"is_chunk:true"`. Extend with `AND` for extra constraints. |
590593

591594

592595
## SplunkConfiguration

‎docs/openapi.json‎

Lines changed: 38 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -4447,7 +4447,7 @@
44474447
],
44484448
"summary": "Handle A2A Jsonrpc",
44494449
"description": "Handle A2A JSON-RPC requests following the A2A protocol specification.\n\nThis endpoint uses the DefaultRequestHandler from the A2A SDK to handle\nall JSON-RPC requests including message/send, message/stream, etc.\n\nThe A2A SDK application is created per-request to include authentication\ncontext while still leveraging FastAPI's authorization middleware.\n\nAutomatically detects streaming requests (message/stream JSON-RPC method)\nand returns a StreamingResponse to enable real-time chunk delivery.\n\nArgs:\n request: FastAPI request object\n auth: Authentication tuple\n mcp_headers: MCP headers for context propagation\n\nReturns:\n JSON-RPC response or streaming response",
4450-
"operationId": "handle_a2a_jsonrpc_a2a_post",
4450+
"operationId": "handle_a2a_jsonrpc_a2a_get",
44514451
"responses": {
44524452
"200": {
44534453
"description": "Successful Response",
@@ -4465,7 +4465,7 @@
44654465
],
44664466
"summary": "Handle A2A Jsonrpc",
44674467
"description": "Handle A2A JSON-RPC requests following the A2A protocol specification.\n\nThis endpoint uses the DefaultRequestHandler from the A2A SDK to handle\nall JSON-RPC requests including message/send, message/stream, etc.\n\nThe A2A SDK application is created per-request to include authentication\ncontext while still leveraging FastAPI's authorization middleware.\n\nAutomatically detects streaming requests (message/stream JSON-RPC method)\nand returns a StreamingResponse to enable real-time chunk delivery.\n\nArgs:\n request: FastAPI request object\n auth: Authentication tuple\n mcp_headers: MCP headers for context propagation\n\nReturns:\n JSON-RPC response or streaming response",
4468-
"operationId": "handle_a2a_jsonrpc_a2a_post",
4468+
"operationId": "handle_a2a_jsonrpc_a2a_get",
44694469
"responses": {
44704470
"200": {
44714471
"description": "Successful Response",
@@ -5522,20 +5522,6 @@
55225522
"title": "ByokRag",
55235523
"description": "BYOK (Bring Your Own Knowledge) RAG configuration."
55245524
},
5525-
"ByokRagConfiguration": {
5526-
"properties": {
5527-
"enabled": {
5528-
"type": "boolean",
5529-
"title": "BYOK RAG enabled",
5530-
"description": "When True, queries BYOK vector stores for RAG context.",
5531-
"default": false
5532-
}
5533-
},
5534-
"additionalProperties": false,
5535-
"type": "object",
5536-
"title": "ByokRagConfiguration",
5537-
"description": "BYOK RAG configuration."
5538-
},
55395525
"CORSConfiguration": {
55405526
"properties": {
55415527
"allow_origins": {
@@ -5739,6 +5725,11 @@
57395725
"$ref": "#/components/schemas/RagConfiguration",
57405726
"title": "RAG configuration",
57415727
"description": "Configuration for all RAG strategies (inline and tool-based)."
5728+
},
5729+
"okp": {
5730+
"$ref": "#/components/schemas/OkpConfiguration",
5731+
"title": "OKP configuration",
5732+
"description": "OKP provider settings. Only used when 'okp-rag' is listed in rag.inline or rag.tool."
57425733
}
57435734
},
57445735
"additionalProperties": false,
@@ -6980,24 +6971,6 @@
69806971
}
69816972
]
69826973
},
6983-
"InlineRagConfiguration": {
6984-
"properties": {
6985-
"okp": {
6986-
"$ref": "#/components/schemas/OkpRagConfiguration",
6987-
"title": "OKP RAG configuration",
6988-
"description": "Configuration for OKP RAG (inline)."
6989-
},
6990-
"byok": {
6991-
"$ref": "#/components/schemas/ByokRagConfiguration",
6992-
"title": "BYOK RAG configuration",
6993-
"description": "Configuration for BYOK RAG (inline)."
6994-
}
6995-
},
6996-
"additionalProperties": false,
6997-
"type": "object",
6998-
"title": "InlineRagConfiguration",
6999-
"description": "Inline RAG configuration.\n\nControls inline RAG from OKP and BYOK vector stores."
7000-
},
70016974
"InternalServerErrorResponse": {
70026975
"properties": {
70036976
"status_code": {
@@ -7607,25 +7580,25 @@
76077580
"title": "OAuthFlows",
76087581
"description": "Defines the configuration for the supported OAuth 2.0 flows."
76097582
},
7610-
"OkpRagConfiguration": {
7583+
"OkpConfiguration": {
76117584
"properties": {
7612-
"enabled": {
7613-
"type": "boolean",
7614-
"title": "OKP RAG enabled",
7615-
"description": "When True, queries OKP for RAG context.",
7616-
"default": false
7617-
},
76187585
"offline": {
76197586
"type": "boolean",
7620-
"title": "Offline mode",
7621-
"description": "When True, use parent_id for chunk source URLs. When False, use reference_url for chunk source URLs.",
7587+
"title": "OKP offline mode",
7588+
"description": "When True, use parent_id for OKP chunk source URLs. When False, use reference_url for chunk source URLs.",
76227589
"default": true
7590+
},
7591+
"chunk_filter_query": {
7592+
"type": "string",
7593+
"title": "OKP chunk filter query",
7594+
"description": "OKP filter query applied to every OKP search request. Defaults to 'is_chunk:true' to restrict results to chunk documents. To add extra constraints, extend the expression using boolean syntax, e.g. 'is_chunk:true AND product:*openshift*'.",
7595+
"default": "is_chunk:true"
76237596
}
76247597
},
76257598
"additionalProperties": false,
76267599
"type": "object",
7627-
"title": "OkpRagConfiguration",
7628-
"description": "OKP RAG configuration."
7600+
"title": "OkpConfiguration",
7601+
"description": "OKP (Offline Knowledge Portal) provider configuration.\n\nControls provider-specific behaviour for the OKP vector store.\nOnly relevant when ``\"okp-rag\"`` is listed in ``rag.inline`` or ``rag.tool``."
76297602
},
76307603
"OpenIdConnectSecurityScheme": {
76317604
"properties": {
@@ -8804,20 +8777,33 @@
88048777
"RagConfiguration": {
88058778
"properties": {
88068779
"inline": {
8807-
"$ref": "#/components/schemas/InlineRagConfiguration",
8808-
"title": "Inline RAG configuration",
8809-
"description": "Configuration for inline RAG from OKP and BYOK vector stores."
8780+
"items": {
8781+
"type": "string"
8782+
},
8783+
"type": "array",
8784+
"title": "Inline RAG IDs",
8785+
"description": "RAG IDs whose sources are injected as context before the LLM call. Use 'okp-rag' to enable OKP inline RAG. Empty by default (no inline RAG)."
88108786
},
88118787
"tool": {
8812-
"$ref": "#/components/schemas/ToolRagConfiguration",
8813-
"title": "Tool RAG configuration",
8814-
"description": "Configuration for exposing RAG as a tool that the LLM can call."
8788+
"anyOf": [
8789+
{
8790+
"items": {
8791+
"type": "string"
8792+
},
8793+
"type": "array"
8794+
},
8795+
{
8796+
"type": "null"
8797+
}
8798+
],
8799+
"title": "Tool RAG IDs",
8800+
"description": "RAG IDs made available to the LLM as a file_search tool. Use 'okp-rag' to include the OKP vector store. When omitted, all registered BYOK vector stores are used (backward compatibility)."
88158801
}
88168802
},
88178803
"additionalProperties": false,
88188804
"type": "object",
88198805
"title": "RagConfiguration",
8820-
"description": "RAG strategy configuration.\n\nControls different RAG strategies: inline and tool-based."
8806+
"description": "RAG strategy configuration.\n\nControls which RAG sources are used for inline and tool-based retrieval.\n\nEach strategy lists RAG IDs to include. The special ID ``\"okp-rag\"`` defined in constants,\nactivates the OKP provider; all other IDs refer to entries in ``byok_rag``.\n\nBackward compatibility:\n - ``inline`` defaults to ``[]`` (no inline RAG).\n - ``tool`` defaults to ``None`` which means all registered vector stores\n are used (identical to the previous ``tool.byok.enabled = True`` default)."
88218807
},
88228808
"ReadinessResponse": {
88238809
"properties": {
@@ -9581,19 +9567,6 @@
95819567
"title": "ToolCallSummary",
95829568
"description": "Model representing a tool call made during response generation (for tool_calls list)."
95839569
},
9584-
"ToolRagConfiguration": {
9585-
"properties": {
9586-
"byok": {
9587-
"$ref": "#/components/schemas/ByokRagConfiguration",
9588-
"title": "BYOK RAG configuration",
9589-
"description": "Configuration for BYOK RAG as a tool."
9590-
}
9591-
},
9592-
"additionalProperties": false,
9593-
"type": "object",
9594-
"title": "ToolRagConfiguration",
9595-
"description": "Tool RAG configuration.\n\nControls whether RAG functionality is exposed as a tool that the LLM can call."
9596-
},
95979570
"ToolResultSummary": {
95989571
"properties": {
95999572
"id": {

0 commit comments

Comments
 (0)