Refactored shield-related documentation

Author: asimurka

README.md

@@ -769,22 +769,29 @@ For the configuration guide, skill authoring instructions, and examples, see the
 
 ## Safety Shields
 
-A single Llama Stack configuration file can include multiple safety shields, which are utilized in agent
-configurations to monitor input and/or output streams. LCS uses the following naming convention to specify how each safety shield is
-utilized:
+Safety shields used by `/query`, `/streaming_query`, `/responses`, and `/rlsapi`
+are **owned by Lightspeed Core Stack** and configured in `lightspeed-stack.yaml`
+(not via the Llama Stack / OGX Safety or Moderations APIs).
 
-1. If the `shield_id` starts with `input_`, it will be used for input only.
-1. If the `shield_id` starts with `output_`, it will be used for output only.
-1. If the `shield_id` starts with `inout_`, it will be used both for input and output.
-1. Otherwise, it will be used for input only.
+Supported shield types (`provider_id`):
 
-Additionally, an optional list parameter `shield_ids` can be specified in `/query` and `/streaming_query` endpoints to override which shields are applied. You can use this config to disable shield overrides:
+- `question_validity` — topic / off-topic classification
+- `redaction` — regex-based PII redaction
+
+List configured shields with `GET /v1/shields`. Optionally override which
+shields apply with the `shield_ids` request field (`null` = all, `[]` = none,
+or a list of configured `identifier` values). To forbid client overrides on
+`/query` and `/streaming_query`:
 
 ```yaml
 customization:
   disable_shield_ids_override: true
 ```
 
+For configuration details, endpoint application (direct-run vs agent
+capabilities), and examples, see the
+[Safety Shields Guide](docs/user_doc/shields_guide.md).
+
 ## Authentication
 
 See [authentication and authorization](docs/auth.md).

docs/README.md

@@ -19,6 +19,8 @@ See the full documentation at [`../README.md`](../README.md) or browse sub-pages
 
 [Agent skills](https://lightspeed-core.github.io/lightspeed-stack/user_doc/skills_guide.html)
 
+[Safety shields](https://lightspeed-core.github.io/lightspeed-stack/user_doc/shields_guide.html)
+
 [A2A [Agent-to-Agent] Protocol](https://lightspeed-core.github.io/lightspeed-stack/user_doc/a2a_protocol.html)
 
 [RAG configuration guide](https://lightspeed-core.github.io/lightspeed-stack/user_doc/rag_guide.html)

docs/basic_info/overview.md

@@ -6,9 +6,11 @@
 
 **Lightspeed Core Stack (LCore)** is an enterprise-grade middleware service that provides a robust layer between client applications and AI Large Language Model (LLM) backends. It adds essential enterprise features such as authentication, authorization, quota management, caching, and observability to LLM interactions.
 
-Current version of LCore is built on **Llama Stack** - open-source framework that provides standardized APIs for building LLM applications. Llama Stack offers a unified interface for models, RAG (vector stores), tools, and safety (shields) across different providers. LCore communicates with Llama Stack to orchestrate all LLM operations.
+Current version of LCore is built on **OGX (Llama Stack)** - open-source framework that provides standardized APIs for building LLM applications. OGX offers a unified interface for models, RAG (vector stores), and tools across different providers. LCore communicates with OGX to orchestrate all LLM operations.
 
-To enhance LLM responses, LCore leverages **RAG (Retrieval-Augmented Generation)**, which retrieves relevant context from vector databases before generating answers. Llama Stack manages the vector stores, and LCore queries them to inject relevant documentation, knowledge bases, or previous conversations into the LLM prompt.
+To enhance LLM responses, LCore leverages **RAG (Retrieval-Augmented Generation)**, which retrieves relevant context from vector databases before generating answers. OGX manages the vector stores, and LCore queries them to inject relevant documentation, knowledge bases, or previous conversations into the LLM prompt.
+
+LCore also provides **safety shields** such as topic validation and PII redaction. These are configured in LCore and applied on request endpoints before or during agent processing.
 
 ### Key Features
 

docs/devel_doc/ARCHITECTURE.md

@@ -24,10 +24,12 @@
 
 **Lightspeed Core Stack (LCORE)** is an enterprise-grade middleware service that provides a robust layer between client applications and AI Large Language Model (LLM) backends. It adds essential enterprise features such as authentication, authorization, quota management, caching, and observability to LLM interactions.
 
-LCore is built on **Llama Stack** - Meta's open-source framework that provides standardized APIs for building LLM applications. Llama Stack offers a unified interface for models, RAG (vector stores), tools, and safety (shields) across different providers. LCore communicates with Llama Stack to orchestrate all LLM operations.
+LCore is built on **Llama Stack / OGX** - an open-source framework that provides standardized APIs for building LLM applications. It offers a unified interface for models, RAG (vector stores), and tools across different providers. LCore communicates with the stack to orchestrate all LLM operations.
 
 To enhance LLM responses, LCore leverages **RAG (Retrieval-Augmented Generation)**, which retrieves relevant context from vector databases before generating answers. Llama Stack manages the vector stores, and LCore queries them to inject relevant documentation, knowledge bases, or previous conversations into the LLM prompt.
 
+To keep requests on-topic and protect sensitive data, LCore applies **safety shields**, which validate user questions and redact PII from model traffic. Shields are owned by LCore and configured in the service configuration.
+
 ### 1.2 Key Features
 
 - **Multi-Provider Support**: Works with multiple LLM providers (Ollama, OpenAI, Watsonx, etc.)
@@ -61,6 +63,7 @@ To enhance LLM responses, LCore leverages **RAG (Retrieval-Augmented Generation)
 │  ┌───────────────────────────────────────────────────┐  │
 │  │          Request Processing                       │  │
 │  │  • LLM Orchestration (via Llama Stack)            │  │
+│  │  • Safety Shields                                 │  │
 │  │  • Tool Integration (MCP servers)                 │  │
 │  │  • RAG & Context Management                       │  │
 │  └───────────────────────────────────────────────────┘  │
@@ -79,8 +82,8 @@ To enhance LLM responses, LCore leverages **RAG (Retrieval-Augmented Generation)
           │                  │
           │  • Models & LLMs │
           │  • RAG Stores    │
-          │  • Shields       │
-          └────────┬─────────┘
+          └──────────────────┘
+                   │
                    │ (manages & invokes)
                    ▼
           ┌──────────────────┐
@@ -229,7 +232,6 @@ The system defines 30+ actions that can be authorized. Examples (see `docs/auth.
 - **Models**: List available LLM models
 - **Responses**: Generate LLM responses (OpenAI-compatible)
 - **Conversations**: Manage conversation history
-- **Shields**: List and apply guardrails (content filtering, safety checks)
 - **Vector Stores**: Access RAG databases for context injection
 - **Toolgroups**: Register MCP servers as tools
 
@@ -394,11 +396,12 @@ Here's how a real query flows through the system:
 4. **Quota Check** - User has 50,000 tokens available ✅
 5. **Model Selection** - Use configured default model (e.g., `meta-llama/Llama-3.1-8B-Instruct`)
 6. **Context Building** - Retrieve conversation history, query RAG vector stores for relevant docs, determine available MCP tools
-7. **Llama Stack Call** - Send complete request with system prompt, RAG context, MCP tools, and shields
-8. **LLM Processing** - Llama Stack generates response, may invoke MCP tools, returns token counts
-9. **Post-Processing** - Apply shields, generate conversation summary if new
-10. **Store Results** - Save to Cache DB, User DB, consume quota, update metrics
-11. **Return Response** - Complete LLM response with referenced documents, token usage, and remaining quota
+7. **Shield moderation** - LCore-owned direct-run moderation (and agent capabilities where applicable) using shields configured in LCORE config
+8. **Llama Stack / agent call** - Send request with system prompt, RAG context, and MCP tools
+9. **LLM Processing** - Stack / agent generates response, may invoke MCP tools, returns token counts
+10. **Post-Processing** - Generate conversation summary if new
+11. **Store Results** - Save to Cache DB, User DB, consume quota, update metrics
+12. **Return Response** - Complete LLM response with referenced documents, token usage, and remaining quota
 
 **Key Takeaways:**
 - RAG enhances responses with relevant documentation
@@ -497,8 +500,8 @@ This section documents the REST API endpoints exposed by LCore for client intera
 - `GET /v1/mcp-servers` - List all registered MCP servers (static and dynamic)
 - `DELETE /v1/mcp-servers/{name}` - Unregister a dynamically registered MCP server
 
-**List Shields:** `GET /shields`
-- Returns available guardrails
+**List Shields:** `GET /v1/shields`
+- Returns list of shields configured in LCORE
 
 **List RAG Databases:** `GET /rags`
 - Returns configured vector stores

docs/devel_doc/providers.md

@@ -198,14 +198,12 @@ make run CONFIG=examples/lightspeed-stack-azure-entraid-service.yaml
 
 ## Safety Providers
 
-| Name         | Type   | Pip Dependencies                                                                     | Supported in LCS |
-|--------------|--------|--------------------------------------------------------------------------------------|:----------------:|
-| code-scanner | inline | `codeshield`                                                                         | ❌               |
-| llama-guard  | inline | —                                                                                    | ❌               |
-| prompt-guard | inline | `transformers[accelerate]`, `torch --index-url https://download.pytorch.org/whl/cpu` | ❌               |
-| bedrock      | remote | `boto3`                                                                              | ❌               |
-| nvidia       | remote | `requests`                                                                           | ❌               |
-| sambanova    | remote | `litellm`, `requests`                                                                | ❌               |
+Shields are owned by LCORE (configured under `shields:` block), not as OGX `providers.safety` entries.
+
+| Name               | Type  | Pip Dependencies | Supported in LCS |
+|--------------------|-------|------------------|:----------------:|
+| question_validity  | lcore | —                | ✅               |
+| redaction          | lcore | —                | ✅               |
 
 ---
 
@@ -342,7 +340,6 @@ make run CONFIG=examples/lightspeed-stack-azure-entraid-service.yaml
     Some of APIs are associated with a set of **Resources**. Here is the mapping of APIs to resources:
 
     - **Inference**, **Eval** and **Post Training** are associated with **Model** resources.
-    - **Safety** is associated with **Shield** resources.
     - **Tool Runtime** is associated with **ToolGroup** resources.
     - **DatasetIO** is associated with **Dataset** resources.
     - **VectorIO** is associated with **VectorDB** resources.
@@ -359,9 +356,6 @@ make run CONFIG=examples/lightspeed-stack-azure-entraid-service.yaml
         provider_id: openai
         model_type: llm
         provider_model_id: gpt-4-turbo  # provider label
-    
-    shields:
-        ...
     ```
     **Note** It is necessary for llama-stack to know which resources to use for a given provider. This means you need to explicitly register resources (including models) before you can use them with the associated APIs.
 

docs/devel_doc/responses.md

@@ -107,7 +107,7 @@ The following fields are LCORE-specific request extensions and are not part of t
 | Field | Type | Description | Required |
 |-------|------|-------------|----------|
 | `generate_topic_summary` | boolean | Generate topic summary for new conversations. Default: true | No |
-| `shield_ids` | array[string] | Shield IDs to apply. If omitted, all configured shields in LCORE are used | No |
+| `shield_ids` | array[string] | LCORE-configured shield `name` values to apply. If omitted, all configured shields are used. Not Llama Stack Safety resource names. | No |
 | `solr` | object | Optional `mode` and `filters`. Legacy top-level filter-only objects are still accepted. | No |
 
 
@@ -600,7 +600,7 @@ In streaming mode, server-deployed MCP events (e.g. `mcp_call`, `mcp_list_tools`
 The API introduces extensions that are not part of the OpenResponses specification:
 
 - `generate_topic_summary` (request) — When set to `true` and a new conversation is created, a topic summary is automatically generated and stored in conversation metadata.
-- `shield_ids` (request) — Optional list of safety shield IDs to apply. If omitted, all configured shields are used.
+- `shield_ids` (request) — Optional list of LCORE shield `name` values to apply. If omitted, all shields from `lightspeed-stack.yaml` are used. See the [Safety Shields Guide](../user_doc/shields_guide.md).
 - `solr` (request) — Object with optional `mode` (`semantic`, `hybrid`, or `lexical`) and `filters` (Solr vector_io provider payload). Legacy filter-only objects (no `mode`/`filters` wrapper) still work.
 - `available_quotas` (response) — Provides real-time quota information from all configured quota limiters.
 

docs/models/requests.md

@@ -832,7 +832,7 @@ Attributes:
     generate_topic_summary: Whether to generate topic summary for new conversations.
     media_type: The optional media type for response format (application/json or text/plain).
     vector_store_ids: The optional list of specific vector store IDs to query for RAG.
-    shield_ids: The optional list of safety shield IDs to apply.
+    shield_ids: The optional list of configured shield names to apply.
     solr: Optional Solr inline RAG options (mode, filters) or legacy filter-only dict.
 
 
@@ -848,7 +848,7 @@ Attributes:
 | generate_topic_summary | boolean | Whether to generate topic summary for new conversations |
 | media_type | string | Media type for the response format |
 | vector_store_ids | array | Optional list of specific vector store IDs to query for RAG. If not provided, all available vector stores will be queried. |
-| shield_ids | array | Optional list of safety shield IDs to apply. If None, all configured shields are used.  |
+| shield_ids | array | Optional list of configured shield names to apply. If None, all configured shields are used. |
 | solr |  | Solr inline RAG config: mode (semantic, hybrid, lexical) and filters; a legacy filter-only object (e.g. fq) is still accepted. |
 
 
@@ -900,8 +900,8 @@ Attributes:
         calls, MCP tools). Defaults to all tools available to the model.
     generate_topic_summary: LCORE-specific flag indicating whether to generate a
         topic summary for new conversations. Defaults to True.
-    shield_ids: LCORE-specific list of safety shield IDs to apply. If None, all
-        configured shields are used.
+    shield_ids: LCORE-specific list of configured shield names to apply.
+            If None, all configured shields are used.
     solr: Optional Solr inline RAG options (mode, filters) or legacy filter-only dict.
 
 

docs/testing/e2e_scenarios.md

@@ -102,8 +102,7 @@
 * Check if the OpenAPI endpoint works as expected
 * Check if info endpoint is working
 * Check if info endpoint reports error when llama-stack connection is not working
-* Check if shields endpoint is working
-* Check if shields endpoint reports error when llama-stack is unreachable
+* Check if shields endpoint is working (lists LCORE-configured shields)
 * Check if tools endpoint is working
 * Check if tools endpoint reports error when llama-stack is unreachable
 * Check if metrics endpoint is working

docs/testing/e2e_testing.md

@@ -24,7 +24,7 @@ This guide describes how to run, extend, and understand the Lightspeed Core Stac
 
 - **Framework**: [Behave](https://behave.readthedocs.io/) (Python BDD).
 - **Scope**: REST API of the Lightspeed Core Stack (query, streaming_query, models, info, health, feedback, conversations, RBAC, MCP, etc.).
-- **Execution**: Tests run in a **separate process** from the app. They send HTTP requests to the service and (in server mode) optionally talk to the Llama Stack service for shield setup.
+- **Execution**: Tests run in a **separate process** from the app. They send HTTP requests to the service. LCORE shields are configured in `lightspeed-stack.yaml` (not via Llama Stack Safety APIs).
 - **Environments**: Local (Docker Compose) or Prow/OpenShift (containers/pods). Mode is detected via `E2E_DEPLOYMENT_MODE` and `RUNNING_PROW`.
 
 ---
@@ -206,8 +206,8 @@ You can put several tags on one scenario. To document why a scenario is skipped,
 
 - **before_all**: Sets `deployment_mode`, `is_library_mode`, detects or overrides `default_model` / `default_provider`, sets `faiss_vector_store_id`.
 - **before_feature**: Applies feature-level config and restarts container for `Authorized`, `RBAC`, `RHIdentity`, `Feedback`, `MCP`.
-- **before_scenario**: Skips scenarios for `@skip`, `@local`, `@skip-in-library-mode`; applies scenario config for `InvalidFeedbackStorageConfig` / `NoCacheConfig`; for `@disable-shields` (server mode) unregisters the shield.
-- **after_scenario**: Restores Llama Stack if it was disrupted; restores config and restarts for scenario config tags; for `@disable-shields` re-registers the shield.
+- **before_scenario**: Skips scenarios for `@skip`, `@local`, `@skip-in-library-mode`; applies scenario config for `InvalidFeedbackStorageConfig` / `NoCacheConfig`.
+- **after_scenario**: Restores Llama Stack if it was disrupted; restores config and restarts for scenario config tags.
 - **after_feature**: Restores config and restarts for `Authorized`, `RBAC`, `RHIdentity`, `MCP`; deletes feedback conversations for `Feedback`.
 
 ---

docs/user_doc/a2a_protocol.md

@@ -43,7 +43,7 @@ The A2A protocol is an open standard for agent-to-agent communication that allow
 │  ┌──────────────────────────────────────────────────────────┐   │
 │  │                  Llama Stack Client                      │   │
 │  │  - Responses API (streaming responses)                   │   │
-│  │  - Tools, Shields, RAG integration                       │   │
+│  │  - Tools, RAG integration                                │   │
 │  └──────────────────────────────────────────────────────────┘   │
 └─────────────────────────────────────────────────────────────────┘
 ```