Skip to content

Commit 0501018

Browse files
committed
sq
Signed-off-by: Haoyu Sun <hasun@redhat.com>
1 parent ef3b3ab commit 0501018

3 files changed

Lines changed: 213 additions & 121 deletions

File tree

Containerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# vim: set filetype=dockerfile
22
ARG LIGHTSPEED_RAG_CONTENT_IMAGE=quay.io/redhat-user-workloads/crt-nshift-lightspeed-tenant/own-app-lightspeed-rag-content@sha256:51c25627274f0c8a1651dbc986a713bf4fc388b1b1037e3df759a28049d81382
3-
ARG BUILDER_BASE_IMAGE=registry.access.redhat.com/ubi9/python-312
4-
ARG RUNTIME_BASE_IMAGE=registry.access.redhat.com/ubi9/python-312-minimal
3+
ARG BUILDER_BASE_IMAGE=registry.redhat.io/rhel9/python-312@sha256:46f883684d02cef2a7abb0c4124f18308ad920018d76c5c56f130dae02bfed05
4+
ARG RUNTIME_BASE_IMAGE=registry.redhat.io/rhel9/python-312-minimal@sha256:804b928fd278fa03c2edf0352378eca73c8efcf665c6e0180e074340b9f22a50
55
FROM --platform=linux/amd64 ${LIGHTSPEED_RAG_CONTENT_IMAGE} AS lightspeed-rag-content
66

77
FROM --platform=$BUILDPLATFORM ${BUILDER_BASE_IMAGE} AS builder

README.md

Lines changed: 96 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -18,98 +18,98 @@ configure model, and connect to it.
1818
<!-- the following line is used by tool to autogenerate Table of Content when the document is changed -->
1919
<!-- vim-markdown-toc GFM -->
2020

21-
* [About The Project](#about-the-project)
22-
* [Prerequisites](#prerequisites)
23-
* [Installation](#installation)
24-
* [1. Clone the repo](#1-clone-the-repo)
25-
* [2. Install python packages](#2-install-python-packages)
26-
* [3. Get API keys](#3-get-api-keys)
27-
* [OpenAI](#openai)
28-
* [Azure OpenAI](#azure-openai)
29-
* [WatsonX](#watsonx)
30-
* [OpenShift AI](#openshift-ai)
31-
* [RHEL AI](#rhel-ai)
32-
* [Locally running InstructLab](#locally-running-instructlab)
33-
* [4. Store local copies of API keys securely](#4-store-local-copies-of-api-keys-securely)
34-
* [Configuration](#configuration)
35-
* [1. Configure OpenShift LightSpeed (OLS)](#1-configure-openshift-lightspeed-ols)
36-
* [2. Configure LLM providers](#2-configure-llm-providers)
37-
* [OpenAI provider](#openai-provider)
38-
* [Azure OpenAI](#azure-openai-1)
39-
* [WatsonX](#watsonx-1)
40-
* [RHEL AI provider](#rhel-ai-provider)
41-
* [Red Hat OpenShift AI](#red-hat-openshift-ai)
42-
* [Local *ollama* server](#local-ollama-server)
43-
* [3. Configure OLS Authentication](#3-configure-ols-authentication)
44-
* [3.1. K8S-based auth mechanism](#31-k8s-based-auth-mechanism)
45-
* [3.2. no-op auth mechanism](#32-no-op-auth-mechanism)
46-
* [4. Configure OLS TLS communication](#4-configure-ols-tls-communication)
47-
* [5. (Optional) Configure RAG](#5-optional-configure-rag)
48-
* [5.1 OCP documentation](#51-ocp-documentation)
49-
* [5.2 BYOK](#52-byok)
50-
* [5.3 Confirming the OLS is loading the configured vector databases.](#53-confirming-the-ols-is-loading-the-configured-vector-databases)
51-
* [6. (Optional) Configure conversation cache](#6-optional-configure-conversation-cache)
52-
* [7. (Optional) Incorporating additional CA(s). You have the option to include an extra TLS certificate into the OLS trust store as follows.](#7-optional-incorporating-additional-cas-you-have-the-option-to-include-an-extra-tls-certificate-into-the-ols-trust-store-as-follows)
53-
* [8. (Optional) Configure the number of workers](#8-optional-configure-the-number-of-workers)
54-
* [9. Registering a new LLM provider](#9-registering-a-new-llm-provider)
55-
* [10. TLS security profiles](#10-tls-security-profiles)
56-
* [11. System prompt](#11-system-prompt)
57-
* [12. Quota limits](#12-quota-limits)
58-
* [Tokens and token quota limits](#tokens-and-token-quota-limits)
59-
* [Quota limiter features](#quota-limiter-features)
60-
* [Configuration format](#configuration-format)
61-
* [13. Configuration dump](#13-configuration-dump)
62-
* [14. Cluster introspection](#14-cluster-introspection)
63-
* [MCP Server Configuration](#mcp-server-configuration)
64-
* [Basic Configuration](#basic-configuration)
65-
* [MCP Server Authentication](#mcp-server-authentication)
66-
* [1. Static Tokens from Files (Recommended for Service Credentials)](#1-static-tokens-from-files-recommended-for-service-credentials)
67-
* [2. Kubernetes Token (User Context)](#2-kubernetes-token-user-context)
68-
* [3. Client-Provided Tokens (Per-Request)](#3-client-provided-tokens-per-request)
69-
* [OpenShift MCP Server Example](#openshift-mcp-server-example)
70-
* [Usage](#usage)
71-
* [Deployments](#deployments)
72-
* [Local Deployment](#local-deployment)
73-
* [Run the server](#run-the-server)
74-
* [Optionally run with podman](#optionally-run-with-podman)
75-
* [Validation if the logged-in user is authorized to access service](#validation-if-the-logged-in-user-is-authorized-to-access-service)
76-
* [Swagger UI](#swagger-ui)
77-
* [OpenAPI](#openapi)
78-
* [Metrics](#metrics)
79-
* [Gradio UI](#gradio-ui)
80-
* [Swagger UI](#swagger-ui-1)
81-
* [CPU profiling](#cpu-profiling)
82-
* [Memory profiling](#memory-profiling)
83-
* [Deploying OLS on OpenShift](#deploying-ols-on-openshift)
84-
* [Project structure](#project-structure)
85-
* [Overall architecture](#overall-architecture)
86-
* [FastAPI server](#fastapi-server)
87-
* [Authorization checker](#authorization-checker)
88-
* [Query handler](#query-handler)
89-
* [Redactor](#redactor)
90-
* [Question validator](#question-validator)
91-
* [Document summarizer](#document-summarizer)
92-
* [Conversation history cache interface](#conversation-history-cache-interface)
93-
* [Conversation history cache implementations](#conversation-history-cache-implementations)
94-
* [In-memory cache](#in-memory-cache)
95-
* [Postgres cache](#postgres-cache)
96-
* [LLM providers registry](#llm-providers-registry)
97-
* [LLM providers interface implementations](#llm-providers-interface-implementations)
98-
* [Sequence diagram](#sequence-diagram)
99-
* [Token truncation algorithm](#token-truncation-algorithm)
100-
* [Development workflow](#development-workflow)
101-
* [Additional tools](#additional-tools)
102-
* [Utility to generate OpenAPI schema](#utility-to-generate-openapi-schema)
103-
* [Path](#path)
104-
* [Usage](#usage-1)
105-
* [Konflux](#konflux)
106-
* [Updating Dependencies for Hermetic Builds](#updating-dependencies-for-hermetic-builds)
107-
* [When to Update Dependency Files](#when-to-update-dependency-files)
108-
* [Updating Python Dependencies](#updating-python-dependencies)
109-
* [Updating RPM Dependencies](#updating-rpm-dependencies)
110-
* [Question and Answer Quality Evaluation](#question-and-answer-quality-evaluation)
111-
* [Contributing](#contributing)
112-
* [License](#license)
21+
- [About The Project](#about-the-project)
22+
- [Prerequisites](#prerequisites)
23+
- [Installation](#installation)
24+
- [1. Clone the repo](#1-clone-the-repo)
25+
- [2. Install python packages](#2-install-python-packages)
26+
- [3. Get API keys](#3-get-api-keys)
27+
- [OpenAI](#openai)
28+
- [Azure OpenAI](#azure-openai)
29+
- [WatsonX](#watsonx)
30+
- [OpenShift AI](#openshift-ai)
31+
- [RHEL AI](#rhel-ai)
32+
- [Locally running InstructLab](#locally-running-instructlab)
33+
- [4. Store local copies of API keys securely](#4-store-local-copies-of-api-keys-securely)
34+
- [Configuration](#configuration)
35+
- [1. Configure OpenShift LightSpeed (OLS)](#1-configure-openshift-lightspeed-ols)
36+
- [2. Configure LLM providers](#2-configure-llm-providers)
37+
- [OpenAI provider](#openai-provider)
38+
- [Azure OpenAI](#azure-openai-1)
39+
- [WatsonX](#watsonx-1)
40+
- [RHEL AI provider](#rhel-ai-provider)
41+
- [Red Hat OpenShift AI](#red-hat-openshift-ai)
42+
- [Local *ollama* server](#local-ollama-server)
43+
- [3. Configure OLS Authentication](#3-configure-ols-authentication)
44+
- [3.1. K8S-based auth mechanism](#31-k8s-based-auth-mechanism)
45+
- [3.2. no-op auth mechanism](#32-no-op-auth-mechanism)
46+
- [4. Configure OLS TLS communication](#4-configure-ols-tls-communication)
47+
- [5. (Optional) Configure RAG](#5-optional-configure-rag)
48+
- [5.1 OCP documentation](#51-ocp-documentation)
49+
- [5.2 BYOK](#52-byok)
50+
- [5.3 Confirming the OLS is loading the configured vector databases.](#53-confirming-the-ols-is-loading-the-configured-vector-databases)
51+
- [6. (Optional) Configure conversation cache](#6-optional-configure-conversation-cache)
52+
- [7. (Optional) Incorporating additional CA(s). You have the option to include an extra TLS certificate into the OLS trust store as follows.](#7-optional-incorporating-additional-cas-you-have-the-option-to-include-an-extra-tls-certificate-into-the-ols-trust-store-as-follows)
53+
- [8. (Optional) Configure the number of workers](#8-optional-configure-the-number-of-workers)
54+
- [9. Registering a new LLM provider](#9-registering-a-new-llm-provider)
55+
- [10. TLS security profiles](#10-tls-security-profiles)
56+
- [11. System prompt](#11-system-prompt)
57+
- [12. Quota limits](#12-quota-limits)
58+
- [Tokens and token quota limits](#tokens-and-token-quota-limits)
59+
- [Quota limiter features](#quota-limiter-features)
60+
- [Configuration format](#configuration-format)
61+
- [13. Configuration dump](#13-configuration-dump)
62+
- [14. Cluster introspection](#14-cluster-introspection)
63+
- [MCP Server Configuration](#mcp-server-configuration)
64+
- [Basic Configuration](#basic-configuration)
65+
- [MCP Server Authentication](#mcp-server-authentication)
66+
- [1. Static Tokens from Files (Recommended for Service Credentials)](#1-static-tokens-from-files-recommended-for-service-credentials)
67+
- [2. Kubernetes Token (User Context)](#2-kubernetes-token-user-context)
68+
- [3. Client-Provided Tokens (Per-Request)](#3-client-provided-tokens-per-request)
69+
- [OpenShift MCP Server Example](#openshift-mcp-server-example)
70+
- [Usage](#usage)
71+
- [Deployments](#deployments)
72+
- [Local Deployment](#local-deployment)
73+
- [Run the server](#run-the-server)
74+
- [Optionally run with podman](#optionally-run-with-podman)
75+
- [Validation if the logged-in user is authorized to access service](#validation-if-the-logged-in-user-is-authorized-to-access-service)
76+
- [Swagger UI](#swagger-ui)
77+
- [OpenAPI](#openapi)
78+
- [Metrics](#metrics)
79+
- [Gradio UI](#gradio-ui)
80+
- [Swagger UI](#swagger-ui-1)
81+
- [CPU profiling](#cpu-profiling)
82+
- [Memory profiling](#memory-profiling)
83+
- [Deploying OLS on OpenShift](#deploying-ols-on-openshift)
84+
- [Project structure](#project-structure)
85+
- [Overall architecture](#overall-architecture)
86+
- [FastAPI server](#fastapi-server)
87+
- [Authorization checker](#authorization-checker)
88+
- [Query handler](#query-handler)
89+
- [Redactor](#redactor)
90+
- [Question validator](#question-validator)
91+
- [Document summarizer](#document-summarizer)
92+
- [Conversation history cache interface](#conversation-history-cache-interface)
93+
- [Conversation history cache implementations](#conversation-history-cache-implementations)
94+
- [In-memory cache](#in-memory-cache)
95+
- [Postgres cache](#postgres-cache)
96+
- [LLM providers registry](#llm-providers-registry)
97+
- [LLM providers interface implementations](#llm-providers-interface-implementations)
98+
- [Sequence diagram](#sequence-diagram)
99+
- [Token truncation algorithm](#token-truncation-algorithm)
100+
- [Development workflow](#development-workflow)
101+
- [Additional tools](#additional-tools)
102+
- [Utility to generate OpenAPI schema](#utility-to-generate-openapi-schema)
103+
- [Path](#path)
104+
- [Usage](#usage-1)
105+
- [Konflux](#konflux)
106+
- [Updating Dependencies for Hermetic Builds](#updating-dependencies-for-hermetic-builds)
107+
- [When to Update Dependency Files](#when-to-update-dependency-files)
108+
- [Updating Python Dependencies](#updating-python-dependencies)
109+
- [Updating RPM Dependencies](#updating-rpm-dependencies)
110+
- [Question and Answer Quality Evaluation](#question-and-answer-quality-evaluation)
111+
- [Contributing](#contributing)
112+
- [License](#license)
113113

114114
<!-- vim-markdown-toc -->
115115

@@ -1177,7 +1177,7 @@ make schema
11771177
# Konflux
11781178

11791179
The official image of Lightspeed Service is built on [Konflux](https://konflux-ui.apps.stone-prd-rh01.pg1f.p1.openshiftapps.com/ns/crt-nshift-lightspeed-tenant/applications/ols).
1180-
We have both x86_64 and ARM64 images.
1180+
We have both amd64 and aarch64 images.
11811181

11821182
## Updating Dependencies for Hermetic Builds
11831183

@@ -1211,19 +1211,18 @@ The script also updates the Tekton pipeline configurations (`.tekton/lightspeed-
12111211
**Prerequisites:**
12121212
- Install [rpm-lockfile-prototype](https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#installation)
12131213
- Have an active RHEL Subscription, get activation keys from [RH console](https://console.redhat.com/insights/connector/activation-keys)
1214-
- Have `dnf` installed in system
12151214

12161215
**Steps:**
12171216

12181217
1. **List your RPM packages** in `rpms.in.yaml` under the `packages` field
12191218

12201219
2. **If you changed the base image**, extract its repo file:
12211220
```shell
1222-
# UBI images
1223-
podman run -it $BASE_IMAGE cat /etc/yum.repos.d/ubi.repo > ubi.repo
12241221
# RHEL images
12251222
podman run -it $BASE_IMAGE cat /etc/yum.repos.d/redhat.repo > redhat.repo
12261223
```
1224+
The base image is configured: 1. directly in the [ContainerFile](Containerfile) for Prow CI builds. 2. in [build.args](build.args) for Konflux builds.
1225+
12271226
If the repo file contains too many entries, we can filter them and keep only required repositories.
12281227
Here is the command to check active repositories:
12291228
```shell
@@ -1242,7 +1241,7 @@ make konflux-rpm-lock
12421241
This creates `rpms.lock.yaml` with pinned RPM versions.
12431242

12441243

1245-
## Question and Answer Quality Evaluation
1244+
# Question and Answer Quality Evaluation
12461245
There is an extensive suite of evaluation tools and scripts available in this
12471246
repository if you are interested in exploring different LLMs and their
12481247
performance. Please look at

0 commit comments

Comments
 (0)