@@ -18,98 +18,98 @@ configure model, and connect to it.
1818<!-- the following line is used by tool to autogenerate Table of Content when the document is changed -->
1919<!-- vim-markdown-toc GFM -->
2020
21- * [ About The Project] ( #about-the-project )
22- * [ Prerequisites] ( #prerequisites )
23- * [ Installation] ( #installation )
24- * [ 1. Clone the repo] ( #1-clone-the-repo )
25- * [ 2. Install python packages] ( #2-install-python-packages )
26- * [ 3. Get API keys] ( #3-get-api-keys )
27- * [ OpenAI] ( #openai )
28- * [ Azure OpenAI] ( #azure-openai )
29- * [ WatsonX] ( #watsonx )
30- * [ OpenShift AI] ( #openshift-ai )
31- * [ RHEL AI] ( #rhel-ai )
32- * [ Locally running InstructLab] ( #locally-running-instructlab )
33- * [ 4. Store local copies of API keys securely] ( #4-store-local-copies-of-api-keys-securely )
34- * [ Configuration] ( #configuration )
35- * [ 1. Configure OpenShift LightSpeed (OLS)] ( #1-configure-openshift-lightspeed-ols )
36- * [ 2. Configure LLM providers] ( #2-configure-llm-providers )
37- * [ OpenAI provider] ( #openai-provider )
38- * [ Azure OpenAI] ( #azure-openai-1 )
39- * [ WatsonX] ( #watsonx-1 )
40- * [ RHEL AI provider] ( #rhel-ai-provider )
41- * [ Red Hat OpenShift AI] ( #red-hat-openshift-ai )
42- * [ Local * ollama* server] ( #local-ollama-server )
43- * [ 3. Configure OLS Authentication] ( #3-configure-ols-authentication )
44- * [ 3.1. K8S-based auth mechanism] ( #31-k8s-based-auth-mechanism )
45- * [ 3.2. no-op auth mechanism] ( #32-no-op-auth-mechanism )
46- * [ 4. Configure OLS TLS communication] ( #4-configure-ols-tls-communication )
47- * [ 5. (Optional) Configure RAG] ( #5-optional-configure-rag )
48- * [ 5.1 OCP documentation] ( #51-ocp-documentation )
49- * [ 5.2 BYOK] ( #52-byok )
50- * [ 5.3 Confirming the OLS is loading the configured vector databases.] ( #53-confirming-the-ols-is-loading-the-configured-vector-databases )
51- * [ 6. (Optional) Configure conversation cache] ( #6-optional-configure-conversation-cache )
52- * [ 7. (Optional) Incorporating additional CA(s). You have the option to include an extra TLS certificate into the OLS trust store as follows.] ( #7-optional-incorporating-additional-cas-you-have-the-option-to-include-an-extra-tls-certificate-into-the-ols-trust-store-as-follows )
53- * [ 8. (Optional) Configure the number of workers] ( #8-optional-configure-the-number-of-workers )
54- * [ 9. Registering a new LLM provider] ( #9-registering-a-new-llm-provider )
55- * [ 10. TLS security profiles] ( #10-tls-security-profiles )
56- * [ 11. System prompt] ( #11-system-prompt )
57- * [ 12. Quota limits] ( #12-quota-limits )
58- * [ Tokens and token quota limits] ( #tokens-and-token-quota-limits )
59- * [ Quota limiter features] ( #quota-limiter-features )
60- * [ Configuration format] ( #configuration-format )
61- * [ 13. Configuration dump] ( #13-configuration-dump )
62- * [ 14. Cluster introspection] ( #14-cluster-introspection )
63- * [ MCP Server Configuration] ( #mcp-server-configuration )
64- * [ Basic Configuration] ( #basic-configuration )
65- * [ MCP Server Authentication] ( #mcp-server-authentication )
66- * [ 1. Static Tokens from Files (Recommended for Service Credentials)] ( #1-static-tokens-from-files-recommended-for-service-credentials )
67- * [ 2. Kubernetes Token (User Context)] ( #2-kubernetes-token-user-context )
68- * [ 3. Client-Provided Tokens (Per-Request)] ( #3-client-provided-tokens-per-request )
69- * [ OpenShift MCP Server Example] ( #openshift-mcp-server-example )
70- * [ Usage] ( #usage )
71- * [ Deployments] ( #deployments )
72- * [ Local Deployment] ( #local-deployment )
73- * [ Run the server] ( #run-the-server )
74- * [ Optionally run with podman] ( #optionally-run-with-podman )
75- * [ Validation if the logged-in user is authorized to access service] ( #validation-if-the-logged-in-user-is-authorized-to-access-service )
76- * [ Swagger UI] ( #swagger-ui )
77- * [ OpenAPI] ( #openapi )
78- * [ Metrics] ( #metrics )
79- * [ Gradio UI] ( #gradio-ui )
80- * [ Swagger UI] ( #swagger-ui-1 )
81- * [ CPU profiling] ( #cpu-profiling )
82- * [ Memory profiling] ( #memory-profiling )
83- * [ Deploying OLS on OpenShift] ( #deploying-ols-on-openshift )
84- * [ Project structure] ( #project-structure )
85- * [ Overall architecture] ( #overall-architecture )
86- * [ FastAPI server] ( #fastapi-server )
87- * [ Authorization checker] ( #authorization-checker )
88- * [ Query handler] ( #query-handler )
89- * [ Redactor] ( #redactor )
90- * [ Question validator] ( #question-validator )
91- * [ Document summarizer] ( #document-summarizer )
92- * [ Conversation history cache interface] ( #conversation-history-cache-interface )
93- * [ Conversation history cache implementations] ( #conversation-history-cache-implementations )
94- * [ In-memory cache] ( #in-memory-cache )
95- * [ Postgres cache] ( #postgres-cache )
96- * [ LLM providers registry] ( #llm-providers-registry )
97- * [ LLM providers interface implementations] ( #llm-providers-interface-implementations )
98- * [ Sequence diagram] ( #sequence-diagram )
99- * [ Token truncation algorithm] ( #token-truncation-algorithm )
100- * [ Development workflow] ( #development-workflow )
101- * [ Additional tools] ( #additional-tools )
102- * [ Utility to generate OpenAPI schema] ( #utility-to-generate-openapi-schema )
103- * [ Path] ( #path )
104- * [ Usage] ( #usage-1 )
105- * [ Konflux] ( #konflux )
106- * [ Updating Dependencies for Hermetic Builds] ( #updating-dependencies-for-hermetic-builds )
107- * [ When to Update Dependency Files] ( #when-to-update-dependency-files )
108- * [ Updating Python Dependencies] ( #updating-python-dependencies )
109- * [ Updating RPM Dependencies] ( #updating-rpm-dependencies )
110- * [ Question and Answer Quality Evaluation] ( #question-and-answer-quality-evaluation )
111- * [ Contributing] ( #contributing )
112- * [ License] ( #license )
21+ - [ About The Project] ( #about-the-project )
22+ - [ Prerequisites] ( #prerequisites )
23+ - [ Installation] ( #installation )
24+ - [ 1. Clone the repo] ( #1-clone-the-repo )
25+ - [ 2. Install python packages] ( #2-install-python-packages )
26+ - [ 3. Get API keys] ( #3-get-api-keys )
27+ - [ OpenAI] ( #openai )
28+ - [ Azure OpenAI] ( #azure-openai )
29+ - [ WatsonX] ( #watsonx )
30+ - [ OpenShift AI] ( #openshift-ai )
31+ - [ RHEL AI] ( #rhel-ai )
32+ - [ Locally running InstructLab] ( #locally-running-instructlab )
33+ - [ 4. Store local copies of API keys securely] ( #4-store-local-copies-of-api-keys-securely )
34+ - [ Configuration] ( #configuration )
35+ - [ 1. Configure OpenShift LightSpeed (OLS)] ( #1-configure-openshift-lightspeed-ols )
36+ - [ 2. Configure LLM providers] ( #2-configure-llm-providers )
37+ - [ OpenAI provider] ( #openai-provider )
38+ - [ Azure OpenAI] ( #azure-openai-1 )
39+ - [ WatsonX] ( #watsonx-1 )
40+ - [ RHEL AI provider] ( #rhel-ai-provider )
41+ - [ Red Hat OpenShift AI] ( #red-hat-openshift-ai )
42+ - [ Local * ollama* server] ( #local-ollama-server )
43+ - [ 3. Configure OLS Authentication] ( #3-configure-ols-authentication )
44+ - [ 3.1. K8S-based auth mechanism] ( #31-k8s-based-auth-mechanism )
45+ - [ 3.2. no-op auth mechanism] ( #32-no-op-auth-mechanism )
46+ - [ 4. Configure OLS TLS communication] ( #4-configure-ols-tls-communication )
47+ - [ 5. (Optional) Configure RAG] ( #5-optional-configure-rag )
48+ - [ 5.1 OCP documentation] ( #51-ocp-documentation )
49+ - [ 5.2 BYOK] ( #52-byok )
50+ - [ 5.3 Confirming the OLS is loading the configured vector databases.] ( #53-confirming-the-ols-is-loading-the-configured-vector-databases )
51+ - [ 6. (Optional) Configure conversation cache] ( #6-optional-configure-conversation-cache )
52+ - [ 7. (Optional) Incorporating additional CA(s). You have the option to include an extra TLS certificate into the OLS trust store as follows.] ( #7-optional-incorporating-additional-cas-you-have-the-option-to-include-an-extra-tls-certificate-into-the-ols-trust-store-as-follows )
53+ - [ 8. (Optional) Configure the number of workers] ( #8-optional-configure-the-number-of-workers )
54+ - [ 9. Registering a new LLM provider] ( #9-registering-a-new-llm-provider )
55+ - [ 10. TLS security profiles] ( #10-tls-security-profiles )
56+ - [ 11. System prompt] ( #11-system-prompt )
57+ - [ 12. Quota limits] ( #12-quota-limits )
58+ - [ Tokens and token quota limits] ( #tokens-and-token-quota-limits )
59+ - [ Quota limiter features] ( #quota-limiter-features )
60+ - [ Configuration format] ( #configuration-format )
61+ - [ 13. Configuration dump] ( #13-configuration-dump )
62+ - [ 14. Cluster introspection] ( #14-cluster-introspection )
63+ - [ MCP Server Configuration] ( #mcp-server-configuration )
64+ - [ Basic Configuration] ( #basic-configuration )
65+ - [ MCP Server Authentication] ( #mcp-server-authentication )
66+ - [ 1. Static Tokens from Files (Recommended for Service Credentials)] ( #1-static-tokens-from-files-recommended-for-service-credentials )
67+ - [ 2. Kubernetes Token (User Context)] ( #2-kubernetes-token-user-context )
68+ - [ 3. Client-Provided Tokens (Per-Request)] ( #3-client-provided-tokens-per-request )
69+ - [ OpenShift MCP Server Example] ( #openshift-mcp-server-example )
70+ - [ Usage] ( #usage )
71+ - [ Deployments] ( #deployments )
72+ - [ Local Deployment] ( #local-deployment )
73+ - [ Run the server] ( #run-the-server )
74+ - [ Optionally run with podman] ( #optionally-run-with-podman )
75+ - [ Validation if the logged-in user is authorized to access service] ( #validation-if-the-logged-in-user-is-authorized-to-access-service )
76+ - [ Swagger UI] ( #swagger-ui )
77+ - [ OpenAPI] ( #openapi )
78+ - [ Metrics] ( #metrics )
79+ - [ Gradio UI] ( #gradio-ui )
80+ - [ Swagger UI] ( #swagger-ui-1 )
81+ - [ CPU profiling] ( #cpu-profiling )
82+ - [ Memory profiling] ( #memory-profiling )
83+ - [ Deploying OLS on OpenShift] ( #deploying-ols-on-openshift )
84+ - [ Project structure] ( #project-structure )
85+ - [ Overall architecture] ( #overall-architecture )
86+ - [ FastAPI server] ( #fastapi-server )
87+ - [ Authorization checker] ( #authorization-checker )
88+ - [ Query handler] ( #query-handler )
89+ - [ Redactor] ( #redactor )
90+ - [ Question validator] ( #question-validator )
91+ - [ Document summarizer] ( #document-summarizer )
92+ - [ Conversation history cache interface] ( #conversation-history-cache-interface )
93+ - [ Conversation history cache implementations] ( #conversation-history-cache-implementations )
94+ - [ In-memory cache] ( #in-memory-cache )
95+ - [ Postgres cache] ( #postgres-cache )
96+ - [ LLM providers registry] ( #llm-providers-registry )
97+ - [ LLM providers interface implementations] ( #llm-providers-interface-implementations )
98+ - [ Sequence diagram] ( #sequence-diagram )
99+ - [ Token truncation algorithm] ( #token-truncation-algorithm )
100+ - [ Development workflow] ( #development-workflow )
101+ - [ Additional tools] ( #additional-tools )
102+ - [ Utility to generate OpenAPI schema] ( #utility-to-generate-openapi-schema )
103+ - [ Path] ( #path )
104+ - [ Usage] ( #usage-1 )
105+ - [ Konflux] ( #konflux )
106+ - [ Updating Dependencies for Hermetic Builds] ( #updating-dependencies-for-hermetic-builds )
107+ - [ When to Update Dependency Files] ( #when-to-update-dependency-files )
108+ - [ Updating Python Dependencies] ( #updating-python-dependencies )
109+ - [ Updating RPM Dependencies] ( #updating-rpm-dependencies )
110+ - [ Question and Answer Quality Evaluation] ( #question-and-answer-quality-evaluation )
111+ - [ Contributing] ( #contributing )
112+ - [ License] ( #license )
113113
114114<!-- vim-markdown-toc -->
115115
@@ -1177,7 +1177,7 @@ make schema
11771177# Konflux
11781178
11791179The official image of Lightspeed Service is built on [ Konflux] ( https://konflux-ui.apps.stone-prd-rh01.pg1f.p1.openshiftapps.com/ns/crt-nshift-lightspeed-tenant/applications/ols ) .
1180- We have both x86_64 and ARM64 images.
1180+ We have both amd64 and aarch64 images.
11811181
11821182## Updating Dependencies for Hermetic Builds
11831183
@@ -1211,19 +1211,18 @@ The script also updates the Tekton pipeline configurations (`.tekton/lightspeed-
12111211** Prerequisites:**
12121212- Install [ rpm-lockfile-prototype] ( https://github.com/konflux-ci/rpm-lockfile-prototype?tab=readme-ov-file#installation )
12131213- Have an active RHEL Subscription, get activation keys from [ RH console] ( https://console.redhat.com/insights/connector/activation-keys )
1214- - Have ` dnf ` installed in system
12151214
12161215** Steps:**
12171216
121812171 . ** List your RPM packages** in ` rpms.in.yaml ` under the ` packages ` field
12191218
122012192 . ** If you changed the base image** , extract its repo file:
12211220``` shell
1222- # UBI images
1223- podman run -it $BASE_IMAGE cat /etc/yum.repos.d/ubi.repo > ubi.repo
12241221# RHEL images
12251222podman run -it $BASE_IMAGE cat /etc/yum.repos.d/redhat.repo > redhat.repo
12261223```
1224+ The base image is configured: 1. directly in the [ ContainerFile] ( Containerfile ) for Prow CI builds. 2. in [ build.args] ( build.args ) for Konflux builds.
1225+
12271226If the repo file contains too many entries, we can filter them and keep only required repositories.
12281227Here is the command to check active repositories:
12291228``` shell
@@ -1242,7 +1241,7 @@ make konflux-rpm-lock
12421241This creates ` rpms.lock.yaml ` with pinned RPM versions.
12431242
12441243
1245- ## Question and Answer Quality Evaluation
1244+ # Question and Answer Quality Evaluation
12461245There is an extensive suite of evaluation tools and scripts available in this
12471246repository if you are interested in exploring different LLMs and their
12481247performance. Please look at
0 commit comments