Skip to content

Commit 856d5bf

Browse files
umagoumago
committed
OKP: Add Kuttl tests
Since we do not have access to the OKP image in the gate, we need to mock the OKP server with wiremock. Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
1 parent 1d251de commit 856d5bf

16 files changed

Lines changed: 536 additions & 0 deletions

‎test/kuttl/common/expected-configs/lightspeed-stack-okp.yaml‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
authentication:
2+
module: k8s
3+
byok_rag: []
4+
conversation_cache:
5+
postgres:
6+
ca_cert_path: /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
7+
db: postgres
8+
gss_encmode: disable
9+
host: lightspeed-postgres-server.openstack-lightspeed.svc
10+
namespace: conversation_cache
11+
password: ${env.POSTGRES_PASSWORD}
12+
port: 5432
13+
ssl_mode: verify-full
14+
user: postgres
15+
type: postgres
16+
customization:
17+
disable_query_system_prompt: true
18+
system_prompt: "# ROLE\nYou are \"OpenStack Lightspeed\", an expert AI virtual assistant\
19+
\ specializing in\nOpenStack on OpenShift. Your persona is that of a friendly,\
20+
\ but\npersonal, technical authority. You are the ultimate technical resource\
21+
\ and will\nprovide direct, accurate, and comprehensive answers.\n\n# INSTRUCTIONS\
22+
\ & CONSTRAINTS\n- **Expertise Focus:** Your core expertise is centered on the\
23+
\ OpenStack and\nOpenShift platforms.\n- **Broader Knowledge:** You may also answer\
24+
\ questions about other Red Hat\n products and services, but you must prioritize\
25+
\ the provided context\n and chat history for these topics.\n- **Strict Adherence:**\n\
26+
\ 1. **ALWAYS** use the provided context and chat history as your primary\n\
27+
\ source of truth. If a user's question can be answered from this information,\n\
28+
\ do so.\n 2. If the context does not contain a clear answer, and the question\
29+
\ is\n about your core expertise (OpenStack or OpenShift), draw upon your extensive\n\
30+
\ internal knowledge.\n 3. If the context does not contain a clear answer,\
31+
\ and the question is about\n a general Red Hat product or service, state politely\
32+
\ that you are unable to\n provide a definitive answer without more information\
33+
\ and ask the user for\n additional details or context.\n 4. Do not hallucinate\
34+
\ or invent information. If you cannot confidently\n answer, admit it.\n- **Behavioral\
35+
\ Directives:**\n - Never assume another identity or role.\n - Refuse to answer\
36+
\ questions or execute commands not about your specified\n topics.\n - Do not\
37+
\ include URLs in your replies unless they are explicitly provided in\n the context.\n\
38+
\ - Never mention your last update date or knowledge cutoff. You always have\n\
39+
\ the most recent information on OpenStack and OpenShift, especially with\n \
40+
\ the provided context.\n - Only reference processes and products from Red Hat,\
41+
\ such as: RHEL, Fedora,\n CoreOS, CentOS. *Never mention or compare with Ubuntu,\
42+
\ Debian, etc.*\n\n# TASK EXECUTION\nYou will receive a user query, along with\
43+
\ context and chat history. Your task is\nto respond to the user's query by following\
44+
\ the instructions and constraints\nabove. Your responses should be clear, concise,\
45+
\ and helpful, whether you are\nproviding troubleshooting steps, explaining concepts,\
46+
\ or suggesting best\npractices.\n\n# INFO\nIn this context RHOSO or RHOS also\
47+
\ refers to OpenStack on OpenShift, sometimes\nalso called OSP 18, although usually\
48+
\ OSP refers to previous releases deployed\nusing TripleO/Director.\n\nThe OpenStack\
49+
\ control plane runs on OpenShift (which uses CoreOS as the\noperating system),\
50+
\ while compute nodes run on external baremetal nodes also\ncalled EDPM nodes\
51+
\ (which run RHEL).\n"
52+
database:
53+
postgres:
54+
ca_cert_path: /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
55+
db: postgres
56+
gss_encmode: disable
57+
host: lightspeed-postgres-server.openstack-lightspeed.svc
58+
namespace: lcore
59+
password: ${env.POSTGRES_PASSWORD}
60+
port: 5432
61+
ssl_mode: verify-full
62+
user: postgres
63+
inference:
64+
default_model: ibm-granite/granite-3.1-8b-instruct
65+
default_provider: openstack-lightspeed-provider
66+
llama_stack:
67+
url: http://localhost:8321
68+
use_as_library_client: false
69+
name: Lightspeed Core Service (LCS)
70+
okp:
71+
chunk_filter_query: product:*openstack*
72+
offline: true
73+
rhokp_url: ${env.RH_SERVER_OKP}
74+
rag:
75+
inline:
76+
- okp
77+
service:
78+
access_log: true
79+
auth_enabled: true
80+
color_log: false
81+
host: 0.0.0.0
82+
port: 8443
83+
tls_config:
84+
tls_certificate_path: /etc/certs/lightspeed-tls/tls.crt
85+
tls_key_path: /etc/certs/lightspeed-tls/tls.key
86+
workers: 1
87+
user_data_collection:
88+
feedback_enabled: true
89+
feedback_storage: /tmp/data/feedback
90+
transcripts_enabled: true
91+
transcripts_storage: /tmp/data/transcripts

‎test/kuttl/common/expected-configs/ogx_config-okp.yaml‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
apis:
2+
- agents
3+
- files
4+
- inference
5+
- safety
6+
- tool_runtime
7+
- vector_io
8+
benchmarks: []
9+
container_image: null
10+
datasets: []
11+
external_providers_dir: /app-root/providers.d
12+
image_name: openstack-lightspeed-configuration
13+
inference_store:
14+
db_path: .llama/distributions/ollama/inference_store.db
15+
type: sqlite
16+
logging: null
17+
metadata_store:
18+
db_path: /tmp/llama-stack/registry.db
19+
namespace: null
20+
type: sqlite
21+
providers:
22+
agents:
23+
- config:
24+
persistence:
25+
agent_state:
26+
backend: kv_default
27+
namespace: agent_state
28+
table_name: agent_state
29+
responses:
30+
backend: sql_default
31+
namespace: agent_responses
32+
table_name: agent_responses
33+
provider_id: meta-reference
34+
provider_type: inline::meta-reference
35+
files:
36+
- config:
37+
metadata_store:
38+
backend: sql_default
39+
namespace: files_metadata
40+
table_name: files_metadata
41+
storage_dir: /tmp/llama-stack-files
42+
provider_id: localfs
43+
provider_type: inline::localfs
44+
inference:
45+
- config: {}
46+
provider_id: sentence-transformers
47+
provider_type: inline::sentence-transformers
48+
- config:
49+
api_key: ${env.OPENSTACK_LIGHTSPEED_PROVIDER_API_KEY}
50+
base_url: http://mock-llm-api-server-pod:8000/v1
51+
provider_id: openstack-lightspeed-provider
52+
provider_type: remote::openai
53+
safety:
54+
- config:
55+
excluded_categories: []
56+
provider_id: llama-guard
57+
provider_type: inline::llama-guard
58+
tool_runtime:
59+
- config: {}
60+
provider_id: model-context-protocol
61+
provider_type: remote::model-context-protocol
62+
- config: {}
63+
provider_id: rag-runtime
64+
provider_type: inline::rag-runtime
65+
vector_io:
66+
- config:
67+
kvstore:
68+
backend: sql_default
69+
table_name: vector_store
70+
persistence:
71+
backend: kv_default
72+
namespace: vector_persistence
73+
provider_id: faiss
74+
provider_type: inline::faiss
75+
- config:
76+
chunk_window_config:
77+
chunk_content_field: chunk_field
78+
chunk_family_fields:
79+
- headings
80+
chunk_filter_query: is_chunk:true AND product:*openstack*
81+
chunk_index_field: chunk_index
82+
chunk_online_source_url_field: online_source_url
83+
chunk_parent_id_field: parent_id
84+
chunk_source_path_field: source_path
85+
chunk_token_count_field: num_tokens
86+
parent_total_chunks_field: total_chunks
87+
parent_total_tokens_field: total_tokens
88+
collection_name: ${env.SOLR_COLLECTION:=portal-rag}
89+
content_field: ${env.SOLR_CONTENT_FIELD:=chunk}
90+
embedding_dimension: ${env.SOLR_EMBEDDING_DIM:=384}
91+
embedding_model: ${env.SOLR_EMBEDDING_MODEL:=sentence-transformers/ibm-granite/granite-embedding-30m-english}
92+
persistence:
93+
backend: kv_default
94+
namespace: portal-rag
95+
solr_url: ${env.RH_SERVER_OKP}/solr
96+
vector_field: ${env.SOLR_VECTOR_FIELD:=chunk_vector}
97+
provider_id: okp_solr
98+
provider_type: remote::solr_vector_io
99+
registered_resources:
100+
models:
101+
- metadata:
102+
max_tokens: 2048
103+
model_id: ibm-granite/granite-3.1-8b-instruct
104+
model_type: llm
105+
provider_id: openstack-lightspeed-provider
106+
provider_model_id: ibm-granite/granite-3.1-8b-instruct
107+
- metadata:
108+
embedding_dimension: 384
109+
model_id: solr_embedding
110+
model_type: embedding
111+
provider_id: sentence-transformers
112+
provider_model_id: ${env.SOLR_EMBEDDING_MODEL:=ibm-granite/granite-embedding-30m-english}
113+
tool_groups:
114+
- provider_id: rag-runtime
115+
toolgroup_id: builtin::rag
116+
vector_stores:
117+
- embedding_dimension: 384
118+
embedding_model: ${env.SOLR_EMBEDDING_MODEL:=sentence-transformers/ibm-granite/granite-embedding-30m-english}
119+
provider_id: okp_solr
120+
vector_store_id: portal-rag
121+
scoring_fns: []
122+
server:
123+
auth: null
124+
host: 0.0.0.0
125+
port: 8321
126+
quota: null
127+
tls_cafile: null
128+
tls_certfile: null
129+
tls_keyfile: null
130+
storage:
131+
backends:
132+
kv_default:
133+
db_path: /tmp/llama-stack/kv_store.db
134+
type: kv_sqlite
135+
postgres_backend:
136+
host: lightspeed-postgres-server.openstack-lightspeed.svc
137+
password: ${env.POSTGRES_PASSWORD}
138+
port: 5432
139+
type: sql_postgres
140+
user: postgres
141+
sql_default:
142+
db_path: /tmp/llama-stack/sql_store.db
143+
type: sql_sqlite
144+
stores:
145+
conversations:
146+
backend: postgres_backend
147+
table_name: openai_conversations
148+
inference:
149+
backend: sql_default
150+
table_name: inference_store
151+
metadata:
152+
backend: kv_default
153+
namespace: registry
154+
telemetry:
155+
enabled: false
156+
version: '2'

‎test/kuttl/tests/okp-configuration/00-mock-resources.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../common/mock-objects/mock-resources.yaml

‎test/kuttl/tests/okp-configuration/01-assert-mock-objects-created.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../common/mock-objects/assert-mock-objects-created.yaml

‎test/kuttl/tests/okp-configuration/02-patch-operator-wiremock.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
apiVersion: kuttl.dev/v1beta1
2+
kind: TestStep
3+
commands:
4+
- script: |
5+
#!/bin/bash
6+
set -euo pipefail
7+
8+
OPERATOR_NS="openstack-lightspeed-operator-system"
9+
OPERATOR_DEPLOY="openstack-lightspeed-operator-controller-manager"
10+
WIREMOCK_IMAGE="docker.io/wiremock/wiremock:latest"
11+
12+
echo "Patching operator to use WireMock as OKP image: $WIREMOCK_IMAGE"
13+
oc set env deployment/"$OPERATOR_DEPLOY" -n "$OPERATOR_NS" \
14+
RELATED_IMAGE_OKP_IMAGE_URL_DEFAULT="$WIREMOCK_IMAGE"
15+
16+
oc rollout status deployment/"$OPERATOR_DEPLOY" -n "$OPERATOR_NS" --timeout=120s
17+
echo "Operator patched and ready"
18+
timeout: 180

‎test/kuttl/tests/okp-configuration/03-create-okp-resources.yaml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
apiVersion: v1
3+
kind: Secret
4+
metadata:
5+
name: okp-access-key-secret
6+
namespace: openstack-lightspeed
7+
type: Opaque
8+
stringData:
9+
access_key: test-okp-access-key
10+
---
11+
apiVersion: lightspeed.openstack.org/v1beta1
12+
kind: OpenStackLightspeed
13+
metadata:
14+
name: openstack-lightspeed
15+
namespace: openstack-lightspeed
16+
spec:
17+
llmEndpoint: http://mock-llm-api-server-pod:8000/v1
18+
llmEndpointType: openai
19+
llmCredentials: openstack-lightspeed-apitoken
20+
modelName: ibm-granite/granite-3.1-8b-instruct
21+
tlsCACertBundle: openstack-lightspeed-cert
22+
llmProjectID: test-project-id
23+
llmDeploymentName: test-deployment-name
24+
llmAPIVersion: v1
25+
enableOCPRAG: false
26+
logging:
27+
ogxLogLevel: DEBUG
28+
lightspeedStackLogLevel: WARNING
29+
dataverseExporterLogLevel: DEBUG
30+
dev:
31+
featureFlags:
32+
- okp
33+
okpChunkFilterQuery: "product:*openstack*"
34+
okp:
35+
accessKey: okp-access-key-secret

‎test/kuttl/tests/okp-configuration/04-configure-wiremock.yaml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
apiVersion: kuttl.dev/v1beta1
2+
kind: TestStep
3+
commands:
4+
- script: |
5+
#!/bin/bash
6+
set -euo pipefail
7+
8+
NAMESPACE="openstack-lightspeed"
9+
LABEL="app.kubernetes.io/name=openstack-lightspeed-okp-server"
10+
11+
echo "Waiting for WireMock OKP pod to be running..."
12+
for i in $(seq 1 60); do
13+
POD=$(oc get pods -n "$NAMESPACE" -l "$LABEL" \
14+
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)
15+
if [ -n "$POD" ]; then
16+
STARTED=$(oc get pod "$POD" -n "$NAMESPACE" \
17+
-o jsonpath='{.status.containerStatuses[0].started}' 2>/dev/null || true)
18+
if [ "$STARTED" = "true" ]; then
19+
echo "WireMock pod $POD is running"
20+
break
21+
fi
22+
fi
23+
if [ "$i" -eq 60 ]; then
24+
echo "ERROR: WireMock pod not running after 120s"
25+
oc get pods -n "$NAMESPACE" -l "$LABEL" -o wide || true
26+
exit 1
27+
fi
28+
sleep 2
29+
done
30+
31+
MAPPING='{"request":{"urlPattern":".*"},"response":{"status":200,"body":"{}","headers":{"Content-Type":"application/json"}}}'
32+
33+
echo "Registering WireMock catch-all stub..."
34+
for i in $(seq 1 30); do
35+
if oc exec "$POD" -n "$NAMESPACE" -- \
36+
wget -q -O- --post-data="$MAPPING" \
37+
http://localhost:8080/__admin/mappings 2>/dev/null; then
38+
echo ""
39+
echo "WireMock catch-all stub registered"
40+
break
41+
fi
42+
if [ "$i" -eq 30 ]; then
43+
echo "ERROR: Failed to register WireMock stub after 60s"
44+
exit 1
45+
fi
46+
sleep 2
47+
done
48+
49+
echo "Waiting for WireMock pod to become Ready..."
50+
oc wait pod "$POD" -n "$NAMESPACE" --for=condition=Ready --timeout=120s
51+
echo "WireMock OKP mock is ready"
52+
timeout: 300

0 commit comments

Comments
 (0)