Skip to content

Commit e6b1d6c

Browse files
openshift-merge-bot[bot]openshift-merge-bot[bot]
authored
Merge pull request #857 from openstack-k8s-operators/document-kuttl-tests
Update and extend docs for Kuttl tests
2 parents 64eb992 + 7d076f2 commit e6b1d6c

9 files changed

Lines changed: 465 additions & 7 deletions

File tree

‎README.md‎

Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -299,17 +299,43 @@ The following procedure is to be performed after the
299299
openstack-edpm-ipam False Deployment in progress
300300
```
301301
302-
## Running kuttl tests locally
302+
## Running kuttl tests in a development environment
303303
304-
For the default suite, simply run `make kuttl-test`.
304+
### Quick start
305305
306-
For standalone suites, you must follow these steps:
306+
To run all the test suites, simply run:
307307
308-
1. Set up the testing namespace using install_yamls: `cd install_yamls && make kuttl_common_prep heat heat_deploy NAMESPACE=telemetry-kuttl-tests`
309-
2. (Optionally) Edit the list of suites to run: `cd telemetry-operator && vi kuttl-test.yaml` (Comment out any suites you don't need from the testDirs list)
310-
3. Run kuttl specifying that config file and namespace: `kubectl-kuttl test --config ./kuttl-test.yaml --namespace telemetry-kuttl-tests`
308+
```bash
309+
cd install_yamls
310+
make telemetry_kuttl
311+
```
312+
313+
> **NOTE**: It is not necessary to have a full RHOSO deployment available, only 1. make_crc and 2 make crc_storage is required
314+
315+
### Running individual test suites
316+
317+
To run a single test suite (e.g., autoscaling):
318+
319+
```bash
320+
cd install_yamls
321+
make kuttl_common_prep ovn heat heat_deploy certmanager telemetry telemetry_deploy_prep # This sets up the tests dependencies. Only run it if make telemetry_kuttl wasn't run before
322+
cd telemetry-operator/
323+
oc kuttl test --test tls # Replace tls with the name of the test suite that wants to be run
324+
```
325+
326+
> **NOTE**: Make sure to have Kuttl dependencies are available.
327+
328+
### Debug mode
329+
330+
To keep the namespace after test completion for debugging:
331+
332+
```bash
333+
oc kuttl test --test tls --skip-delete
334+
```
335+
336+
### Known Issues
311337
312-
> NOTE: (May 2024) These tests appear very reliable when running a single suite, but occasional flakiness (~ 25% failure) has been observed when they all run serially. This problem appears to be order/timing related and may or may not affect the automated CI.
338+
> **NOTE**: These tests appear very reliable when running a single suite. However, occasional flakiness (~ 25% failure) has been observed when all tests run serially. This problem appears to be order/timing related and may or may not affect the automated CI.
313339
314340
## Destroy the environment to start again
315341

‎test/kuttl/tests/autoscaling/README.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# autoscaling kuttl test
2+
3+
The "autoscaling" kuttl test covers the deployment and configuration of the Autoscaling CR, which manages Aodh, the OpenStack Alarming service used for autoscaling capabilities.
4+
5+
## Topology
6+
7+
The target topology for this test is:
8+
9+
1. one Autoscaling object managing Aodh service
10+
2. Aodh StatefulSet with 1 replica containing 4 containers:
11+
- aodh-api
12+
- aodh-evaluator
13+
- aodh-notifier
14+
- aodh-listener
15+
3. Keystone integration (KeystoneService and KeystoneEndpoint)
16+
4. Internal and public service endpoints
17+
18+
The autoscaling test steps cover:
19+
- Basic deployment with password-based authentication
20+
- Custom configuration mounting
21+
- Application credential authentication
22+
- Application credential rotation
23+
24+
### Prerequisites
25+
26+
As a prerequisite for this test, we assume:
27+
28+
1. an existing `MariaDB/Galera` entity in the target namespace
29+
2. an existing `Keystone` deployed via keystone-operator
30+
3. an existing `Memcached` instance in the target namespace
31+
4. an existing `RabbitMQ` cluster in the target namespace
32+
5. an existing `Heat` instance in the target namespace
33+
6. a running `telemetry-operator` deployed in the cluster
34+
35+
These resources are deployed via the `deps/` directory using kustomize.
36+
37+
### Run the autoscaling tests
38+
39+
Once the kuttl requirements are satisfied, the actual tests can be executed from the telemetry-operator root directory via:
40+
41+
```bash
42+
cd telemetry-operator/
43+
oc kuttl test --test autoscaling --namespace telemetry-kuttl-autoscaling
44+
```
45+
46+
Same tests can be executed from a different location via:
47+
48+
```bash
49+
oc kuttl test --config <KUTTL_CONFIG> <KUTTL_TESTS_DIR> --test autoscaling --namespace telemetry-kuttl-autoscaling
50+
```
51+
52+
> **NOTE**: <KUTTL_CONFIG> is the path where kuttl-test.yaml for the repo is located and <KUTTL_TESTS_DIR> is where the tests are located.

‎test/kuttl/tests/ceilometer/README.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# ceilometer kuttl test
2+
3+
The "ceilometer" kuttl test covers the deployment and configuration of the Ceilometer CR, which manages the OpenStack Metering service for collecting and processing measurements from OpenStack services.
4+
5+
## Topology
6+
7+
The target topology for this test is:
8+
9+
1. one Ceilometer object
10+
2. Ceilometer StatefulSet with 1 replica containing 4 containers:
11+
- ceilometer-central-agent (collects metrics via polling)
12+
- ceilometer-notification-agent (processes notifications)
13+
- sg-core (Smart Gateway core for service graph)
14+
- proxy-httpd (HTTP proxy for metrics exposure)
15+
3. kube-state-metrics StatefulSet (when ksmEnabled: true)
16+
4. mysqld-exporter StatefulSet (when mysqldExporterEnabled: true)
17+
5. Keystone integration (KeystoneService)
18+
19+
The ceilometer test steps cover:
20+
- Basic deployment with exporters enabled
21+
- Custom configuration mounting
22+
- Application credential authentication
23+
- Application credential rotation
24+
25+
### Prerequisites
26+
27+
As a prerequisite for this test, we assume:
28+
29+
1. an existing `Keystone` deployed via keystone-operator
30+
2. an existing `RabbitMQ` cluster in the target namespace for notifications
31+
3. a running `telemetry-operator` deployed in the cluster
32+
4. access to MariaDB for mysqld-exporter (if enabled)
33+
34+
These resources are deployed via the `deps/` directory using kustomize.
35+
36+
### Run the ceilometer tests
37+
38+
Once the kuttl requirements are satisfied, the actual tests can be executed from the telemetry-operator root directory via:
39+
40+
```bash
41+
cd telemetry-operator/
42+
oc kuttl test --test ceilometer --namespace telemetry-kuttl-ceilometer
43+
```
44+
45+
Same tests can be executed from a different location via:
46+
47+
```bash
48+
oc kuttl test --config <KUTTL_CONFIG> <KUTTL_TESTS_DIR> --test ceilometer --namespace telemetry-kuttl-ceilometer
49+
```
50+
51+
> **NOTE**: <KUTTL_CONFIG> is the path where kuttl-test.yaml for the repo is located and <KUTTL_TESTS_DIR> is where the tests are located.

‎test/kuttl/tests/cloudkitty/README.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# cloudkitty kuttl test
2+
3+
The "cloudkitty" kuttl test covers the deployment and configuration of the CloudKitty CR, which manages the OpenStack Rating service for cloud resources.
4+
5+
## Topology
6+
7+
The target topology for this test is:
8+
9+
1. one CloudKitty object managing two sub-components:
10+
- CloudKittyAPI (API service with 1 replica)
11+
- CloudKittyProc (rating processor with 1 replica)
12+
2. LokiStack for metrics storage backend
13+
3. CloudKittyAPI pod containing:
14+
- cloudkitty-api container
15+
- log sidecar container
16+
4. CloudKittyProc pod containing:
17+
- cloudkitty-proc container
18+
5. S3 storage (MinIO) for LokiStack backend
19+
6. TLS certificates for Loki communication
20+
7. Keystone integration (KeystoneService and KeystoneEndpoint)
21+
22+
The cloudkitty test steps cover:
23+
- Basic deployment with LokiStack integration
24+
- Custom configuration mounting for both API and Proc
25+
- Application credential authentication
26+
- Application credential rotation
27+
28+
### Prerequisites
29+
30+
As a prerequisite for this test, we assume:
31+
32+
1. an existing `MariaDB/Galera` entity in the target namespace
33+
2. an existing `Keystone` deployed via keystone-operator
34+
3. an existing `Memcached` instance in the target namespace
35+
4. an existing `RabbitMQ` cluster in the target namespace
36+
5. a running `Loki Operator` in the cluster
37+
6. S3-compatible storage (MinIO) for LokiStack backend
38+
7. a running `telemetry-operator` deployed in the cluster
39+
40+
These resources are deployed via the `deps/` directory using kustomize.
41+
42+
**Note:** This test has a longer timeout (600s) because LokiStack can take significant time to become ready.
43+
44+
### Run the cloudkitty tests
45+
46+
Once the kuttl requirements are satisfied, the actual tests can be executed from the telemetry-operator root directory via:
47+
48+
```bash
49+
cd telemetry-operator/
50+
oc kuttl test --test cloudkitty --namespace telemetry-kuttl-cloudkitty
51+
```
52+
53+
Same tests can be executed from a different location via:
54+
55+
```bash
56+
oc kuttl test --config <KUTTL_CONFIG> <KUTTL_TESTS_DIR> --test cloudkitty --namespace telemetry-kuttl-cloudkitty
57+
```
58+
59+
> **NOTE**: <KUTTL_CONFIG> is the path where kuttl-test.yaml for the repo is located and <KUTTL_TESTS_DIR> is where the tests are located.

‎test/kuttl/tests/default/README.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# default kuttl test
2+
3+
The "default" kuttl test covers the integrated deployment of all major telemetry components together, validating that they can coexist and interoperate in a complete telemetry stack.
4+
5+
## Topology
6+
7+
The target topology for this test is:
8+
9+
1. Complete telemetry stack including:
10+
- **Ceilometer** with kube-state-metrics
11+
- **MetricStorage** with Prometheus monitoring stack
12+
- **Autoscaling** with Aodh service
13+
- **CloudKitty** with LokiStack for billing/rating
14+
2. All components deployed and ready simultaneously
15+
3. Full integration with:
16+
- Keystone for authentication
17+
- MariaDB for database backend
18+
- RabbitMQ for messaging
19+
- Memcached for caching
20+
- S3 storage (MinIO) for CloudKitty/Loki
21+
22+
This test validates the complete "batteries-included" telemetry deployment scenario.
23+
24+
### Prerequisites
25+
26+
As a prerequisite for this test, we assume:
27+
28+
1. an existing `MariaDB/Galera` entity in the target namespace
29+
2. an existing `Keystone` deployed via keystone-operator
30+
3. an existing `Memcached` instance in the target namespace
31+
4. an existing `RabbitMQ` cluster in the target namespace
32+
5. an existing `Heat` instance in the target namespace
33+
6. a running `Loki Operator` in the cluster
34+
7. Red Hat Observability Stack (RHOBS) components for metrics
35+
8. S3-compatible storage (MinIO) for LokiStack backend
36+
9. Certificate infrastructure for TLS
37+
10. a running `telemetry-operator` deployed in the cluster
38+
39+
These resources are deployed via the `deps/` directory using kustomize.
40+
41+
### Run the default tests
42+
43+
Once the kuttl requirements are satisfied, the actual tests can be executed from the telemetry-operator root directory via:
44+
45+
```bash
46+
cd telemetry-operator/
47+
oc kuttl test --test default --namespace telemetry-kuttl-default
48+
```
49+
50+
Same tests can be executed from a different location via:
51+
52+
```bash
53+
oc kuttl test --config <KUTTL_CONFIG> <KUTTL_TESTS_DIR> --test default --namespace telemetry-kuttl-default
54+
```
55+
56+
> **NOTE**: <KUTTL_CONFIG> is the path where kuttl-test.yaml for the repo is located and <KUTTL_TESTS_DIR> is where the tests are located.

‎test/kuttl/tests/logging/README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# logging kuttl test
2+
3+
The "logging" kuttl test covers the deployment and configuration of the Logging CR, which manages log forwarding from OpenStack compute nodes to a centralized logging infrastructure.
4+
5+
## Topology
6+
7+
The target topology for this test is:
8+
9+
1. one Logging object managing log collection configuration
10+
2. LoadBalancer service for receiving logs from compute nodes
11+
3. Integration with Cluster Logging Operator (CLO) via Vector
12+
4. Configuration secrets for compute nodes
13+
14+
The logging test validates:
15+
- Service endpoint creation for log ingestion
16+
- Proper MetalLB configuration for load balancing
17+
- Compute node configuration secret generation
18+
19+
### Prerequisites
20+
21+
As a prerequisite for this test, we assume:
22+
23+
1. MetalLB or equivalent LoadBalancer provider in the cluster
24+
2. Cluster Logging Operator (CLO) deployed
25+
3. Vector collector pods managed by CLO
26+
4. a running `telemetry-operator` deployed in the cluster
27+
28+
These resources are deployed via the `deps/` directory using kustomize.
29+
30+
### Run the logging tests
31+
32+
Once the kuttl requirements are satisfied, the actual tests can be executed from the telemetry-operator root directory via:
33+
34+
```bash
35+
cd telemetry-operator/
36+
oc kuttl test --test logging --namespace telemetry-kuttl-logging
37+
```
38+
39+
Same tests can be executed from a different location via:
40+
41+
```bash
42+
oc kuttl test --config <KUTTL_CONFIG> <KUTTL_TESTS_DIR> --test logging --namespace telemetry-kuttl-logging
43+
```
44+
45+
> **NOTE**: <KUTTL_CONFIG> is the path where kuttl-test.yaml for the repo is located and <KUTTL_TESTS_DIR> is where the tests are located.

‎test/kuttl/tests/metricstorage/README.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# metricstorage kuttl test
2+
3+
The "metricstorage" kuttl test covers the deployment and configuration of the MetricStorage CR, which manages Prometheus-based metrics collection and storage for OpenStack services.
4+
5+
## Topology
6+
7+
The target topology for this test is:
8+
9+
1. one MetricStorage object managing:
10+
- MonitoringStack (Prometheus deployment)
11+
- ScrapeConfigs for metrics collection
12+
- PrometheusRules for alerting
13+
- Dashboard configurations (optional)
14+
2. Prometheus StatefulSet with persistent storage
15+
3. Integration with Red Hat Observability Stack (RHOBS)
16+
17+
The metricstorage test steps cover:
18+
- Basic deployment with default monitoring stack
19+
- Dashboard configuration management
20+
- Custom monitoring stack integration
21+
- Network attachments for multi-network setups
22+
23+
### Prerequisites
24+
25+
As a prerequisite for this test, we assume:
26+
27+
1. Red Hat Observability Stack (RHOBS) operators installed:
28+
- monitoring.rhobs/v1 API group available
29+
- MonitoringStack CRD
30+
- ScrapeConfig CRD
31+
- PrometheusRule CRD
32+
2. Storage provisioner for persistent volumes
33+
3. OpenShift infrastructure for dashboard integration (optional)
34+
4. a running `telemetry-operator` deployed in the cluster
35+
36+
These resources are deployed via the `deps/` directory using kustomize.
37+
38+
### Run the metricstorage tests
39+
40+
Once the kuttl requirements are satisfied, the actual tests can be executed from the telemetry-operator root directory via:
41+
42+
```bash
43+
cd telemetry-operator/
44+
oc kuttl test --test metricstorage --namespace telemetry-kuttl-metricstorage
45+
```
46+
47+
Same tests can be executed from a different location via:
48+
49+
```bash
50+
oc kuttl test --config <KUTTL_CONFIG> <KUTTL_TESTS_DIR> --test metricstorage --namespace telemetry-kuttl-metricstorage
51+
```
52+
53+
> **NOTE**: <KUTTL_CONFIG> is the path where kuttl-test.yaml for the repo is located and <KUTTL_TESTS_DIR> is where the tests are located.

0 commit comments

Comments
 (0)