[rally] Add Rally role and test_operator integration

Add a new `rally` Ansible role that runs OpenStack Rally benchmarks
inside a podman container (quay.io/airshipit/xrally-openstack:3.0.0).
The role auto-discovers OpenStack credentials from the cluster's
KeystoneAPI resource and appends the deployment CA to the trust bundle
so SSL verification works without any manual configuration.

Wire Rally into the `test_operator` multi-stage framework via a new
`rally_runner.yml` dispatcher and `cifmw_test_operator_rally_*`
defaults, so it can run as a named stage alongside Tempest.

Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: lkuchlan <lkuchlan@redhat.com>

Author: lkuchlan

hooks/playbooks/rally_prerequisites.yaml

@@ -0,0 +1,16 @@
+---
+- name: Ensure Rally prerequisites exist
+  hosts: "{{ cifmw_target_hook_host | default('localhost') }}"
+  gather_facts: false
+  environment:
+    KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
+    PATH: "{{ cifmw_path }}"
+  tasks:
+    - name: Create m1.tiny flavor if missing
+      ansible.builtin.shell: |
+        set -xe -o pipefail
+        oc -n {{ namespace }} rsh openstackclient \
+            openstack flavor show m1.tiny &>/dev/null || \
+            oc -n {{ namespace }} rsh openstackclient \
+            openstack flavor create m1.tiny \
+              --ram 512 --disk 1 --vcpus 1 --public

roles/rally/OWNERS

@@ -0,0 +1,7 @@
+# See the OWNERS docs at https://www.kubernetes.dev/docs/guide/owners/
+
+approvers:
+  - ciops-team
+
+reviewers:
+  - ciops-team

roles/rally/README.md

@@ -0,0 +1,80 @@
+# rally
+Role to setup and run Rally benchmarking tests against an OpenStack deployment.
+
+Rally is run inside the `quay.io/airshipit/xrally-openstack` container via podman.
+OpenStack credentials are discovered automatically from the Kubernetes KeystoneAPI resource,
+or can be supplied directly via `cifmw_rally_os_*` variables.
+
+## Prerequisites
+- `oc` CLI available and pointing to the target OpenStack cluster (used for credential and CA discovery).
+- `cifmw_openstack_namespace` must be set to the OpenStack namespace (typically `openstack`).
+
+## Privilege escalation
+become - Required to install podman and fix artifact directory ownership after the container run.
+
+## Parameters
+
+* `cifmw_rally_artifacts_basedir`: (String) Directory where all Rally artifacts are stored.
+  Default: `{{ cifmw_basedir }}/tests/rally`
+* `cifmw_rally_registry`: (String) Container registry. Default: `quay.io`
+* `cifmw_rally_namespace`: (String) Registry namespace. Default: `airshipit`
+* `cifmw_rally_container`: (String) Container image name. Default: `xrally-openstack`
+* `cifmw_rally_image`: (String) Full image reference. Composed from registry/namespace/container.
+* `cifmw_rally_image_tag`: (String) Container image tag. Default: `3.0.0`
+* `cifmw_rally_dry_run`: (Boolean) Skip actual container execution. Default: `false`
+* `cifmw_rally_remove_container`: (Boolean) Remove container after run. Default: `true`
+* `cifmw_rally_fail_on_task_failure`: (Boolean) Fail the play if Rally exits non-zero. Default: `true`
+* `cifmw_rally_deployment_name`: (String) Rally deployment name. Default: `cifmw`
+* `cifmw_rally_concurrency`: (Integer) Concurrency passed to the Rally task via `--task-args`. Default: `1`
+* `cifmw_rally_task_file`: (String) Filename of the Rally task YAML inside the tasks directory.
+  Must exist in the role's `files/` directory or be pre-staged in `cifmw_rally_artifacts_basedir/tasks/`
+  via a pre-test hook. Default: `default-task.yaml`
+* `cifmw_rally_task_extra_args`: (String) Extra arguments passed verbatim to `rally task start`. Default: `""`
+* `cifmw_rally_dns_servers`: (List) DNS servers used inside the Rally container. Default: `["192.168.122.10"]`
+* `cifmw_rally_os_auth_url`: (String) OpenStack auth URL. Auto-discovered from KeystoneAPI if unset.
+* `cifmw_rally_os_username`: (String) OpenStack username. Default: `admin`
+* `cifmw_rally_os_password`: (String) OpenStack password. Auto-discovered from Kubernetes secret if unset.
+* `cifmw_rally_os_project_name`: (String) OpenStack project. Default: `admin`
+* `cifmw_rally_os_user_domain_name`: (String) User domain. Default: `Default`
+* `cifmw_rally_os_project_domain_name`: (String) Project domain. Default: `Default`
+* `cifmw_rally_os_region_name`: (String) OpenStack region. Default: `regionOne`
+
+## Standalone usage
+
+```yaml
+- hosts: hypervisor
+  roles:
+    - role: rally
+      vars:
+        cifmw_rally_concurrency: 2
+        cifmw_rally_task_file: my-benchmark.yaml
+        cifmw_rally_fail_on_task_failure: false
+```
+
+Set `cifmw_run_test_role: rally` in a scenario `05-tests.yaml` to invoke this role
+from the cifmw_setup workflow without using the test_operator.
+
+## Usage as a test_operator stage
+
+Add a `rally` stage to `cifmw_test_operator_stages` in your scenario's `05-tests.yaml`:
+
+```yaml
+cifmw_run_tests: true
+cifmw_run_test_role: test_operator
+cifmw_test_operator_stages:
+  - name: tempest
+    type: tempest
+  - name: rally
+    type: rally
+    test_vars:
+      cifmw_test_operator_rally_concurrency: 2
+      cifmw_test_operator_rally_fail_on_task_failure: false
+```
+
+## Custom task files
+
+To use a custom Rally task file, either:
+
+1. Add it to the role's `files/` directory and set `cifmw_rally_task_file` to its filename.
+2. Pre-stage it in `cifmw_rally_artifacts_basedir/tasks/` via a `pre_test_stage_hooks` hook
+   and set `cifmw_rally_task_file` to the filename.

roles/rally/defaults/main.yml

@@ -0,0 +1,35 @@
+---
+# Copyright Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+# All variables intended for modification should be placed in this file.
+# All variables within this role should have a prefix of "cifmw_rally"
+cifmw_rally_artifacts_basedir: "{{ cifmw_basedir }}/tests/rally"
+cifmw_rally_registry: "quay.io"
+cifmw_rally_namespace: "airshipit"
+cifmw_rally_container: "xrally-openstack"
+cifmw_rally_image: "{{ cifmw_rally_registry }}/{{ cifmw_rally_namespace }}/{{ cifmw_rally_container }}"
+cifmw_rally_image_tag: "3.0.0"
+cifmw_rally_dry_run: false
+cifmw_rally_remove_container: true
+cifmw_rally_fail_on_task_failure: true
+cifmw_rally_deployment_name: "cifmw"
+cifmw_rally_concurrency: 1
+cifmw_rally_task_file: ""
+cifmw_rally_openstack_tests: ""
+cifmw_rally_task_extra_args: ""
+cifmw_rally_dns_servers:
+  - "192.168.122.10"

roles/rally/meta/main.yml

@@ -0,0 +1,29 @@
+---
+# Copyright Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+galaxy_info:
+  author: CI Framework
+  description: CI Framework Role -- rally
+  company: Red Hat
+  license: Apache-2.0
+  min_ansible_version: "2.14"
+  namespace: cifmw
+  galaxy_tags:
+    - cifmw
+    - rally
+
+dependencies: []

roles/rally/molecule/default/converge.yml

@@ -0,0 +1,23 @@
+---
+# Copyright Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+- name: Converge
+  hosts: all
+  vars:
+    cifmw_rally_dry_run: true
+  roles:
+    - role: "rally"

roles/rally/molecule/default/molecule.yml

@@ -0,0 +1,9 @@
+---
+# Mainly used to override the defaults set in .config/molecule/
+# By default, it uses the "config_podman.yml" - in CI, it will use
+# "config_local.yml".
+log: true
+
+provisioner:
+  name: ansible
+  log: true

roles/rally/molecule/default/prepare.yml

@@ -0,0 +1,28 @@
+---
+# Copyright Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+- name: Prepare
+  hosts: all
+  vars:
+    cifmw_basedir: "{{ ansible_user_dir }}/ci-framework-data"
+    cifmw_install_yamls_tasks_out: "{{ ansible_user_dir }}/zuul-jobs/roles/install_yamls_makes/tasks"
+    cifmw_install_yamls_defaults:
+      NAMESPACE: openstack
+  roles:
+    - role: test_deps
+    - role: ci_setup
+    - role: install_yamls

roles/rally/tasks/create-deployment.yml

@@ -0,0 +1,55 @@
+---
+# Copyright Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+- name: Discover credentials from Kubernetes when not overridden
+  when: cifmw_rally_os_auth_url is not defined
+  block:
+    - name: Get keystone data
+      register: _cifmw_rally_keystone_data
+      environment:
+        KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
+        PATH: "{{ cifmw_path }}"
+      ansible.builtin.command:
+        cmd: "oc get keystoneapi keystone -n {{ cifmw_openstack_namespace }} -o json"
+
+    - name: Set keystone vars
+      vars:
+        _keystone_json: "{{ _cifmw_rally_keystone_data.stdout | from_json }}"
+      ansible.builtin.set_fact:
+        _cifmw_rally_keystone_secret_name: "{{ _keystone_json['spec']['secret'] }}"
+        _cifmw_rally_keystone_passwd_select: "{{ _keystone_json['spec']['passwordSelectors']['admin'] }}"
+        _cifmw_rally_keystone_api: "{{ _keystone_json['status']['apiEndpoints']['public'] }}"
+
+    - name: Get credentials data
+      register: _cifmw_rally_os_password_data
+      environment:
+        KUBECONFIG: "{{ cifmw_openshift_kubeconfig }}"
+        PATH: "{{ cifmw_path }}"
+      ansible.builtin.command:
+        cmd: >-
+          oc get secret {{ _cifmw_rally_keystone_secret_name }}
+          -n {{ cifmw_openstack_namespace }} -o json
+
+    - name: Set OpenStack credentials facts
+      vars:
+        _pw_json: "{{ _cifmw_rally_os_password_data.stdout | from_json }}"
+      ansible.builtin.set_fact:
+        cifmw_rally_os_auth_url: "{{ _cifmw_rally_keystone_api }}"
+        cifmw_rally_os_password: >-
+          {{
+            _pw_json['data'][_cifmw_rally_keystone_passwd_select] |
+            ansible.builtin.b64decode
+          }}