Skip to content

Commit a9f87ac

Browse files
evallespclaude
evallesp
and
claude
committed
[os_must_gather] Add exit rc to os-must-gather main task
With this we allow to run the rescue block. Previously any timeout in the main task would return the code from the echo which was always success. Also, we're removing from the rescue block the always block. And we've create a always section for finding existing os-must-gather directories and the symlink creation. Also we've changed the dest-dir for the generic fallback command to match the same folder of the symlink. Removed oc inspect as we're not getting so many errors from oc adm must-gather so probably this wouldn't be usefull. Finally, we've parametriced SOS_EDPM as cifmw_os_must_gather_sos_edpm and we've added default value to "all" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Enrique Vallespi Gil <evallesp@redhat.com>
1 parent 1636490 commit a9f87ac

11 files changed

Lines changed: 458 additions & 50 deletions

File tree

‎roles/os_must_gather/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ testing the new changes.
1212
* `cifmw_os_must_gather_output_dir`: (String) Directory to store logs generated by must-gather tool
1313
* `cifmw_os_must_gather_repo_path`: (string) Path to local clone of openstack-must-gather git repo
1414
* `cifmw_os_must_gather_timeout`: (String) Timeout for must-gather command
15+
* `cifmw_os_must_gather_sos_edpm`: (String) Indicates where to run the SOS report. Default all
1516
* `cifmw_os_must_gather_host_network`: (Bool) Flag to gather host network data
1617
* `cifmw_os_must_gather_namespaces`: (List) List of namespaces required by the gather task in case of failure
1718
* `cifmw_os_must_gather_additional_namespaces`: (String) List of comma separated additional namespaces. Defaults to `kuttl,openshift-storage,sushy-emulator`

‎roles/os_must_gather/defaults/main.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ cifmw_os_must_gather_image_registry: "quay.rdoproject.org/openstack-k8s-operator
2323
cifmw_os_must_gather_output_dir: "{{ cifmw_basedir }}"
2424
cifmw_os_must_gather_output_log_dir: "{{ cifmw_os_must_gather_output_dir }}/logs/openstack-must-gather"
2525
cifmw_os_must_gather_repo_path: "{{ ansible_user_dir }}/src/github.com/openstack-k8s-operators/openstack-must-gather"
26+
cifmw_os_must_gather_sos_edpm: "all"
2627
cifmw_os_must_gather_timeout: "30m"
2728
cifmw_os_must_gather_volume_percentage: 80
2829
cifmw_os_must_gather_additional_namespaces: "kuttl,openshift-storage,openshift-marketplace,openshift-operators,sushy-emulator,tobiko"

‎roles/os_must_gather/molecule/inspect_namespace/converge.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
18+
- name: Converge - Test inspect_namespace fallback handling
19+
hosts: all
20+
vars:
21+
cifmw_path: "{{ ansible_user_dir }}/mock-bin:{{ ansible_env.PATH }}"
22+
cifmw_openshift_kubeconfig: "{{ ansible_user_dir }}/fake-kubeconfig"
23+
cifmw_os_must_gather_timeout: "30m"
24+
cifmw_basedir: "{{ ansible_user_dir }}/test-output"
25+
cifmw_os_must_gather_output_dir: "{{ cifmw_basedir }}"
26+
cifmw_os_must_gather_output_log_dir: "{{ cifmw_os_must_gather_output_dir }}/logs/openstack-must-gather"
27+
cifmw_os_must_gather_image: "fake-image:latest"
28+
cifmw_os_must_gather_namespaces:
29+
- openstack
30+
- openshift-operators
31+
cifmw_os_must_gather_additional_namespaces: "kuttl,openshift-storage"
32+
zuul_change_list: []
33+
tasks:
34+
- name: Run os_must_gather role (expect non-timeout failure with inspect fallback)
35+
block:
36+
- name: Include os_must_gather role
37+
ansible.builtin.include_role:
38+
name: os_must_gather
39+
rescue:
40+
- name: Note that failure occurred
41+
ansible.builtin.debug:
42+
msg: "Role failed as expected due to simulated must-gather failure (rc=1)"

‎roles/os_must_gather/molecule/inspect_namespace/molecule.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
# Mainly used to override the defaults set in .config/molecule/
3+
# By default, it uses the "config_podman.yml" - locally, it will use
4+
# "config_local.yml" in CI.
5+
dependency:
6+
name: galaxy
7+
options:
8+
requirements-file: ../../../../requirements.yml
9+
10+
log: true
11+
12+
provisioner:
13+
name: ansible
14+
log: true

‎roles/os_must_gather/molecule/inspect_namespace/prepare.yml‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
18+
- name: Prepare inspect_namespace test environment
19+
hosts: all
20+
tasks:
21+
- name: Create mock bin directory
22+
ansible.builtin.file:
23+
path: "{{ ansible_user_dir }}/mock-bin"
24+
state: directory
25+
mode: '0755'
26+
27+
- name: Create mock oc command that simulates must-gather failure
28+
ansible.builtin.copy:
29+
dest: "{{ ansible_user_dir }}/mock-bin/oc"
30+
mode: '0755'
31+
content: |
32+
#!/bin/bash
33+
# Mock oc command for testing inspect_namespace fallback
34+
35+
# Check if this is the must-gather command (which should fail)
36+
if [[ "$*" == *"must-gather"* ]] && [[ "$*" != *"adm inspect"* ]]; then
37+
echo "Mock oc adm must-gather starting..."
38+
echo "Command: $@"
39+
echo "Error: Image pull failed" >&2
40+
# Exit with a non-timeout error code
41+
exit 1
42+
fi
43+
44+
# Check if this is the oc adm inspect command (fallback)
45+
if [[ "$*" == *"adm inspect"* ]]; then
46+
echo "Mock oc adm inspect running..."
47+
echo "Command: $@"
48+
# Create a simple output to verify this was called
49+
dest_dir=$(echo "$@" | grep -oP '(?<=--dest-dir=)\S+')
50+
if [ -n "$dest_dir" ]; then
51+
mkdir -p "$dest_dir"
52+
echo "Namespace inspection completed" > "$dest_dir/inspect.log"
53+
fi
54+
exit 0
55+
fi
56+
57+
# For any other oc command, just succeed
58+
echo "Mock oc command: $@"
59+
exit 0
60+
61+
- name: Create fake kubeconfig
62+
ansible.builtin.copy:
63+
dest: "{{ ansible_user_dir }}/fake-kubeconfig"
64+
mode: '0644'
65+
content: |
66+
apiVersion: v1
67+
kind: Config
68+
current-context: fake
69+
70+
- name: Create output directories
71+
ansible.builtin.file:
72+
path: "{{ item }}"
73+
state: directory
74+
mode: '0755'
75+
loop:
76+
- "{{ ansible_user_dir }}/test-output"
77+
- "{{ ansible_user_dir }}/test-output/artifacts"
78+
- "{{ ansible_user_dir }}/test-output/logs/openstack-must-gather"

‎roles/os_must_gather/molecule/inspect_namespace/verify.yml‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
18+
- name: Verify inspect_namespace fallback handling
19+
hosts: all
20+
tasks:
21+
- name: Check that logs directory exists
22+
ansible.builtin.stat:
23+
path: "{{ ansible_user_dir }}/test-output/logs"
24+
register: logs_dir
25+
26+
- name: Assert logs directory was created
27+
ansible.builtin.assert:
28+
that:
29+
- logs_dir.stat.exists
30+
- logs_dir.stat.isdir
31+
fail_msg: "Logs directory not found"
32+
success_msg: "Logs directory exists"
33+
34+
- name: Find must-gather script log
35+
ansible.builtin.find:
36+
paths: "{{ ansible_user_dir }}/test-output/logs"
37+
patterns: "ci_script_*_run_openstack_must_gather.log"
38+
register: must_gather_logs
39+
40+
- name: Assert must-gather log was created
41+
ansible.builtin.assert:
42+
that:
43+
- must_gather_logs.matched > 0
44+
fail_msg: "Must-gather log file not found"
45+
success_msg: "Must-gather log file exists"
46+
47+
- name: Read must-gather log
48+
ansible.builtin.slurp:
49+
src: "{{ must_gather_logs.files[0].path }}"
50+
register: log_content
51+
when: must_gather_logs.matched > 0
52+
53+
- name: Verify must-gather failed with non-timeout error
54+
ansible.builtin.assert:
55+
that:
56+
- "'Error: Image pull failed' in (log_content.content | b64decode)"
57+
when: must_gather_logs.matched > 0
58+
59+
- name: Check that oc_inspect directory was created
60+
ansible.builtin.stat:
61+
path: "{{ ansible_user_dir }}/test-output/logs/oc_inspect"
62+
register: oc_inspect_dir
63+
64+
- name: Assert oc_inspect directory exists
65+
ansible.builtin.assert:
66+
that:
67+
- oc_inspect_dir.stat.exists
68+
- oc_inspect_dir.stat.isdir
69+
fail_msg: "oc_inspect directory not found - fallback didn't run"
70+
success_msg: "oc_inspect directory exists - fallback was triggered"
71+
72+
- name: Find inspect script logs
73+
ansible.builtin.find:
74+
paths: "{{ ansible_user_dir }}/test-output/logs"
75+
patterns: "ci_script_*_inspect_namespaces_as_fallback*.log"
76+
register: inspect_logs
77+
78+
- name: Verify inspect commands were executed
79+
ansible.builtin.assert:
80+
that:
81+
- inspect_logs.matched > 0
82+
fail_msg: "No inspect logs found - oc adm inspect fallback didn't run"
83+
success_msg: "Inspect logs found - oc adm inspect fallback executed"

‎roles/os_must_gather/molecule/timeout/converge.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
18+
- name: Converge - Test timeout handling
19+
hosts: all
20+
vars:
21+
cifmw_path: "{{ ansible_user_dir }}/mock-bin:{{ ansible_env.PATH }}"
22+
cifmw_openshift_kubeconfig: "{{ ansible_user_dir }}/fake-kubeconfig"
23+
cifmw_os_must_gather_timeout: "5s"
24+
cifmw_basedir: "{{ ansible_user_dir }}/test-output"
25+
cifmw_os_must_gather_output_dir: "{{ cifmw_basedir }}"
26+
cifmw_os_must_gather_output_log_dir: "{{ cifmw_os_must_gather_output_dir }}/logs/openstack-must-gather"
27+
cifmw_os_must_gather_image: "fake-image:latest"
28+
zuul_change_list: []
29+
tasks:
30+
- name: Run os_must_gather role (expect timeout failures)
31+
block:
32+
- name: Include os_must_gather role
33+
ansible.builtin.include_role:
34+
name: os_must_gather
35+
rescue:
36+
- name: Note that failure occurred
37+
ansible.builtin.debug:
38+
msg: "Role failed as expected due to simulated timeout (rc=124)"

‎roles/os_must_gather/molecule/timeout/molecule.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
# Mainly used to override the defaults set in .config/molecule/
3+
# By default, it uses the "config_podman.yml" - locally, it will use
4+
# "config_local.yml" in CI.
5+
dependency:
6+
name: galaxy
7+
options:
8+
requirements-file: ../../../../requirements.yml
9+
10+
log: true
11+
12+
provisioner:
13+
name: ansible
14+
log: true

‎roles/os_must_gather/molecule/timeout/prepare.yml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
---
2+
# Copyright Red Hat, Inc.
3+
# All Rights Reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License"); you may
6+
# not use this file except in compliance with the License. You may obtain
7+
# a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13+
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14+
# License for the specific language governing permissions and limitations
15+
# under the License.
16+
17+
18+
- name: Prepare timeout test environment
19+
hosts: all
20+
tasks:
21+
- name: Create mock bin directory
22+
ansible.builtin.file:
23+
path: "{{ ansible_user_dir }}/mock-bin"
24+
state: directory
25+
mode: '0755'
26+
27+
- name: Create mock oc command that simulates timeout
28+
ansible.builtin.copy:
29+
dest: "{{ ansible_user_dir }}/mock-bin/oc"
30+
mode: '0755'
31+
content: |
32+
#!/bin/bash
33+
echo "Mock oc adm must-gather starting..."
34+
echo "Command: $@"
35+
36+
# Sleep to simulate some work being done
37+
sleep 2
38+
39+
# Simulate timeout by exiting with code 124 (timeout command's exit code)
40+
echo "Simulating timeout..."
41+
exit 124
42+
43+
- name: Create fake kubeconfig
44+
ansible.builtin.copy:
45+
dest: "{{ ansible_user_dir }}/fake-kubeconfig"
46+
mode: '0644'
47+
content: |
48+
apiVersion: v1
49+
kind: Config
50+
current-context: fake
51+
52+
- name: Create output directories
53+
ansible.builtin.file:
54+
path: "{{ item }}"
55+
state: directory
56+
mode: '0755'
57+
loop:
58+
- "{{ ansible_user_dir }}/test-output"
59+
- "{{ ansible_user_dir }}/test-output/artifacts"
60+
- "{{ ansible_user_dir }}/test-output/logs/openstack-must-gather"

0 commit comments

Comments
 (0)