feat: Add resource-agents-patch.sh script (#9)

clobrano · fonta-rh · web-flow · commit 0a551a85a676 · 2025-07-31T07:35:53.000Z
* feat: Add resource-agents-patch.sh script

This commit introduces the resource-agents-patch.sh script.

This script installs an RPM package on all OpenShift cluster nodes using
rpm-ostree. It performs the following actions:

- Checks for required tools (oc and jq).
- Verifies that the user is logged into an OpenShift cluster.
- Discovers the IPs of all nodes in the cluster.
- Copies the RPM to each node.
- Installs the RPM using rpm-ostree.
- Provides instructions for rebooting the nodes.

This script simplifies the process of patching resource agents on OpenShift clusters.

* Add Ansible playbook to install resource agents RPM

This change adds an alternative method to update the resource agents on
the target node installing the RPM via playbook.

usage:

$ ansible-playbook -i inventory.ini \
    resource-agents-patch.yml \
    -e "rpm_full_path=/path/to/rpm"

cat inventory.ini
    [all]
    node1 ansible_host=192.168.111.20
    node2 ansible_host=192.168.111.21

* Added README

* Added inventory sample, updated README to clarify

---------

Co-authored-by: Pablo Fontanilla &lt;pfontani@redhat.com&gt;
diff --git a/helpers/README.md b/helpers/README.md
@@ -0,0 +1,57 @@
+# Helpers
+
+Utilities for installing RPM packages on OpenShift cluster nodes using rpm-ostree.
+
+## Description
+
+This directory contains scripts and playbooks for patching OpenShift cluster nodes with RPM packages, specifically designed for resource agent updates. The tools use rpm-ostree's override functionality to replace packages on immutable CoreOS nodes.
+
+## Requirements
+
+### For resource-agents-patch.sh
+- `oc` CLI tool (logged into OpenShift cluster)
+- `jq` for JSON processing
+- SSH access to cluster nodes
+
+### For resource-agents-patch.yml
+- Ansible
+- Inventory file containing OpenShift cluster nodes (separate from hypervisor deployment inventory, see `inventory_ocp_hosts.sample`)
+- SSH access configured for `core` user
+
+## Usage
+
+### Shell Script
+
+```bash
+./resource-agents-patch.sh /path/to/package.rpm
+```
+
+**Process:**
+1. Validates required tools and RPM file
+2. Discovers all node IPs via OpenShift API
+3. Copies RPM to each node using SCP
+4. Installs package with `rpm-ostree override replace`
+5. Provides manual reboot commands
+
+### Ansible Playbook
+
+```bash
+ansible-playbook -i inventory_ocp_hosts resource-agents-patch.yml -e rpm_full_path=/path/to/package.rpm
+```
+
+**Note**: The inventory file should list the OpenShift cluster nodes (VMs), not the hypervisor host. Copy `inventory_ocp_hosts.sample` to `inventory_ocp_hosts` and update with your cluster node IPs.
+
+**Process:**
+1. Validates RPM file existence
+2. Copies RPM to all nodes
+3. Installs using rpm-ostree override
+4. Reboots nodes one at a time
+5. Verifies etcd health after reboot
+
+## Notes
+
+- Both tools use `rpm-ostree override replace` which is appropriate for updating existing packages
+- Node reboots are required to activate rpm-ostree changes
+- The Ansible playbook handles rebooting automatically; the shell script requires manual intervention
+- Plan reboots carefully to maintain cluster availability
+- Monitor cluster health during the patching process 
diff --git a/helpers/inventory_ocp_hosts.sample b/helpers/inventory_ocp_hosts.sample
@@ -0,0 +1,4 @@
+[cluster_nodes]
+core@10.0.0.10 ansible_ssh_extra_args='-o ServerAliveInterval=30 -o ServerAliveCountMax=120'
+core@10.0.0.11 ansible_ssh_extra_args='-o ServerAliveInterval=30 -o ServerAliveCountMax=120'
+core@10.0.0.12 ansible_ssh_extra_args='-o ServerAliveInterval=30 -o ServerAliveCountMax=120' 
diff --git a/helpers/resource-agents-patch.sh b/helpers/resource-agents-patch.sh
@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# -*- coding: UTF-8 -*-
+set -o errexit
+set -o pipefail
+set -o nounset
+
+# Check if required tools are available
+command -v oc >/dev/null 2>&1 || { echo "Error: 'oc' command not found. Please ensure you're logged into an OpenShift cluster." >&2; exit 1; }
+command -v jq >/dev/null 2>&1 || { echo "Error: 'jq' command not found. Please install jq." >&2; exit 1; }
+
+RPM_FULL_PATH="$1"
+PACKAGE=$(basename "$RPM_FULL_PATH")
+
+# Function to show usage
+usage() {
+    echo "Usage: $0 <rpm-file-path>"
+    echo -e "\nThis script installs an RPM package on all OpenShift cluster nodes using rpm-ostree.\n"
+    echo "Arguments:"
+    echo -e "  rpm-file-path    Path to the RPM file to install\n"
+    echo "Example:"
+    echo "  $0 /path/to/package.rpm"
+    exit 1
+}
+
+# Check arguments
+if [[ $# -ne 1 ]]; then
+    usage
+fi
+
+if [[ ! -f "$RPM_FULL_PATH" ]]; then
+    echo "Error: RPM file '$RPM_FULL_PATH' not found." >&2
+    exit 1
+fi
+
+# Get the list of all node IPs
+echo "Discovering OpenShift cluster nodes..."
+mapfile -t NODE_IPS< <(oc get nodes -o json | jq -r '.items[].status.addresses[0].address')
+
+if [[ ${#NODE_IPS[@]} -eq 0 ]]; then
+    echo "Error: No nodes found in the cluster." >&2
+    echo "Make sure you're connected to an OpenShift cluster." >&2
+    exit 1
+fi
+
+echo "Found ${#NODE_IPS[@]} node(s):"
+for ip in "${NODE_IPS[@]}"; do
+    echo "  - $ip"
+done
+
+echo -e "\nInstalling RPM package '$PACKAGE' on ${#NODE_IPS[@]} node(s)...\n"
+
+set -x
+for IP in "${NODE_IPS[@]}"; do
+    echo "Processing node: $IP"
+    scp "$RPM_FULL_PATH" "core@$IP":/var/home/core
+    ssh -t "core@$IP" -- sudo rpm-ostree -C override replace /var/home/core/"$PACKAGE"
+    sleep 2
+done
+
+echo -e "\nRPM installation completed on all nodes."
+echo -e "\nIMPORTANT: The nodes need to be rebooted to apply the rpm-ostree changes."
+echo "Plan the reboots carefully to maintain cluster availability:"
+echo "- Consider rebooting nodes one at a time"
+echo "- Wait for each node to become Ready before rebooting the next"
+echo "- Monitor cluster health during the process"
+echo -e "\nReboot commands (run manually when ready):"
+for IP in "${NODE_IPS[@]}"; do
+    echo "  ssh core@$IP sudo systemctl reboot"
+done
+
diff --git a/helpers/resource-agents-patch.yml b/helpers/resource-agents-patch.yml
@@ -0,0 +1,55 @@
+---
+- name: Install RPM package on OpenShift nodes using rpm-ostree
+  hosts: all
+  become: true
+  remote_user: core
+  tasks:
+    - name: Store RPM full path in a registry
+      stat:
+        path: "{{ rpm_full_path }}"
+      register: rpm_file
+      delegate_to: localhost
+      run_once: true
+
+    - name: Fail if RPM file does not exist
+      fail:
+        msg: "RPM file '{{ rpm_full_path }}' not found."
+      when: not rpm_file.stat.exists
+
+    - name: Copy RPM file to nodes
+      copy:
+        src: "{{ rpm_full_path }}"
+        dest: /var/home/core/
+        owner: core
+        group: core
+      register: copy_result
+
+    - name: Install RPM package using rpm-ostree
+      command: rpm-ostree -C override replace /var/home/core/{{ rpm_package }}
+      register: rpm_ostree_result
+      async: 600  # 10 minutes timeout
+      poll: 30    # check every 30 seconds
+
+    - name: Display rpm-ostree command output
+      debug:
+        msg: |
+          Command: {{ rpm_ostree_result.cmd }}
+          Return code: {{ rpm_ostree_result.rc }}
+          STDOUT: {{ rpm_ostree_result.stdout }}
+          STDERR: {{ rpm_ostree_result.stderr }}
+      when: rpm_ostree_result is defined
+
+    - name: Reboot nodes to apply changes
+      reboot:
+        msg: "Rebooting node to apply rpm-ostree changes"
+        connect_timeout: 5
+        reboot_timeout: 600  # Extended timeout for ostree boot
+        pre_reboot_delay: 10
+        post_reboot_delay: 60  # Wait longer after reboot
+        test_command: podman exec etcd etcdctl member list
+      throttle: 1  # Reboot one node at a time
+
+  vars:
+    rpm_full_path: "{{ rpm_full_path | default(lookup('env', 'RPM_FULL_PATH')) }}"
+    rpm_package: "{{ rpm_full_path | basename }}"
+
