ansible: prepare runner provisioning for Ubuntu bare-metal hosts (#223)

Add support for provisioning Ubuntu 24.04 (Noble) bare-metal AWS hosts as
runners, alongside the existing Amazon Linux 2 (RedHat family) and s390x
LinuxONE (Debian family) hosts, without changing behavior for either.

- base: start (and enable) the docker service on Debian hosts, mirroring
  the RedHat path. On Ubuntu cloud-init first boot docker.io may not be up
  yet when qemu-user-static / the runner units need it; idempotent on s390x.
- base/runner: make the apt installs resilient to first-boot dpkg-lock
  contention from apt-daily/unattended-upgrades (lock_timeout + retries).
- runner: derive runner names from the EC2 instance ID on all EC2 hosts
  (system_vendor == "Amazon EC2"), not just Amazon Linux, so Ubuntu EC2
  hosts get stable, collision-free names. Non-EC2 hosts (s390x) still skip
  the metadata lookup; the retained Amazon-distribution clause keeps AL2
  behavior identical.
- add ansible/requirements.yml (community.docker, amazon.aws) and document
  it, for the manual operator workflow / ansible-core installs.
- fix stale runner_libbpf_ci_repo_branch defaults (master -> main).

Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>

Author: theihor

ansible/README.md

@@ -5,6 +5,17 @@ sudo dnf install -y ansible
 ```
 
 
+## Required collections
+
+The playbook uses modules from `community.docker` (`docker_login`) and `amazon.aws`
+(`ec2_metadata_facts`). The `ansible` metapackage bundles these; if you installed
+`ansible-core` only, install them explicitly:
+
+```
+ansible-galaxy collection install -r ansible/requirements.yml
+```
+
+
 ## Inventory
 
 The [inventory](https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html) is where we define our hosts, hostgroup, possibly variable...

ansible/inventory_example.yml

@@ -61,7 +61,7 @@ all:
   vars:
     ansible_ssh_common_args: -o 'ProxyCommand ......'
     runner_libbpf_ci_repo_url: https://github.com/libbpf/ci
-    runner_libbpf_ci_repo_branch: master
+    runner_libbpf_ci_repo_branch: main
     runner_gh_apps:
     - name: kernel-patches-runner
       id: a-github-app-id

ansible/requirements.yml

@@ -0,0 +1,4 @@
+---
+collections:
+  - name: community.docker
+  - name: amazon.aws

ansible/roles/base/tasks/setup-Debian.yml

@@ -5,8 +5,20 @@
     state: present
     name: "{{ base_packages }}"
     update_cache: yes
+    lock_timeout: 300
+  register: base_packages_install
+  until: base_packages_install is succeeded
+  retries: 30
+  delay: 10
   tags: [install]
 
+- name: Start docker
+  become: true
+  service:
+    name: docker
+    state: started
+    enabled: true
+
 - name: Gather the package facts
   ansible.builtin.package_facts:
 

ansible/roles/runner/defaults/main.yml

@@ -15,7 +15,7 @@ runner_docker_mount_volume: false
 runner_gh_tokens:
   owner/repository: gh token for owner/repository
 runner_libbpf_ci_repo_url: https://github.com/libbpf/ci.git
-runner_libbpf_ci_repo_branch: master
+runner_libbpf_ci_repo_branch: main
 runner_repo_list:
   - {name: owner1/repository1, instances: 2}
   - {name: owner1/repository2, instances: 1}

ansible/roles/runner/tasks/main.yml

@@ -18,6 +18,11 @@
     state: present
     name: python3-docker
     update_cache: yes
+    lock_timeout: 300
+  register: python3_docker_install
+  until: python3_docker_install is succeeded
+  retries: 30
+  delay: 10
   when: ansible_os_family == 'Debian'
 
 - name: Create runner directory
@@ -75,18 +80,20 @@
   set_fact:
     runner_name_prefix: "{{ '%s-' | format(runner_prefix) if runner_prefix }}{{ ansible_hostname }}"
 
-# When running on Amazon Linux hosts, we override the runner_name_prefix with the ec2's instance ID.
-# When testing in Amazon Linux VMs, the `Load ec2 metadata facts` task will fail and we will fallback
-# on using hostname.
-- name: Set runner_name_prefix to instance ID for Amazon hosts
+# On EC2 hosts (Amazon Linux metal and Ubuntu metal alike) we override the runner_name_prefix
+# with the ec2's instance ID for stable, collision-free runner names. Non-EC2 hosts (e.g. s390x
+# LinuxONE, vendor "IBM") are skipped so we never block on the 169.254.169.254 metadata endpoint.
+# If amazon.aws is missing or the metadata endpoint is unreachable, `ignore_errors` lets us fall
+# back to the hostname-based prefix set above.
+- name: Set runner_name_prefix to instance ID for EC2 hosts
   block:
     - name: Load ec2 metadata facts
       amazon.aws.ec2_metadata_facts:
 
     - name: Set runner name prefix with instance ID
       set_fact:
         runner_name_prefix: "{{ '%s-' | format(runner_prefix) if runner_prefix }}{{ ansible_ec2_instance_id }}"
-  when: ansible_distribution == 'Amazon'
+  when: ansible_system_vendor == 'Amazon EC2' or ansible_distribution == 'Amazon'
   ignore_errors: yes
 
 - name: Generate runner env