PoC: triggering of maintenance tests for openSUSE by foursixnine · Pull Request #473 · os-autoinst/os-autoinst-scripts

foursixnine · 2025-10-09T09:24:08Z

Ticket: https://progress.opensuse.org/issues/190158

This script is only a proof of concept based of what I understood of what openqabot does.

I basically started off from #468 for some boilerplate and ended up with this script. So aside from requiring GITEA_TOKEN as environment variable, it also requires osc credentials for the given build service api (--bs, defaults to api.opensuse.org)

To call it:

python3 manual-maintenance-tests-trigger.py --project products/PackageHub --branch leap-16.0 --pr-id 158

Which results in the following openqa-cli call:

openqa-cli schedule --host http://localhost:9526 PRIO=100 \
 CI_TARGET_URL=http://localhost:9526 \
 GITEA_REPO=products/PackageHub \
 GITEA_SHA=f95b1c7c0e005e5a9db095f430fdfbde147d9198797284e066a05e8aaf9f4af2 \
 GITEA_STATUSES_URL=https://src.opensuse.org/api/v1/repos/products/PackageHub/statuses/f95b1c7c0e005e5a9db095f430fdfbde147d9198797284e066a05e8aaf9f4af2 \
 GITEA_PR_URL=https://src.opensuse.org/products/PackageHub/pulls/158 \
 webhook_id=gitea:pr:158 \
 VERSION=16.0 \
 DISTRI=opensuse \
 FLAVOR=staged-updates \
 ARCH=x86_64 \
 BUILD=:158:chromium \
 INCIDENT_REPO=http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-16.0:/PullRequest:/158 \
 INCIDENT_PATCH=158 \
 IMPORT_GPG_KEYS=gpg-pubkey-b3fd7e48-5549fd0f \
 ZYPPER_ADD_REPO_PREFIX=staged-updates \
 INSTALL_PACKAGES=chromedriver chromedriver-debuginfo chromium chromium-debuginfo \
 VERIFY_PACKAGE_VERSIONS=chromedriver 141.0.7390.65-bp160.1.1 chromedriver-debuginfo 141.0.7390.65-bp160.1.1 chromium 141.0.7390.65-bp160.1.1 chromium-debuginfo 141.0.7390.65-bp160.1.1

This is mostly the flow for incident updates (or staged updates), I don't quite get yet how the aggregated updates are handled.

There's also a hardcoded architecture, but that shouldn't be a blocker at this stage, I tried this on my local openQA instance with the following job-group:

defaults:
  x86_64:
    machine: uefi-3G
    priority: 50
products:
  opensuse-staged-updates-x86_64:
    distri: opensuse
    flavor: staged-updates
    version: '16.0'
scenarios:
  x86_64:
    opensuse-staged-updates-x86_64:
      - gnome-agama

Machine and test settings were taken from openqa.opensuse.org

PR data can be passed with --pr-data tests/data/opensuse-maintenance/by-autogits_workflow_pr_bot-151-20251007-154142.json, there's a switch to store the pr data too.

We need to query the source packages of the update to be able to figure out the name of the given staged update, for now we're sticking to the promise that staged updates are single units

Avoid multiple nested ifs, to the code more readable

This is mainly to avoid collisions later on

Build parameter was missing, so query would return all jobs for given parameters

Vogtinator · 2025-10-16T11:03:40Z

+from collections import namedtuple
+import osc.core
+
+USER_AGENT = "manual-trigger.py (https://github.com/os-autoinst/scripts)"


git-openqa-maintenance.py?

Martchus

There are probably still many improvements to be made but it looks like this would generally work.

This is indeed very similar to what I have already created in qem-bot except that no external dashboard is involved. So this is now kind of a mix between qem-bot gitea-sync and qem-bot increment-approve. (It would be nice if we wouldn't have to duplicate our effort here but I also don't see an easy way of streamlining this. In theory qem-bot could be used for openSUSE as well, though.)

It would be nice to document the usage with some examples in the README.

Martchus · 2025-10-27T16:47:13Z

+dry_run = True
+openqa_dry_run = False


It is a bit confusing that dry_run is sometimes shadowed by local variables. Both variables should probably be CLI arguments. Considering you're doing proper CLI argument parsing anyway this shouldn't be a big deal.

Martchus · 2025-10-27T16:54:21Z

+        "scope": "relevant",
+        "latest": "1",
+    }
+    jobs = openqa.openqa_request("GET", "jobs", values)["jobs"]


I guess this is good enough for now. I'd just like to note that there's a route which gets you all jobs of the latest scheduled product matching the parameters you're using here as well: https://github.com/os-autoinst/openQA/blob/345b0552e13c159d5b31d8403813125b29332929/lib/OpenQA/WebAPI/Controller/API/V1/Iso.pm#L61

With this route you could for instance drop the logic to skip clones and it would only check jobs scheduled via the relevant scheduled products.

I created this route for doing the same what you're doing here in qem-bot so you can find Python code using it in https://github.com/openSUSE/qem-bot/blob/master/openqabot/incrementapprover.py. (This code also has unit tests.)

Martchus · 2025-10-27T17:01:30Z

+        else:
+            packages.append(package)
+
+    # In theory every staged update, has a single package


If that's also true in practice then this is different from what I have been dealing with for SLFO/SLE16 on the internal Gitea/IBS.

My theory was wrong :), https://src.opensuse.org/products/PackageHub/pulls/192 has two packages for instance.

This part now computes the shortest package name: https://github.com/foursixnine/openSUSE-release-tools/blob/92f405972f5cba472c3c7e3efb85a82a6b0c44df/git-openqa-maintenance.py#L287

Martchus · 2025-10-27T17:09:46Z

+
+def get_open_prs_for_project_branch(project, branch):
+    pull_requests_url = (
+        GITEA_HOST + f"/api/v1/repos/{project}/pulls?state=open&base_branch={branch}"


When I remember correctly, this will only consider the first 50 PRs or so. You could steal the while loop from https://github.com/openSUSE/qem-bot/blob/44a89a2bb5fe46167169759804a59de3bfc8eb89/openqabot/loader/gitea.py#L138 for paging. (If there are tons of open PRs this won't be very fast, though.)

(Maybe I can make use of the base_branch search parameter in qem-bot as well.)

okurz · 2025-10-28T07:44:38Z

This is indeed very similar to what I have already created in qem-bot except that no external dashboard is involved. So this is now kind of a mix between qem-bot gitea-sync and qem-bot increment-approve. (It would be nice if we wouldn't have to duplicate our effort here but I also don't see an easy way of streamlining this. In theory qem-bot could be used for openSUSE as well, though.)

That's why we have https://progress.opensuse.org/issues/189912 planned. Meanwhile I would recommend to incorporate the necessary functionality within qem-bot to replace this implementation here

foursixnine · 2025-10-28T09:01:50Z

There are probably still many improvements to be made but it looks like this would generally work.

This is indeed very similar to what I have already created in qem-bot except that no external dashboard is involved.

Yeah

So this is now kind of a mix between qem-bot gitea-sync and qem-bot increment-approve. (It would be nice if we wouldn't have to duplicate our effort here but I also don't see an easy way of streamlining this. In theory qem-bot could be used for openSUSE as well, though.)

That is something I would like too, having three different bots doesn't sound very sustainable

In the meantime I extracted this, and created openSUSE/openSUSE-release-tools#3271 which handles the full workflow (an example is the PR I already linked above https://src.opensuse.org/products/PackageHub/pulls/192)

It would be nice to document the usage with some examples in the README.

Martchus · 2025-10-28T09:38:54Z

Ok, good to know that there's a more up-to-date version of this. Then I'll try it out and update my findings on https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Scheduling-maintenance-tests-for-openSUSE-Leap-16.

To avoid confusion I would close this draft in favor of openSUSE/openSUSE-release-tools#3271, or do you still want anything to be merged here?

foursixnine · 2025-10-28T09:56:52Z

To avoid confusion I would close this draft in favor of openSUSE/openSUSE-release-tools#3271, or do you still want anything to be merged here?

No objections, this was never meant to be merged here :)

foursixnine added 25 commits October 8, 2025 13:03

Add script for manual maintenance update scheduling

b944799

Add parameter for repository prefix

0158e91

Prepare settings for the current update

217d66c

Implement get_incident_name

9a6db24

We need to query the source packages of the update to be able to figure out the name of the given staged update, for now we're sticking to the promise that staged updates are single units

Refactor gitea parameters

efe0f49

Remove dead code

b25199c

Refactor openqa test settings generation

a63d3ca

Improve logging messages

e01789d

Add test pull request 151 data

65ef559

Make style checks pass

f85c72e

Remove dead code

e9a1773

Allow querying status of the review

3760270

Add cmdline parameters needed for the execution

527f39c

Remove code for storing PR data

aaa6f76

Add configuration data for other projects

839cc59

Allow processing multiple pull requests in batch

d8794a4

Query openQA build to know overall status of tests

80aaf60

Refactor pr processing

3ce1fd5

Add take_action to handle approval of a PR

586f4a8

Add is_build_finished to query status of build in obs/ibs

72002f9

Minor polishing

7d529ce

Handle updates with patchinfo and multiple source packages

448c402

Allow post requests to gitea to have dry runs

85b308c

Query comments for openqa tests and review status

6dbe64e

Add missing methods to git history

973cb96

foursixnine changed the title ~~PoC: "Manual" triggering of maintenance tests~~ PoC: triggering of maintenance tests for openSUSE Oct 15, 2025

foursixnine added 4 commits October 15, 2025 11:10

Remove old test file

5d2e7d5

Refactor process_pull_request

39ee8d7

Avoid multiple nested ifs, to the code more readable

Make the build parameter more unique

ac3ae3e

This is mainly to avoid collisions later on

Fix query to openQA test results

b373465

Build parameter was missing, so query would return all jobs for given parameters

Rename script

40badf1

foursixnine mentioned this pull request Oct 16, 2025

Add git-openqa-maintenance: Review bot for Leap 16+ openSUSE/openSUSE-release-tools#3271

Merged

Vogtinator reviewed Oct 16, 2025

View reviewed changes

Martchus reviewed Oct 27, 2025

View reviewed changes

foursixnine closed this Oct 28, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PoC: triggering of maintenance tests for openSUSE#473

PoC: triggering of maintenance tests for openSUSE#473
foursixnine wants to merge 30 commits into
os-autoinst:masterfrom
foursixnine:opensuse-manual-maintenance

foursixnine commented Oct 9, 2025 •

edited

Loading

Uh oh!

Vogtinator Oct 16, 2025

Uh oh!

Martchus left a comment •

edited

Loading

Uh oh!

Martchus Oct 27, 2025

Uh oh!

Martchus Oct 27, 2025

Uh oh!

Martchus Oct 27, 2025

Uh oh!

foursixnine Oct 28, 2025 •

edited

Loading

Uh oh!

Martchus Oct 27, 2025

Uh oh!

okurz commented Oct 28, 2025

Uh oh!

foursixnine commented Oct 28, 2025

Uh oh!

Martchus commented Oct 28, 2025 •

edited

Loading

Uh oh!

foursixnine commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		dry_run = True
		openqa_dry_run = False

Conversation

foursixnine commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Vogtinator Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

Martchus left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Martchus Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

Martchus Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

Martchus Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

foursixnine Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Martchus Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

okurz commented Oct 28, 2025

Uh oh!

foursixnine commented Oct 28, 2025

Uh oh!

Martchus commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

foursixnine commented Oct 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

foursixnine commented Oct 9, 2025 •

edited

Loading

Martchus left a comment •

edited

Loading

foursixnine Oct 28, 2025 •

edited

Loading

Martchus commented Oct 28, 2025 •

edited

Loading