Skip to content

Commit fa4a40b

Browse files
llucaxllucax
authored
Simplify CI job matrix (#511)
We are currently not using the distinction between "os" and "arch", and only using both to construct the `runs-on:` string. This makes things much more complicated than necessary, so this commit updates the matrix to use a single 'platform' key that we can use directly in `runs-on`.
2 parents eae6c3f + 936970e commit fa4a40b

9 files changed

Lines changed: 96 additions & 70 deletions

File tree

  • .github/workflows
  • RELEASE_NOTES.md
  • cookiecutter
  • tests_golden/integration/test_cookiecutter_generation
    • actor/frequenz-actor-test/.github/workflows
    • api/frequenz-api-test/.github/workflows
    • app/frequenz-app-test/.github/workflows
    • lib/frequenz-test-python/.github/workflows
    • model/frequenz-model-test/.github/workflows

‎.github/workflows/ci.yaml‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,9 @@ jobs:
2828
strategy:
2929
fail-fast: false
3030
matrix:
31-
arch:
32-
- amd64
33-
- arm
34-
os:
31+
platform:
3532
- ubuntu-24.04
33+
- ubuntu-24.04-arm
3634
python:
3735
- "3.11"
3836
- "3.12"
@@ -41,7 +39,7 @@ jobs:
4139
# that uses the same venv to run multiple linting sessions
4240
- "ci_checks_max"
4341
- "pytest_min"
44-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
42+
runs-on: ${{ matrix.platform }}
4543

4644
steps:
4745
- name: Run nox
@@ -105,15 +103,13 @@ jobs:
105103
strategy:
106104
fail-fast: false
107105
matrix:
108-
arch:
109-
- amd64
110-
- arm
111-
os:
106+
platform:
112107
- ubuntu-24.04
108+
- ubuntu-24.04-arm
113109
python:
114110
- "3.11"
115111
- "3.12"
116-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
112+
runs-on: ${{ matrix.platform }}
117113

118114
steps:
119115
- name: Setup Git

‎RELEASE_NOTES.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ But you might still need to adapt your code:
3939

4040
- Added the [`flake8-datetimez`](https://github.com/pjknkda/flake8-datetimez) plugin to the `flake8` session. This plugin prevents accidental use of naive `datetime` objects by flagging calls that create or return datetimes without timezone information.
4141

42+
- The CI workflow now uses a simpler matrix.
43+
4244
## Bug Fixes
4345

4446
<!-- Here goes notable bug fixes that are worth a special mention or explanation -->

‎cookiecutter/migrate.py‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,9 @@ def main() -> None:
5252
print("Migrating auto-dependabot workflow to use GitHub App token...")
5353
migrate_auto_dependabot_token()
5454
print("=" * 72)
55+
print("Migrating the CI workflows to use a platform matrix...")
56+
migrate_platform_matrix()
57+
print("=" * 72)
5558
print()
5659

5760
if _manual_steps:
@@ -555,6 +558,55 @@ def replace_setuptools_pin(content: str, new_version: str) -> tuple[str, bool]:
555558
return new_content, count > 0
556559

557560

561+
def migrate_platform_matrix() -> None:
562+
"""Migrate CI matrix from arch+os to a single platform entry.
563+
564+
This replaces the old matrix definition that used separate `arch` and `os`
565+
entries with a single `platform` entry using GitHub's native arm64 runners
566+
that are now available to both public and private repositories.
567+
"""
568+
workflow_file = Path(".github/workflows/ci.yaml")
569+
print(f" - {workflow_file}")
570+
if not workflow_file.is_file():
571+
manual_step(
572+
f"Could not find {workflow_file}; please manually migrate to use a"
573+
"please manually migrate to use a `platform` matrix entry."
574+
)
575+
return
576+
577+
content = workflow_file.read_text(encoding="utf-8")
578+
new_content = content
579+
580+
# Replace the arch+os matrix block with platform.
581+
# Handle both "arm" (old) and "arm64" (intermediate) variants.
582+
new_content = re.sub(
583+
r"( +)arch:\n\1 - amd64\n\1 - arm(?:64)?\n\1os:\n\1 - ubuntu-24\.04\n",
584+
r"\g<1>platform:\n\g<1> - ubuntu-24.04\n\g<1> - ubuntu-24.04-arm\n",
585+
new_content,
586+
)
587+
588+
# Replace any runs-on expression referencing matrix.arch with the simpler
589+
# matrix.platform reference.
590+
new_content = re.sub(
591+
r"runs-on: \$\{\{.*matrix\.arch.*\}\}",
592+
"runs-on: ${{ matrix.platform }}",
593+
new_content,
594+
)
595+
596+
if new_content == content:
597+
if "matrix.platform" in content:
598+
print(" Already uses platform matrix")
599+
else:
600+
manual_step(
601+
f"Could not find arch+os matrix pattern in {workflow_file}; "
602+
"please manually migrate to use a `platform` matrix entry."
603+
)
604+
return
605+
606+
replace_file_contents_atomically(workflow_file, content, new_content, count=1)
607+
print(" Migrated arch+os matrix to platform")
608+
609+
558610
def apply_patch(patch_content: str) -> None:
559611
"""Apply a patch using the patch utility."""
560612
subprocess.run(["patch", "-p1"], input=patch_content.encode(), check=True)

‎cookiecutter/{{cookiecutter.github_repo_name}}/.github/workflows/ci.yaml‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -61,11 +61,9 @@ jobs:
6161
strategy:
6262
fail-fast: false
6363
matrix:
64-
arch:
65-
- amd64
66-
- arm
67-
os:
64+
platform:
6865
- ubuntu-24.04
66+
- ubuntu-24.04-arm
6967
python:
7068
- "3.11"
7169
- "3.12"
@@ -74,7 +72,7 @@ jobs:
7472
# that uses the same venv to run multiple linting sessions
7573
- "ci_checks_max"
7674
- "pytest_min"
77-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
75+
runs-on: ${{ matrix.platform }}
7876

7977
steps:
8078
- name: Run nox
@@ -145,15 +143,13 @@ jobs:
145143
strategy:
146144
fail-fast: false
147145
matrix:
148-
arch:
149-
- amd64
150-
- arm
151-
os:
146+
platform:
152147
- ubuntu-24.04
148+
- ubuntu-24.04-arm
153149
python:
154150
- "3.11"
155151
- "3.12"
156-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
152+
runs-on: ${{ matrix.platform }}
157153

158154
steps:
159155
- name: Setup Git

‎tests_golden/integration/test_cookiecutter_generation/actor/frequenz-actor-test/.github/workflows/ci.yaml‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,9 @@ jobs:
2828
strategy:
2929
fail-fast: false
3030
matrix:
31-
arch:
32-
- amd64
33-
- arm
34-
os:
31+
platform:
3532
- ubuntu-24.04
33+
- ubuntu-24.04-arm
3634
python:
3735
- "3.11"
3836
- "3.12"
@@ -41,7 +39,7 @@ jobs:
4139
# that uses the same venv to run multiple linting sessions
4240
- "ci_checks_max"
4341
- "pytest_min"
44-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
42+
runs-on: ${{ matrix.platform }}
4543

4644
steps:
4745
- name: Run nox
@@ -112,15 +110,13 @@ jobs:
112110
strategy:
113111
fail-fast: false
114112
matrix:
115-
arch:
116-
- amd64
117-
- arm
118-
os:
113+
platform:
119114
- ubuntu-24.04
115+
- ubuntu-24.04-arm
120116
python:
121117
- "3.11"
122118
- "3.12"
123-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
119+
runs-on: ${{ matrix.platform }}
124120

125121
steps:
126122
- name: Setup Git

‎tests_golden/integration/test_cookiecutter_generation/api/frequenz-api-test/.github/workflows/ci.yaml‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -58,11 +58,9 @@ jobs:
5858
strategy:
5959
fail-fast: false
6060
matrix:
61-
arch:
62-
- amd64
63-
- arm
64-
os:
61+
platform:
6562
- ubuntu-24.04
63+
- ubuntu-24.04-arm
6664
python:
6765
- "3.11"
6866
- "3.12"
@@ -71,7 +69,7 @@ jobs:
7169
# that uses the same venv to run multiple linting sessions
7270
- "ci_checks_max"
7371
- "pytest_min"
74-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
72+
runs-on: ${{ matrix.platform }}
7573

7674
steps:
7775
- name: Run nox
@@ -142,15 +140,13 @@ jobs:
142140
strategy:
143141
fail-fast: false
144142
matrix:
145-
arch:
146-
- amd64
147-
- arm
148-
os:
143+
platform:
149144
- ubuntu-24.04
145+
- ubuntu-24.04-arm
150146
python:
151147
- "3.11"
152148
- "3.12"
153-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
149+
runs-on: ${{ matrix.platform }}
154150

155151
steps:
156152
- name: Setup Git

‎tests_golden/integration/test_cookiecutter_generation/app/frequenz-app-test/.github/workflows/ci.yaml‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,9 @@ jobs:
2828
strategy:
2929
fail-fast: false
3030
matrix:
31-
arch:
32-
- amd64
33-
- arm
34-
os:
31+
platform:
3532
- ubuntu-24.04
33+
- ubuntu-24.04-arm
3634
python:
3735
- "3.11"
3836
- "3.12"
@@ -41,7 +39,7 @@ jobs:
4139
# that uses the same venv to run multiple linting sessions
4240
- "ci_checks_max"
4341
- "pytest_min"
44-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
42+
runs-on: ${{ matrix.platform }}
4543

4644
steps:
4745
- name: Run nox
@@ -112,15 +110,13 @@ jobs:
112110
strategy:
113111
fail-fast: false
114112
matrix:
115-
arch:
116-
- amd64
117-
- arm
118-
os:
113+
platform:
119114
- ubuntu-24.04
115+
- ubuntu-24.04-arm
120116
python:
121117
- "3.11"
122118
- "3.12"
123-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
119+
runs-on: ${{ matrix.platform }}
124120

125121
steps:
126122
- name: Setup Git

‎tests_golden/integration/test_cookiecutter_generation/lib/frequenz-test-python/.github/workflows/ci.yaml‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,9 @@ jobs:
2828
strategy:
2929
fail-fast: false
3030
matrix:
31-
arch:
32-
- amd64
33-
- arm
34-
os:
31+
platform:
3532
- ubuntu-24.04
33+
- ubuntu-24.04-arm
3634
python:
3735
- "3.11"
3836
- "3.12"
@@ -41,7 +39,7 @@ jobs:
4139
# that uses the same venv to run multiple linting sessions
4240
- "ci_checks_max"
4341
- "pytest_min"
44-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
42+
runs-on: ${{ matrix.platform }}
4543

4644
steps:
4745
- name: Run nox
@@ -112,15 +110,13 @@ jobs:
112110
strategy:
113111
fail-fast: false
114112
matrix:
115-
arch:
116-
- amd64
117-
- arm
118-
os:
113+
platform:
119114
- ubuntu-24.04
115+
- ubuntu-24.04-arm
120116
python:
121117
- "3.11"
122118
- "3.12"
123-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
119+
runs-on: ${{ matrix.platform }}
124120

125121
steps:
126122
- name: Setup Git

‎tests_golden/integration/test_cookiecutter_generation/model/frequenz-model-test/.github/workflows/ci.yaml‎

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,9 @@ jobs:
2828
strategy:
2929
fail-fast: false
3030
matrix:
31-
arch:
32-
- amd64
33-
- arm
34-
os:
31+
platform:
3532
- ubuntu-24.04
33+
- ubuntu-24.04-arm
3634
python:
3735
- "3.11"
3836
- "3.12"
@@ -41,7 +39,7 @@ jobs:
4139
# that uses the same venv to run multiple linting sessions
4240
- "ci_checks_max"
4341
- "pytest_min"
44-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
42+
runs-on: ${{ matrix.platform }}
4543

4644
steps:
4745
- name: Run nox
@@ -112,15 +110,13 @@ jobs:
112110
strategy:
113111
fail-fast: false
114112
matrix:
115-
arch:
116-
- amd64
117-
- arm
118-
os:
113+
platform:
119114
- ubuntu-24.04
115+
- ubuntu-24.04-arm
120116
python:
121117
- "3.11"
122118
- "3.12"
123-
runs-on: ${{ matrix.os }}${{ matrix.arch != 'amd64' && format('-{0}', matrix.arch) || '' }}
119+
runs-on: ${{ matrix.platform }}
124120

125121
steps:
126122
- name: Setup Git

0 commit comments

Comments
 (0)