Skip to content

Commit 50580a8

Browse files
mudlermudler
committed
fix(ci): switch apt mirror per runner — azure on github-hosted, kernel.org on self-hosted
Self-hosted runners (arc-runner-set, bigger-runner) cannot reach azure.archive.ubuntu.com — they live in different networks (e.g. our arc-runner-set Kubernetes cluster) where Azure's mirror IP is not routable. Symptom: "Connection failed [IP: 51.11.236.225 80]" with each Ign:/Err: cycle taking 60s, hanging the build for ~16 minutes before exit 100. Pick the mirror based on `runner.environment`: * github-hosted (ubuntu-latest, ubuntu-24.04-arm) → Azure (http://azure.archive.ubuntu.com / http://azure.ports.ubuntu.com) — same VPC as the runner. * self-hosted (arc-runner-set, bigger-runner) → kernel.org (https://mirrors.edge.kernel.org for both archive and ports) — publicly reachable from any network. The choice now lives in one place: the .github/actions/configure-apt-mirror composite action exposes `effective-mirror` / `effective-ports-mirror` outputs so the reusable workflows can forward the same value as Docker build-args without duplicating the per-runner-environment branch. The now-redundant `apt-mirror` / `apt-ports-mirror` workflow inputs on image_build.yml and backend_build.yml are dropped — defaults live in the composite action and are visible there. Assisted-by: Claude:claude-opus-4-7[1m] [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 parent 8edac61 commit 50580a8

3 files changed

Lines changed: 68 additions & 50 deletions

File tree

‎.github/actions/configure-apt-mirror/action.yml‎

Lines changed: 58 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,79 @@
11
name: 'Configure apt mirror'
22
description: |
33
Reconfigure the GitHub Actions runner's Ubuntu apt sources to use an
4-
alternate mirror. Defaults to the Azure-hosted Ubuntu mirror, which lives
5-
on the same network as the runners and is independent of the canonical
6-
Ubuntu pool — useful as a fallback when archive.ubuntu.com /
7-
security.ubuntu.com / ports.ubuntu.com are degraded.
4+
alternate mirror, and emit the effective URLs as outputs so callers can
5+
forward them as Docker build-args.
86
9-
Pass mirror: '' (empty string) to skip the rewrite and keep upstream.
7+
Two mirror profiles depending on where the runner lives, because the
8+
best mirror differs by network:
9+
10+
* github-hosted runners run on Azure, so they default to the
11+
Azure-hosted Ubuntu mirror (lowest latency, same VPC).
12+
* self-hosted runners (arc-runner-set, bigger-runner, ...) typically
13+
cannot route to azure.archive.ubuntu.com, so they default to the
14+
kernel.org mirror, which is publicly reachable from anywhere.
15+
16+
Pass an empty string to either input to skip the rewrite for that
17+
profile and keep upstream archive.ubuntu.com / ports.ubuntu.com.
1018
1119
inputs:
12-
mirror:
13-
description: 'Replacement URL for archive.ubuntu.com / security.ubuntu.com (empty = keep upstream)'
20+
github-hosted-mirror:
21+
description: 'archive/security mirror URL for github-hosted runners (empty = upstream)'
1422
required: false
1523
default: 'http://azure.archive.ubuntu.com'
16-
ports-mirror:
17-
description: 'Replacement URL for ports.ubuntu.com on arm64 (empty = keep upstream)'
24+
github-hosted-ports-mirror:
25+
description: 'ports.ubuntu.com mirror URL for github-hosted runners (empty = upstream)'
1826
required: false
1927
default: 'http://azure.ports.ubuntu.com'
28+
self-hosted-mirror:
29+
description: 'archive/security mirror URL for self-hosted runners (empty = upstream)'
30+
required: false
31+
default: 'https://mirrors.edge.kernel.org'
32+
self-hosted-ports-mirror:
33+
description: 'ports.ubuntu.com mirror URL for self-hosted runners (empty = upstream)'
34+
required: false
35+
default: 'https://mirrors.edge.kernel.org'
36+
37+
outputs:
38+
effective-mirror:
39+
description: 'The mirror URL actually applied for this runner (or empty)'
40+
value: ${{ steps.pick.outputs.mirror }}
41+
effective-ports-mirror:
42+
description: 'The ports mirror URL actually applied for this runner (or empty)'
43+
value: ${{ steps.pick.outputs.ports-mirror }}
2044

2145
runs:
2246
using: 'composite'
2347
steps:
48+
- name: Pick effective mirror for this runner
49+
id: pick
50+
shell: bash
51+
env:
52+
RUNNER_ENV: ${{ runner.environment }}
53+
GH_MIRROR: ${{ inputs.github-hosted-mirror }}
54+
GH_PORTS_MIRROR: ${{ inputs.github-hosted-ports-mirror }}
55+
SH_MIRROR: ${{ inputs.self-hosted-mirror }}
56+
SH_PORTS_MIRROR: ${{ inputs.self-hosted-ports-mirror }}
57+
run: |
58+
if [ "${RUNNER_ENV}" = "github-hosted" ]; then
59+
MIRROR="${GH_MIRROR}"
60+
PORTS_MIRROR="${GH_PORTS_MIRROR}"
61+
else
62+
MIRROR="${SH_MIRROR}"
63+
PORTS_MIRROR="${SH_PORTS_MIRROR}"
64+
fi
65+
echo "configure-apt-mirror: runner=${RUNNER_ENV} mirror='${MIRROR}' ports-mirror='${PORTS_MIRROR}'"
66+
echo "mirror=${MIRROR}" >> "$GITHUB_OUTPUT"
67+
echo "ports-mirror=${PORTS_MIRROR}" >> "$GITHUB_OUTPUT"
68+
2469
- name: Rewrite apt sources
70+
if: steps.pick.outputs.mirror != '' || steps.pick.outputs.ports-mirror != ''
2571
shell: bash
2672
env:
27-
APT_MIRROR: ${{ inputs.mirror }}
28-
APT_PORTS_MIRROR: ${{ inputs.ports-mirror }}
73+
APT_MIRROR: ${{ steps.pick.outputs.mirror }}
74+
APT_PORTS_MIRROR: ${{ steps.pick.outputs.ports-mirror }}
2975
run: |
3076
set -e
31-
if [ -z "${APT_MIRROR}" ] && [ -z "${APT_PORTS_MIRROR}" ]; then
32-
echo "configure-apt-mirror: both inputs empty, leaving upstream sources untouched"
33-
exit 0
34-
fi
3577
# Ubuntu 24.04 (noble) ships DEB822 sources at
3678
# /etc/apt/sources.list.d/ubuntu.sources; older releases use
3779
# /etc/apt/sources.list. Rewrite whichever exists.
@@ -46,4 +88,4 @@ runs:
4688
sudo sed -i -E "s,https?://ports\.ubuntu\.com,${APT_PORTS_MIRROR},g" "$f"
4789
fi
4890
done
49-
echo "configure-apt-mirror: APT_MIRROR='${APT_MIRROR}' APT_PORTS_MIRROR='${APT_PORTS_MIRROR}'"
91+
echo "Runner apt mirror configured (APT_MIRROR='${APT_MIRROR}', APT_PORTS_MIRROR='${APT_PORTS_MIRROR}')"

‎.github/workflows/backend_build.yml‎

Lines changed: 5 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -63,16 +63,6 @@ on:
6363
required: false
6464
default: ''
6565
type: string
66-
apt-mirror:
67-
description: 'Replacement URL for archive.ubuntu.com / security.ubuntu.com (empty = use upstream)'
68-
required: false
69-
default: 'http://azure.archive.ubuntu.com'
70-
type: string
71-
apt-ports-mirror:
72-
description: 'Replacement URL for ports.ubuntu.com (arm64 etc., empty = use upstream)'
73-
required: false
74-
default: 'http://azure.ports.ubuntu.com'
75-
type: string
7666
secrets:
7767
dockerUsername:
7868
required: false
@@ -96,10 +86,8 @@ jobs:
9686
submodules: true
9787

9888
- name: Configure apt mirror on runner
89+
id: apt_mirror
9990
uses: ./.github/actions/configure-apt-mirror
100-
with:
101-
mirror: ${{ inputs.apt-mirror }}
102-
ports-mirror: ${{ inputs.apt-ports-mirror }}
10391

10492
- name: Free Disk Space (Ubuntu)
10593
if: inputs.runs-on == 'ubuntu-latest'
@@ -237,8 +225,8 @@ jobs:
237225
BACKEND=${{ inputs.backend }}
238226
UBUNTU_VERSION=${{ inputs.ubuntu-version }}
239227
AMDGPU_TARGETS=${{ inputs.amdgpu-targets }}
240-
APT_MIRROR=${{ inputs.apt-mirror }}
241-
APT_PORTS_MIRROR=${{ inputs.apt-ports-mirror }}
228+
APT_MIRROR=${{ steps.apt_mirror.outputs.effective-mirror }}
229+
APT_PORTS_MIRROR=${{ steps.apt_mirror.outputs.effective-ports-mirror }}
242230
DEPS_REFRESH=${{ steps.deps_refresh.outputs.key }}
243231
context: ${{ inputs.context }}
244232
file: ${{ inputs.dockerfile }}
@@ -263,8 +251,8 @@ jobs:
263251
BACKEND=${{ inputs.backend }}
264252
UBUNTU_VERSION=${{ inputs.ubuntu-version }}
265253
AMDGPU_TARGETS=${{ inputs.amdgpu-targets }}
266-
APT_MIRROR=${{ inputs.apt-mirror }}
267-
APT_PORTS_MIRROR=${{ inputs.apt-ports-mirror }}
254+
APT_MIRROR=${{ steps.apt_mirror.outputs.effective-mirror }}
255+
APT_PORTS_MIRROR=${{ steps.apt_mirror.outputs.effective-ports-mirror }}
268256
DEPS_REFRESH=${{ steps.deps_refresh.outputs.key }}
269257
context: ${{ inputs.context }}
270258
file: ${{ inputs.dockerfile }}

‎.github/workflows/image_build.yml‎

Lines changed: 5 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -56,16 +56,6 @@ on:
5656
required: false
5757
default: 'noble'
5858
type: string
59-
apt-mirror:
60-
description: 'Replacement URL for archive.ubuntu.com / security.ubuntu.com (empty = use upstream)'
61-
required: false
62-
default: 'http://azure.archive.ubuntu.com'
63-
type: string
64-
apt-ports-mirror:
65-
description: 'Replacement URL for ports.ubuntu.com (arm64 etc., empty = use upstream)'
66-
required: false
67-
default: 'http://azure.ports.ubuntu.com'
68-
type: string
6959
secrets:
7060
dockerUsername:
7161
required: true
@@ -84,10 +74,8 @@ jobs:
8474
uses: actions/checkout@v6
8575

8676
- name: Configure apt mirror on runner
77+
id: apt_mirror
8778
uses: ./.github/actions/configure-apt-mirror
88-
with:
89-
mirror: ${{ inputs.apt-mirror }}
90-
ports-mirror: ${{ inputs.apt-ports-mirror }}
9179

9280
- name: Free Disk Space (Ubuntu)
9381
if: inputs.runs-on == 'ubuntu-latest'
@@ -214,8 +202,8 @@ jobs:
214202
SKIP_DRIVERS=${{ inputs.skip-drivers }}
215203
UBUNTU_VERSION=${{ inputs.ubuntu-version }}
216204
UBUNTU_CODENAME=${{ inputs.ubuntu-codename }}
217-
APT_MIRROR=${{ inputs.apt-mirror }}
218-
APT_PORTS_MIRROR=${{ inputs.apt-ports-mirror }}
205+
APT_MIRROR=${{ steps.apt_mirror.outputs.effective-mirror }}
206+
APT_PORTS_MIRROR=${{ steps.apt_mirror.outputs.effective-ports-mirror }}
219207
context: .
220208
file: ./Dockerfile
221209
cache-from: type=registry,ref=quay.io/go-skynet/ci-cache:cache-localai${{ inputs.tag-suffix }}
@@ -239,8 +227,8 @@ jobs:
239227
SKIP_DRIVERS=${{ inputs.skip-drivers }}
240228
UBUNTU_VERSION=${{ inputs.ubuntu-version }}
241229
UBUNTU_CODENAME=${{ inputs.ubuntu-codename }}
242-
APT_MIRROR=${{ inputs.apt-mirror }}
243-
APT_PORTS_MIRROR=${{ inputs.apt-ports-mirror }}
230+
APT_MIRROR=${{ steps.apt_mirror.outputs.effective-mirror }}
231+
APT_PORTS_MIRROR=${{ steps.apt_mirror.outputs.effective-ports-mirror }}
244232
context: .
245233
file: ./Dockerfile
246234
cache-from: type=registry,ref=quay.io/go-skynet/ci-cache:cache-localai${{ inputs.tag-suffix }}

0 commit comments

Comments
 (0)