Skip to content

Commit 425f2f2

Browse files
committed
ci: portable retry loop in download-and-unpack for BusyBox wget
The GNU wget retry flags I added in the previous commit (--tries, --waitretry, --timeout, --retry-connrefused) are not supported by BusyBox wget, which is what the Alpine container image ships. Every alpine matrix cell on run 26723340409 died immediately at the first download with "wget: unrecognized option: waitretry=3". Hand-roll the retry loop in bash so we only need flags that work in both GNU and BusyBox wget (-q, -O, -U, -T). Same retry count, same backoff, same temp-file staging — just doesn't lean on wget for the retry semantics anymore. Verified locally on macOS (GNU wget 1.21.3) and on node:25-alpine3.22 (BusyBox wget) — both extract zlib's v1.3.1 tarball cleanly.
1 parent 95a802b commit 425f2f2

1 file changed

Lines changed: 29 additions & 7 deletions

File tree

scripts/ci/download-and-unpack.sh

Lines changed: 29 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,41 @@ download_and_unpack() {
88
mkdir -p "$2"
99
# Download to a temp file first instead of streaming straight into tar.
1010
# The previous `wget -qO- | tar xzf -` pipe couldn't recover from a
11-
# truncated response (e.g. curl.se/GitHub-release occasional hiccups) —
11+
# truncated response (e.g. curl.se / GitHub-release occasional hiccups) —
1212
# tar consumes the partial bytes immediately and fails with
13-
# "gzip: stdin: unexpected end of file" before wget's own retry can fire.
14-
# With a temp file, wget can retry until the file is whole, and we only
13+
# "gzip: stdin: unexpected end of file" before any retry can fire.
14+
# With a temp file, we can retry until the file is whole, and we only
1515
# touch tar once.
16+
#
17+
# The retry loop is done in bash rather than via wget's own retry flags
18+
# because Alpine ships BusyBox wget, which only supports -c/-q/-O/-U/-T —
19+
# no --tries, --waitretry, --retry-connrefused. Hand-rolling the loop
20+
# keeps us portable across GNU wget (Ubuntu/macOS) and BusyBox wget
21+
# (Alpine container) without conditional code.
1622
local tmpfile
1723
tmpfile=$(mktemp)
1824
trap "rm -f \"$tmpfile\"" RETURN
25+
26+
local attempts=5
27+
local i=1
28+
local sleep_sec=3
1929
# User agent for Edge on macOS
20-
wget --tries=5 --waitretry=3 --timeout=30 --retry-connrefused \
21-
-U "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Edg/94.0.992.38" \
22-
-qO "$tmpfile" "$1"
23-
tar xzf "$tmpfile" -C "$2"
30+
local user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Edg/94.0.992.38"
31+
32+
while [ "$i" -le "$attempts" ]; do
33+
# -T sets the read timeout in seconds and is available in both GNU
34+
# and BusyBox wget.
35+
if wget -q -T 60 -U "$user_agent" -O "$tmpfile" "$1"; then
36+
tar xzf "$tmpfile" -C "$2"
37+
return 0
38+
fi
39+
echo "download attempt $i/$attempts for $1 failed; retrying in ${sleep_sec}s..." >&2
40+
i=$((i + 1))
41+
sleep "$sleep_sec"
42+
done
43+
44+
echo "download failed after $attempts attempts: $1" >&2
45+
return 1
2446
}
2547

2648
if [ "${1}" != "--source-only" ]; then

0 commit comments

Comments
 (0)