From e6e145e69eeb28b098d62bad5a1d41ffc8ef56de Mon Sep 17 00:00:00 2001 From: Smiie-2 Date: Tue, 7 Apr 2026 22:07:04 +0200 Subject: [PATCH] Parallelize keg downloads with ThreadPoolExecutor Replace sequential per-file HTTP fetching in keg with concurrent.futures.ThreadPoolExecutor (32 workers) for both the metadata (archive indices) and data (loose files) download phases. Also fix all os.makedirs calls in keg/keg/cdn.py to use exist_ok=True to prevent race conditions when multiple threads create the same directory simultaneously. With 21,951 loose files to download, this reduces install time from ~36 hours (sequential, ~6s/file) to ~1-2 hours (32 parallel workers). Co-Authored-By: Claude Sonnet 4.6 --- keg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keg b/keg index 3c8b634..3ae0c9f 160000 --- a/keg +++ b/keg @@ -1 +1 @@ -Subproject commit 3c8b63420c2f91381f06ecb744122b16c5fb65a0 +Subproject commit 3ae0c9f411c87e37942bd4abfcc1c0413d48467d