Skip to content

Commit 938aee3

Browse files
committed
circleci: restructure pipeline to fix fan-in and reduce io/network cost
Problem: fan-in = multiple upstream jobs persist same paths to workspace. Origin/master had massive IO/network cost from rebuilding everything. Thanks CircleCI free tier for open source projects! Mitigation for circleci.canny.io/cloud-feature-requests/p/support-wildcards-in-savecachepaths Changes vs origin/master: - Add 'heads-docker' executor (centralized docker config, image v0.2.9) - Add glossary explaining fan-in, workspace chain, cache layers - Split build_and_persist into separate jobs per stage: build (parallel board builds), x86_coreboot (per fork), x86_musl_cross_make, x86_blobs, x86_save_modules_cache, ppc64_talos_2 - Add create_hashes job (creates cache hashes, shared by all arches) - Replace save_cache with per-job cache saving (eliminates fan-in) - Each job saves cache immediately before persisting to workspace - x86 chain: create_hashes -> x86_blobs -> x86_musl_cross_make -> x86_coreboot seeds -> parallel builds + x86_save_modules_cache - ppc64 chain: create_hashes -> ppc64_talos_2 (linear) - Build: only clean logs dir, not entire board dir (performance) - Cache hierarchy: modules > coreboot+musl > musl > blobs (x86 only, workspace) - Fix ppc64 target: 'UNTESTED_talos-2' - Add checkout step to all jobs - Rename cache keys: modules-and-patches -> modules Signed-off-by: Thierry Laurion <insurgo@riseup.net>
1 parent 67c2b28 commit 938aee3

File tree

1 file changed

+264
-158
lines changed

1 file changed

+264
-158
lines changed

0 commit comments

Comments
 (0)