Skip to content

ci: optimize distro-qemu tests + GPU e2e tests#232

Closed
gnurizen wants to merge 12 commits into
mainfrom
ci-e2e
Closed

ci: optimize distro-qemu tests + GPU e2e tests#232
gnurizen wants to merge 12 commits into
mainfrom
ci-e2e

Conversation

@gnurizen
Copy link
Copy Markdown
Collaborator

Summary

  • CI optimizations for distro-qemu tests: replace debootstrap with busybox+shared-libs, split build-initramfs + run-qemu into separate jobs, add timing instrumentation, smarter caching
  • End-to-end GPU profiler tests with CUDA USDT probe verification
  • Fix KVM detection to use /dev/kvm instead of x86-specific vmx/svm grep

Test plan

  • distro-qemu tests pass on all kernel/arch combinations
  • GPU e2e tests (cudaverify) pass
  • Build-initramfs job produces correct artifact consumed by run-qemu jobs

🤖 Generated with Claude Code

gnurizen and others added 12 commits March 10, 2026 12:05
- Use native ARM64 runners (ubuntu-24.04-arm) for arm64 jobs instead
  of emulating arm64 on x86 via QEMU without KVM acceleration
- Remove heavy env action from distro-qemu tests (was installing Rust,
  protoc, cross-compile tools - none needed for these tests)
- Cache kernel images to avoid Docker pulls on every run
- Add --no-install-recommends to debootstrap installation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
run-tests.sh had an inverted KVM check (enabled KVM when vmx/svm was
NOT found), causing arm64 integration tests to fail on ARM runners that
lack /dev/kvm. Switch both scripts to check /dev/kvm directly, which
works correctly on both x86 and ARM.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The distro-qemu tests only need a handful of shared libraries (libc,
libm, ld-linux, libgcc_s) and basic shell utilities. Replace the full
debootstrap rootfs (~150-300MB) with a minimal rootfs that copies just
the needed shared libraries from the host and uses busybox-static for
shell/utilities. This reduces the initramfs from ~50MB to ~5MB and
eliminates the debootstrap download/install step entirely.

The rootfs now contains only:
- busybox (static) for sh, mount, dmesg, poweroff, etc.
- Shared libraries detected via ldd on the test binaries
- libm.so (explicitly copied for rtld dlopen test)
- Test binaries and parcagpu .so

Also removes the DISTRO/RELEASE/MIRROR configuration and sudo
requirements (no more chroot operations).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ::> timing markers to build-and-run.sh to profile where time is
spent: parcagpu download, go test -c, rootfs creation, initramfs
creation, QEMU execution, and per-test timing inside the VM.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The initramfs (test binaries + busybox + shared libs) is arch-specific
but kernel-independent. Split into:
- build-distro-qemu-initramfs: builds once per arch, uploads artifact
- distro-qemu-tests: downloads initramfs + kernel, runs QEMU only

This avoids rebuilding Go test binaries and the rootfs for every kernel
version (was ~46s per job on amd64: 27s go build + 15s rootfs + 4s
initramfs). Each test job now only needs QEMU + kernel download.

The build-and-run.sh wrapper is kept for local development convenience.
ParseUSDTProbes now calls LoadSections internally so callers cannot
silently get zero probes when sections have not been loaded yet.

Remove the package-level sync.Once for the cuda_progs prog array
update — it prevented subsequent tracer instances from populating
their own maps (in tests).  UpdateProgArray is already idempotent.

Neither of these changes fix anything but they both seemed like good
changes.
libparcagpucupti.so depends on libstdc++.so.6 which wasn't being
copied because copy_lib_deps only ran on test binaries, not on the
parcagpu shared library itself.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gnurizen gnurizen closed this Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant