debug: add diagnostic logging to EFA NCCL test#6114

Open

Eren-Jeager123 wants to merge 34 commits into

mainfrom

Commits on May 20, 2026

Trigger EFA Test
Eren-Jeager123
committed
fix: set NCCL_NET_PLUGIN=ofi for EFA NCCL plugin discovery
Eren-Jeager123
committed
debug: add EFA/NCCL plugin diagnostics to allreduce test
Eren-Jeager123
committed
debug: move diagnostics to pytest for visible output
Eren-Jeager123
committed
debug: use print() and warn=True for visible diagnostic output
Eren-Jeager123
committed
debug: use print() and warn=True for visible diagnostic output
Eren-Jeager123
committed
fix: add cuda-compat to LD_LIBRARY_PATH for driver forward compatibility
Eren-Jeager123
committed
fix: remove cuda-compat override from allreduce test script
Eren-Jeager123
committed
fix: add cuda-compat to LD_LIBRARY_PATH for driver forward compatibility
Eren-Jeager123
committed
fix: use LD_PRELOAD for cuda-compat in EFA NCCL test
Eren-Jeager123
committed
fix: use resolved path for LD_PRELOAD cuda-compat and add load verification
Eren-Jeager123
committed
fix: stop upgrading cuda-compat to avoid driver mismatch
Eren-Jeager123
committed
cleanup: remove LD_PRELOAD hack, real fix is in Dockerfile cuda-compat
Eren-Jeager123
committed
fix: keep cuda-compat.conf for driver forward compatibility
Eren-Jeager123
committed
chore: retrigger CI
Eren-Jeager123
committed
fix: exclude cuda-compat from security upgrade to match host driver
Eren-Jeager123
committed
fix: add CACHE_REFRESH ARG to invalidate cached security patch layer
Eren-Jeager123
committed
cleanup: remove NCCL_NET_PLUGIN=ofi debug env var
Eren-Jeager123
committed
fix: remove trailing backslash from ENV after NCCL_NET_PLUGIN removal
Eren-Jeager123
committed
fix: remove cuda-compat upgrade to prevent driver version mismatch
Eren-Jeager123
committed
remove .claude from tracking
Eren-Jeager123
committed
debug: add pre-flight checks and log dump for EFA NCCL test
Eren-Jeager123
committed
fix: add CACHE_BUST ARG to force rebuild without stale cuda-compat
Eren-Jeager123
committed
fix: restore EFA test diagnostics (CUDA driver info, SG rules, NCCL allreduce capture)
Eren-Jeager123
committed
debug: add CUDA availability check on p4d, remove CACHE_BUST
Eren-Jeager123
committed
fix: nested quote syntax error in CUDA diagnostic command
Eren-Jeager123
committed
fix: remove python torch check that breaks nested quoting, keep nvidia-smi only
Eren-Jeager123
committed
fix: add diagnostic logging to EFA test for failure analysis
Eren-Jeager123
committed
remove .claude from tracking
Eren-Jeager123
committed
chore: trigger EFA test rebuild
Eren-Jeager123
committed