fix: correct per-PP-stage CPU/SSD layer offset and add transfer correctness test suite#171
Draft
zhjc1124 wants to merge 1 commit into
Draft
Conversation
…ctness test suite Fix the per-PP-stage layer offset logic so that each worker advances its CPU/SSD base pointer by start_layer_id * layer_stride, ensuring the C++ kernel always addresses GPU tensors with layer_id=0 (PP-stage-local) while writing to the correct slice of the shared per-node CPU/SSD pool. Unify the offset implementation and comment style across all five worker types (GPUCPUTransferWorker, tpGPUCPUTransferWorker, GDSTransferWorker, tpGDSTransferWorker, LayerwiseTransferWorker) so the same design principle is consistently expressed regardless of whether the offset is applied via tensor slicing, raw pointer arithmetic, layer_id_list, or a scalar kernel argument. Add tests/test_transfer_correctness.py, a parametrized correctness test suite covering CPU-only, CPU+SSD, GDS, layerwise-CPU-only and layerwise-CPU+SSD backends across same-node PP, simulated cross-node PP, same-node TP, simulated cross-node TP, PP+TP and cross-node PP+TP parallel configurations. Tests automatically downgrade to single-GPU simulation when physical GPUs are insufficient and skip unavailable backends.
zhjc1124
force-pushed
the
fix/single-node-pp-cpu-pool
branch
from
22222ed to
92b4c49
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix the per-PP-stage layer offset logic so that each worker advances its CPU/SSD base pointer by
start_layer_id * layer_stride, ensuring the C++ kernel always addresses GPU tensors withlayer_id=0(PP-stage-local) while writing to the correct slice of the shared per-node CPU/SSD pool.Unify the offset implementation and comment style across all five worker types (
GPUCPUTransferWorker,tpGPUCPUTransferWorker,GDSTransferWorker,tpGDSTransferWorker,LayerwiseTransferWorker) so the same design principle is consistently expressed regardless of whether the offset is applied via tensor slicing, raw pointer arithmetic,layer_id_list, or a scalar kernel argument.Add
tests/test_transfer_correctness.py, a parametrized correctness test suite covering CPU-only, CPU+SSD, GDS, layerwise-CPU-only and layerwise-CPU+SSD backends across same-node PP, simulated cross-node PP, same-node TP, simulated cross-node TP, PP+TP and cross-node PP+TP parallel configurations. Tests automatically downgrade to single-GPU simulation when physical GPUs are insufficient and skip unavailable backends.