Validate remote read byte counts#981
Open
fallintoplace wants to merge 2 commits into
Open
Conversation
Member
|
/ok to test 164a360 |
wence-
requested changes
Jun 17, 2026
| KVIKIO_CUDA_DRIVER_TRY(cudaAPI::instance().StreamSynchronize(stream)); | ||
| copied += chunk_size; | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
This change seems to needlessly introduce another level of intermediate bounce buffer. Why is it not possible to just use the existing setup and check the relevant error condition as part of curl.perform?
Contributor
Author
There was a problem hiding this comment.
Thank you for your attention, I have amended.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
A remote read could return success after libcurl completed even if the write callback received fewer bytes than requested. The easy retry path also reused the same callback offset between attempts, so a partial timed-out attempt could make the next attempt append at the wrong offset.
This keeps the existing host and device callback paths, restarts per-attempt byte state on retries, and rejects successful transfers that did not deliver the requested byte count.
Validation
pre-commit run --files cpp/include/kvikio/detail/remote_callback.hpp cpp/src/detail/remote_callback.cpp cpp/include/kvikio/shim/libcurl.hpp cpp/src/shim/libcurl.cpp cpp/src/remote_handle.cpp cpp/src/detail/multi_poll_reactor.cpp python/kvikio/tests/test_http_io.pypython3 -m py_compile python/kvikio/tests/test_http_io.pygit diff --checkNot run locally: the KvikIO Python/C++ test suite, because this checkout does not have an importable
kvikiopackage and CUDA tooling/GPU access are unavailable (nvccandnvidia-smiare not present).