Use the read buffer in UploadFile for the SFTP write packets#1798
Open
Rob-Hague wants to merge 2 commits into
Open
Use the read buffer in UploadFile for the SFTP write packets#1798Rob-Hague wants to merge 2 commits into
Rob-Hague wants to merge 2 commits into
Conversation
In SftpClient.UploadFile, a buffer is allocated to read from the given stream, and for each read, another array is allocated for the SFTP write packet (which consists of that data prepended with headers). This change effectively leaves space at the start of the buffer for the headers such that it can be used to assemble the packets without that per-packet array allocation. There are cleaner/more general ways to do this (e.g. for all packet types, leave space for the SSH headers as well), but this gets the most impact for about as much effort as I can be bothered with.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces allocations on the SFTP upload path by reusing the upload read buffer to assemble SSH_FXP_WRITE packets (instead of allocating a new packet-sized array per write). It does this by introducing a reusable write-request buffer that reserves space for SFTP headers up front, and by updating the SFTP send/write pipeline to use that buffer.
Changes:
- Introduces
SftpWriteRequestBufferto hold[length,type,requestId,handle,offset,dataLength,data]in a single buffer (optionally fromArrayPool<byte>). - Updates
SftpClient.InternalUploadFileandSftpSessionwrite request sending to use the shared buffer path forSSH_FXP_WRITE. - Updates
ISftpSession.RequestWritesignature/overloads and adjusts affected unit tests andSubsystemSession.SendDatacalls accordingly.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/Renci.SshNet.Tests/Classes/SubsystemSession_SendData_Connected.cs | Updates mock expectations to use SendData(byte[], offset, count) overload. |
| test/Renci.SshNet.Tests/Classes/Sftp/SftpSessionTest_DataReceived_SingleSftpMessageInSshDataMessage.cs | Updates channel send expectations to SendData(byte[], offset, count). |
| test/Renci.SshNet.Tests/Classes/Sftp/SftpSessionTest_DataReceived_MultipleSftpMessagesSplitOverMultipleSshDataMessages.cs | Updates channel send expectations to SendData(byte[], offset, count). |
| test/Renci.SshNet.Tests/Classes/Sftp/SftpSessionTest_DataReceived_MultipleSftpMessagesInSingleSshDataMessage.cs | Updates channel send expectations to SendData(byte[], offset, count). |
| test/Renci.SshNet.Tests/Classes/Sftp/SftpSessionTest_Connected_RequestStatVfs.cs | Updates channel send expectations to SendData(byte[], offset, count). |
| test/Renci.SshNet.Tests/Classes/Sftp/SftpSessionTest_Connected_RequestRead.cs | Updates channel send expectations to SendData(byte[], offset, count). |
| test/Renci.SshNet.Tests/Classes/Sftp/SftpFileStreamTest.cs | Adjusts ISftpSession.RequestWrite verification to match updated signature (no completion callback parameter). |
| test/Renci.SshNet.Tests/Classes/Sftp/Requests/SftpWriteRequestTest.cs | Updates tests to construct SftpWriteRequest from SftpWriteRequestBuffer and validates both serialization paths. |
| src/Renci.SshNet/SubsystemSession.cs | Adds SendData(byte[], offset, count) overload and routes existing SendData(byte[]) through it. |
| src/Renci.SshNet/SftpClient.cs | Uses SftpWriteRequestBuffer (pooled) in InternalUploadFile to avoid per-packet allocations. |
| src/Renci.SshNet/Sftp/SftpSession.cs | Special-cases SftpWriteRequest sending to avoid allocating a byte[]; refactors write request APIs to accept SftpWriteRequestBuffer. |
| src/Renci.SshNet/Sftp/Requests/SftpWriteRequestBuffer.cs | New helper type encapsulating an SFTP write packet buffer with header space and optional pooling. |
| src/Renci.SshNet/Sftp/Requests/SftpWriteRequest.cs | Refactors write request serialization to use SftpWriteRequestBuffer and expose data/handle as spans. |
| src/Renci.SshNet/Sftp/ISftpSession.cs | Updates write request API: removes callback from the wait-handle overload and adds a buffer-based callback overload. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In SftpClient.UploadFile, a buffer is allocated to read from the given stream, and for
each read, another array is allocated for the SFTP write packet (which consists of that
data prepended with headers). This change effectively leaves space at the start of the
buffer for the headers such that it can be used to assemble the packets without that
per-packet array allocation.
There are cleaner/more general ways to do this (e.g. for all packet types, leave space
for the SSH headers as well), but this gets the most impact for about as much effort as
I can be bothered with.