Skip to content

Use chunked transfer encoding for tar uploads#1118

Merged
mishushakov merged 2 commits intomainfrom
mishushakov/single-stream-upload
Feb 6, 2026
Merged

Use chunked transfer encoding for tar uploads#1118
mishushakov merged 2 commits intomainfrom
mishushakov/single-stream-upload

Conversation

@mishushakov
Copy link
Copy Markdown
Member

@mishushakov mishushakov commented Feb 6, 2026

Simplify tar upload to use chunked transfer encoding instead of calculating Content-Length upfront. This eliminates the need to buffer the entire compressed tar stream in memory.

Changes:

  • Remove Content-Length header from upload request
  • Simplify tarFileStreamUpload to directly return the tar stream
  • Let Node.js fetch handle chunked encoding automatically

Note

Medium Risk
Changes the upload transport semantics (no Content-Length, chunked streaming), which can affect compatibility with servers/proxies expecting a known length and could alter upload failure modes.

Overview
Switches template tarball uploads to stream directly via chunked transfer encoding instead of precomputing a Content-Length.

tarFileStreamUpload is simplified to return a single tar/gzip stream (removing the extra pass that buffered/read the stream to compute size), and uploadFile no longer sets the Content-Length header, relying on Node fetch streaming behavior. Adds a changeset to release this as an e2b patch.

Written by Cursor Bugbot for commit 0ff7ad3. This will update automatically on new commits. Configure here.

Replace Content-Length header with chunked transfer encoding to avoid buffering entire tar stream in memory. Simplify tarFileStreamUpload to directly return the stream from tarFileStream.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Feb 6, 2026

🦋 Changeset detected

Latest commit: 0ff7ad3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
e2b Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@baptistecolle
Copy link
Copy Markdown

Just tested it and it works for me.

Thanks @mishushakov!

@mishushakov mishushakov force-pushed the mishushakov/single-stream-upload branch from 0697a0d to 884eb81 Compare February 6, 2026 16:26
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@mishushakov mishushakov marked this pull request as ready for review February 6, 2026 16:30
@noamzbr
Copy link
Copy Markdown
Contributor

noamzbr commented Feb 6, 2026

Works for me, thanks @mishushakov!

@mishushakov mishushakov merged commit 8efc53c into main Feb 6, 2026
16 checks passed
@mishushakov mishushakov deleted the mishushakov/single-stream-upload branch February 6, 2026 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants