feat: allow multipart file uploads to be consumed as streams by Copilot · Pull Request #4424 · graphql-hive/graphql-yoga

Copilot · 2026-03-30T12:57:23Z

codesandbox · 2026-03-30T12:57:26Z

Review or Edit in CodeSandbox

Open the branch in Web Editor • VS Code • Insiders

Open Preview

Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/09f0bf95-0c9b-47b9-9011-39669178dcd0

ardatan · 2026-03-30T13:23:08Z

@@ -1,3 +1,5 @@
+import { Readable } from 'node:stream';
+import Busboy, { BusboyFileStream } from '@fastify/busboy';


@copilot We don't want this to be Node.js specific.

Replaced the @fastify/busboy + node:stream implementation with a self-contained WHATWG-streams-based multipart parser that uses only standard APIs (ReadableStream, TextDecoder, TextEncoder, Uint8Array). It works in Node.js, Cloudflare Workers, Deno, and browsers alike. The @fastify/busboy dependency has been removed from package.json. Commit: 040cdf1.

…HATWG implementation Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/fd8629e4-1198-4bf0-8922-692b34316f43

…allback Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/fd8629e4-1198-4bf0-8922-692b34316f43

theguild-bot · 2026-03-30T13:41:13Z

💻 Website Preview

The latest changes are available as preview in: https://59469410.envelop.pages.dev

Copilot · 2026-03-30T13:42:56Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

checkpoint.prisma.io
- Triggering command: /opt/hostedtoolcache/node/24.14.0/x64/bin/node /opt/hostedtoolcache/node/24.14.0/x64/bin/node /home/REDACTED/work/graphql-yoga/graphql-yoga/node_modules/.pnpm/prisma@6.19.0_typescript@5.9.3/node_modules/prisma/build/child {"product":"prisma","version":"6.19.0","cli_install_type":"local","information":"","local_timestamp":"2026-03-30T13:36:40Z","project_hash":"75b1e306","cli_path":"/home/REDACTED/work/graphql-yoga/graphql-yoga/examples/hackernews/node_modules/prisma/build/ind (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

ardatan

Make sure CI doesn't fail

theguild-bot · 2026-03-30T13:46:19Z

💻 Website Preview

The latest changes are available as preview in: https://f4241516.envelop.pages.dev

…ix TS2322 build error Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/902c7739-006a-40e5-8bc7-6ec571ff493f

Copilot · 2026-03-30T13:58:48Z

Make sure CI doesn't fail

Fixed. The CI failure was a TypeScript 5.9 error — Uint8Array<ArrayBufferLike> (returned by ReadableStreamDefaultReader.read()) was not assignable to Uint8Array<ArrayBuffer> (the previous parameter type of concatBytes/indexOfBytes). Updated both helper functions to accept Uint8Array<ArrayBufferLike>. Commit: 70473b3.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

checkpoint.prisma.io
- Triggering command: /opt/hostedtoolcache/node/24.14.0/x64/bin/node /opt/hostedtoolcache/node/24.14.0/x64/bin/node /home/REDACTED/work/graphql-yoga/graphql-yoga/node_modules/.pnpm/prisma@6.19.0_typescript@5.9.3/node_modules/prisma/build/child {"product":"prisma","version":"6.19.0","cli_install_type":"local","information":"","local_timestamp":"2026-03-30T13:51:38Z","project_hash":"75b1e306","cli_path":"/home/REDACTED/work/graphql-yoga/graphql-yoga/examples/hackernews/node_modules/prisma/build/ind (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

ardatan · 2026-03-30T14:16:09Z

  }
 });
+
+describe('file uploads (streaming mode)', () => {


@copilot make sure it is streamed not buffered first then consumed. So files should be lazily loaded right? So let's test it

theguild-bot · 2026-03-30T14:16:11Z

💻 Website Preview

The latest changes are available as preview in: https://6ff8a251.envelop.pages.dev

theguild-bot · 2026-03-30T14:17:07Z

💻 Website Preview

The latest changes are available as preview in: https://pr-4424.graphql-yoga.pages.dev

…tion Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/c66673b2-a216-4b18-8262-5016670052e5 Co-authored-by: ardatan <20847995+ardatan@users.noreply.github.com>

ardatan · 2026-03-30T14:17:33Z

+- **Only available in Node.js.** In edge runtimes the standard `request.formData()` path is used
+  regardless of this option.
+
 ## Configuring Multipart Request Processing (only for Node.js)


@copilot Implement this in the parser too so we don't have this node.js specific and have this part of our new parser. Update the configuraiton api etc for that

theguild-bot · 2026-03-30T14:17:52Z

✅ Yoga Benchmark Results

     ✓ no_errors{mode:graphql}
     ✓ expected_result{mode:graphql}
     ✓ no_errors{mode:graphql-jit}
     ✓ expected_result{mode:graphql-jit}
     ✓ no_errors{mode:graphql-response-cache}
     ✓ expected_result{mode:graphql-response-cache}
     ✓ no_errors{mode:graphql-no-parse-validate-cache}
     ✓ expected_result{mode:graphql-no-parse-validate-cache}
     ✓ no_errors{mode:uws}
     ✓ expected_result{mode:uws}

     checks.......................................: 100.00% ✓ 683308      ✗ 0     
     data_received................................: 2.8 GB  18 MB/s
     data_sent....................................: 116 MB  774 kB/s
     http_req_blocked.............................: avg=1.38µs   min=901ns    med=1.18µs   max=336.46µs p(90)=1.78µs   p(95)=2.06µs  
     http_req_connecting..........................: avg=2ns      min=0s       med=0s       max=157.84µs p(90)=0s       p(95)=0s      
     http_req_duration............................: avg=355.26µs min=214.85µs med=337.65µs max=19.81ms  p(90)=474.45µs p(95)=494.41µs
       { expected_response:true }.................: avg=355.26µs min=214.85µs med=337.65µs max=19.81ms  p(90)=474.45µs p(95)=494.41µs
     ✓ { mode:graphql-jit }.......................: avg=272.33µs min=214.85µs med=263.55µs max=19.81ms  p(90)=292.56µs p(95)=306.61µs
     ✓ { mode:graphql-no-parse-validate-cache }...: avg=496.95µs min=425.93µs med=479.29µs max=6.94ms   p(90)=516.92µs p(95)=541.15µs
     ✓ { mode:graphql-response-cache }............: avg=335.75µs min=266.61µs med=325.31µs max=9.87ms   p(90)=363.12µs p(95)=377.91µs
     ✓ { mode:graphql }...........................: avg=368.65µs min=296.07µs med=350.49µs max=16.48ms  p(90)=396.06µs p(95)=435.15µs
     ✓ { mode:uws }...............................: avg=358.29µs min=291.26µs med=344.85µs max=6.43ms   p(90)=374.98µs p(95)=388.98µs
     http_req_failed..............................: 0.00%   ✓ 0           ✗ 341654
     http_req_receiving...........................: avg=32.86µs  min=16.47µs  med=31.98µs  max=3.47ms   p(90)=38.51µs  p(95)=40.93µs 
     http_req_sending.............................: avg=8.21µs   min=5.7µs    med=7.06µs   max=452.89µs p(90)=11.25µs  p(95)=13.69µs 
     http_req_tls_handshaking.....................: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting.............................: avg=314.18µs min=182.66µs med=298.27µs max=19.69ms  p(90)=433.16µs p(95)=452.48µs
     http_reqs....................................: 341654  2277.666358/s
     iteration_duration...........................: avg=434.2µs  min=280.81µs med=414.67µs max=20.43ms  p(90)=554.64µs p(95)=575.83µs
     iterations...................................: 341654  2277.666358/s
     vus..........................................: 1       min=1         max=1   
     vus_max......................................: 2       min=2         max=2

phal0r · 2026-04-10T13:44:57Z

hey @ardatan ,

I saw you started working on this. In the meantime I implemented with help of Claude a GatewayPlugin and a YogaPlugin, that allows for streaming. In case you are interested, I can share the code.

ardatan · 2026-04-10T13:49:19Z

Sure @phal0r ! I'd love to see it!

…st, fix dead assignments Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/6298cf19-6020-4a6c-a7dc-61e237ef133f

Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/6298cf19-6020-4a6c-a7dc-61e237ef133f

theguild-bot · 2026-04-10T14:51:51Z

💻 Website Preview

The latest changes are available as preview in: https://e8454ba2.envelop.pages.dev

theguild-bot · 2026-04-10T14:52:33Z

💻 Website Preview

The latest changes are available as preview in: https://pr-4424.graphql-yoga.pages.dev

…; align error message format Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/a8adc1de-80f2-43ac-8de0-42de2415eaf1

theguild-bot · 2026-04-12T20:52:43Z

💻 Website Preview

The latest changes are available as preview in: https://ba9b7d0a.envelop.pages.dev

theguild-bot · 2026-04-12T20:53:34Z

💻 Website Preview

The latest changes are available as preview in: https://pr-4424.graphql-yoga.pages.dev

phal0r · 2026-04-13T10:20:38Z

Sure thing @ardatan !

Some additional notes: I implemented one GatewayPlugin for the hive-gateway and one YogaPlugin for the yoga-graphql server, although I know that the gateway uses yoga under the hood. The implementations are different due to different contexts: For the gateway I assume, that all subgraphs run in the same cluster/network/infra, so usually network is stable, fast and reliable. Thus the stream is just piped through from incoming network to outgoing network. For yoga I choose to write the file to disk and create a read stream from the temporarily stored file, because usually one wants to send this to some object storage with unreliable network over internet.

Of course this is only an inspiration for a final solution, but would love to see those implemented and remove my custom plugins :)

This is the one for Yoga: https://gist.github.com/phal0r/b25878bd83e10b576529f59f6bbb16fc

this is the one for the Gateway: https://gist.github.com/phal0r/507e7cd3b67230b2fd7fbc14efff9da0

I am running both apps with Bun and it works well there. I tested with files > 1GB and container limits of 1GB to ensure it really works. I also created some end2end tests with real file uploads through the network to rule out any inconsistencies.

Along the way were other problems, like the minio js client offering streaming, but being buggy (does not cater for backpressure from the network and thus still loaded the whole in a memory buffer :D )

At the moment this system runs reliable in my tests with big uploads.

Initial plan

26c1a45

Copilot AI assigned Copilot and ardatan Mar 30, 2026

Copilot AI linked an issue Mar 30, 2026 that may be closed by this pull request

Allow to process multipart uploads as streams to reduce memory usage #4409

Open

Copilot started work on behalf of ardatan March 30, 2026 12:57 View session

Copilot AI added 3 commits March 30, 2026 13:16

feat: add streaming multipart upload support via multipart.stream option

00bbc2e

Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/09f0bf95-0c9b-47b9-9011-39669178dcd0

chore: remove accidentally committed .cache files

7633bf3

Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/09f0bf95-0c9b-47b9-9011-39669178dcd0

fix: address code review - safe ArrayBuffer slice and cleaner tests

3e782de

Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/09f0bf95-0c9b-47b9-9011-39669178dcd0

ardatan reviewed Mar 30, 2026

View reviewed changes

Copilot AI changed the title ~~[WIP] Add support for processing multipart uploads as streams~~ feat: allow multipart file uploads to be consumed as streams Mar 30, 2026

Copilot AI requested a review from ardatan March 30, 2026 13:24

Copilot finished work on behalf of ardatan March 30, 2026 13:25

Copilot started work on behalf of ardatan March 30, 2026 13:25 View session

refactor: replace busboy-based streaming parser with cross-platform W…

0bb4c7b

…HATWG implementation Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/fd8629e4-1198-4bf0-8922-692b34316f43

github-code-quality Bot found potential problems Mar 30, 2026

View reviewed changes

Comment thread packages/graphql-yoga/src/plugins/request-parser/post-multipart.ts Fixed

Comment thread packages/graphql-yoga/src/plugins/request-parser/post-multipart.ts Fixed

fix: use DASH constant instead of magic bytes, add fileName nullish f…

040cdf1

…allback Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/fd8629e4-1198-4bf0-8922-692b34316f43

theguild-bot deployed to preview March 30, 2026 13:41 View deployment

github-code-quality Bot found potential problems Mar 30, 2026

View reviewed changes

Comment thread packages/graphql-yoga/src/plugins/request-parser/post-multipart.ts Fixed

Comment thread packages/graphql-yoga/src/plugins/request-parser/post-multipart.ts Fixed

Copilot finished work on behalf of ardatan March 30, 2026 13:43

ardatan requested changes Mar 30, 2026

View reviewed changes

Copilot started work on behalf of ardatan March 30, 2026 13:45 View session

theguild-bot deployed to preview March 30, 2026 13:46 View deployment

fix: use Uint8Array<ArrayBufferLike> in concatBytes/indexOfBytes to f…

70473b3

…ix TS2322 build error Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/902c7739-006a-40e5-8bc7-6ec571ff493f

github-code-quality Bot found potential problems Mar 30, 2026

View reviewed changes

Comment thread packages/graphql-yoga/src/plugins/request-parser/post-multipart.ts Fixed

Comment thread packages/graphql-yoga/src/plugins/request-parser/post-multipart.ts Fixed

Copilot AI requested a review from ardatan March 30, 2026 14:00

ardatan reviewed Mar 30, 2026

View reviewed changes

theguild-bot deployed to preview March 30, 2026 14:17 View deployment

docs: remove Node.js-specific framing from streaming file uploads sec…

b6b1f16

…tion Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/c66673b2-a216-4b18-8262-5016670052e5 Co-authored-by: ardatan <20847995+ardatan@users.noreply.github.com>

ardatan reviewed Mar 30, 2026

View reviewed changes

Copilot AI requested a review from ardatan March 30, 2026 14:19

Copilot finished work on behalf of ardatan March 30, 2026 14:19

Copilot started work on behalf of theguild-bot March 30, 2026 14:19 View session

Copilot AI requested a review from theguild-bot March 30, 2026 14:19

Copilot stopped work on behalf of theguild-bot due to an error March 30, 2026 14:19
An unexpected error occurred. For more details, see the detailed logs in GitHub Actions.

Copilot started work on behalf of ardatan April 10, 2026 13:51 View session

Copilot AI added 2 commits April 10, 2026 14:03

feat: make streaming default, add limits support, add lazy-loading te…

063dcac

…st, fix dead assignments Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/6298cf19-6020-4a6c-a7dc-61e237ef133f

refactor: deduplicate fileSize error creation and fix docs runtime claim

ebf613d

Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/6298cf19-6020-4a6c-a7dc-61e237ef133f

Copilot finished work on behalf of ardatan April 10, 2026 14:06

theguild-bot deployed to preview April 10, 2026 14:51 View deployment

theguild-bot deployed to preview April 10, 2026 14:52 View deployment

Copilot started work on behalf of ardatan April 10, 2026 20:38 View session

fix: update incremental-delivery test to use new multipart.limits API…

abad80d

…; align error message format Agent-Logs-Url: https://github.com/graphql-hive/graphql-yoga/sessions/a8adc1de-80f2-43ac-8de0-42de2415eaf1

Copilot finished work on behalf of ardatan April 10, 2026 20:45

theguild-bot deployed to preview April 12, 2026 20:52 View deployment

theguild-bot deployed to preview April 12, 2026 20:53 View deployment

		@@ -1,3 +1,5 @@
		import { Readable } from 'node:stream';
		import Busboy, { BusboyFileStream } from '@fastify/busboy';

Conversation

Copilot AI commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codesandbox Bot commented Mar 30, 2026

Review or Edit in CodeSandbox

Uh oh!

ardatan Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

theguild-bot commented Mar 30, 2026

💻 Website Preview

Uh oh!

Uh oh!

Uh oh!

Copilot AI commented Mar 30, 2026

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

ardatan left a comment

Choose a reason for hiding this comment

Uh oh!

theguild-bot commented Mar 30, 2026

💻 Website Preview

Uh oh!

Copilot AI commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Uh oh!

Uh oh!

ardatan Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

theguild-bot commented Mar 30, 2026

💻 Website Preview

Uh oh!

theguild-bot commented Mar 30, 2026

💻 Website Preview

Uh oh!

ardatan Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

theguild-bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Yoga Benchmark Results

Uh oh!

phal0r commented Apr 10, 2026

Uh oh!

ardatan commented Apr 10, 2026

Uh oh!

theguild-bot commented Apr 10, 2026

💻 Website Preview

Uh oh!

theguild-bot commented Apr 10, 2026

💻 Website Preview

Uh oh!

theguild-bot commented Apr 12, 2026

💻 Website Preview

Uh oh!

theguild-bot commented Apr 12, 2026

💻 Website Preview

Uh oh!

phal0r commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Mar 30, 2026 •

edited

Loading

Copilot AI commented Mar 30, 2026 •

edited

Loading

theguild-bot commented Mar 30, 2026 •

edited

Loading