feat: support concurrent chunk uploads#116
Conversation
Greptile SummaryThis PR refactors the chunked file upload logic in both
Confidence Score: 3/5The concurrent upload logic introduces real correctness risks that have been flagged but not yet addressed; merging as-is could silently corrupt a resumed upload after a transient failure. Three substantive defects identified in prior review rounds are still present: (1) when any uploadChunk call throws, Future.wait surfaces the error but remaining workers keep running, mutating shared state and potentially landing chunks on the server so a retry resumes from the wrong offset; (2) finalResponse is only set inside uploadNext, so if isUploadComplete never fires the caller receives the first chunk response instead of the completed upload metadata; (3) the first sequential chunk in client_io.dart still opens and closes its own RandomAccessFile independently. lib/src/client_browser.dart and lib/src/client_io.dart - specifically the uploadNext worker loop and Future.wait call in both files. Important Files Changed
Reviews (6): Last reviewed commit: "Commit from GitHub Actions (Format and p..." | Re-trigger Greptile |
| } | ||
|
|
||
| final concurrency = min(8, chunks.length); | ||
| await Future.wait(List.generate(concurrency, (_) => uploadNext())); |
There was a problem hiding this comment.
Uncancelled workers continue uploading after a failure
Future.wait with the default eagerError: true propagates the first error immediately, but does not cancel the remaining uploadNext futures — they keep running on the event loop. Concretely: if chunk N receives a 5xx from the server, the caller's await throws, but up to 7 other workers are still sending HTTP requests. onProgress keeps firing on the closed-over closures, and completedChunks/lastResponse keep mutating. If the caller catches the error and retries, the resume-offset query will see a higher chunksUploaded than anticipated (those background requests may have been accepted by the server), leading to duplicate-chunk or skipped-chunk scenarios. The sequential implementation avoided this entirely because a thrown exception exited the loop immediately. The same issue is present in client_io.dart at the equivalent line.
This PR updates the SDK to support concurrent chunk uploads.