feat(attachments): streaming attachment transport + saveFileFromUri by khawarizmus · Pull Request #1039 · powersync-ja/powersync-js

khawarizmus · 2026-07-14T15:14:13Z

Summary

Addresses the streaming request in discussion #968. The AttachmentQueue currently round-trips every file body through an ArrayBuffer in JS memory (localStorage.readFile → remoteStorage.uploadFile, and the reverse on download), which causes memory pressure / OOM for large files on lower-end devices.

This models the remote side (transfer and delete) as a single adapter so implementations can stream file-URI to the network natively, never materializing the file in the JS heap.

What's included

AttachmentTransportAdapter (common) — owns all remote operations: upload/download/delete. Supply this to fully replace remoteStorage.
AttachmentQueue options are now a discriminated union — provide exactly one of remoteStorage or transportAdapter. Supplying both, or neither, is a compile-time type error (a runtime guard also covers plain-JS callers). When only remoteStorage is given, an internal buffered transport is composed from it automatically, so existing setups are unchanged.
saveFileFromUri — registers a file already on disk and queues it for upload without reading it into memory. Only available when the queue is configured with a StreamingLocalStorageAdapter (enforced in the type system).
StreamingLocalStorageAdapter — a subinterface of LocalStorageAdapter adding buffer-free moveFile; implemented by the Node, Expo, and RN-FS adapters.
ExpoFileSystemTransportAdapter (attachments-storage-react-native) — native transport using Expo's uploadAsync/downloadAsync (+ a deleteFile callback) for buffer-free transfer.
ReactNativeFSTransportAdapter (attachments-storage-react-native) — native transport using @dr.pogodin/react-native-fs's uploadFiles/downloadFile (+ a deleteFile callback).

Backward compatibility

Default behavior is unchanged. When no transportAdapter is supplied, a buffered transport is composed internally from localStorage/remoteStorage, and RemoteStorageAdapter is unchanged. Existing { localStorage, remoteStorage } setups still compile and behave identically.

One intended behavior change for the RemoteStorageAdapter: downloadFile now receives the record with its destination localUri populated (previously null for fresh downloads).

Generated with Claude, tested and reviewed manually.

changeset-bot · 2026-07-14T15:14:21Z

🦋 Changeset detected

Latest commit: 037d1fc

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 7 packages

Name	Type
@powersync/common	Minor
@powersync/attachments-storage-react-native	Minor
@powersync/node	Minor
@powersync/react-native	Patch
@powersync/tanstack-react-query	Patch
@powersync/web	Patch
@powersync/diagnostics-app	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

simolus3

I think the changes and the BufferedAttachmentTransport fallback are clean, but we can probably restructure the public interface a bit to get rid of runtime errors.

simolus3 · 2026-07-15T13:17:35Z

+ * Describes the HTTP request used to upload a file's bytes to remote storage.
+ * Typically points at a presigned URL.
+ */
+export interface ExpoUploadRequest {


Would it be possible to add an optional dependency on expo and then use expo types here?

I could not import the types from Expo. The dependency is already present.

simolus3 · 2026-07-16T07:28:49Z

+ * @experimental
+ * @alpha This is currently experimental and may change without a major version bump.


Do we even need to export this from common? I assume users wouldn't construct this directly, they'd rely on this being the default right?

That's true. I think there is no usage outside the common package for this. It's dropped.

simolus3 · 2026-07-16T07:31:12Z

+   * default {@link BufferedAttachmentTransport}, which delegates all three operations
+   * to it.
+   *
+   * Required unless a {@link AttachmentQueueOptions.transportAdapter} is provided. If


Can we encode this requirement in types? E.g. we could have export type AttachmentQueueOptions = { ... remaining fields ... } & ({ remoteStorage: RemoteStorageAdapter } | { transportAdapter: AttachmentTransportAdapter })

This has been addressed now.

simolus3 · 2026-07-16T07:32:11Z

+  /** Adapter for remote file storage operations. Undefined when a custom `transportAdapter` is used. */
+  readonly remoteStorage?: RemoteStorageAdapter;


remoteStorage is not used anywhere anymore. Given that attachments are still in beta and we're about to make a breaking change, let's just remove the field.

Removed. The remoteStorage option still works (it builds the default buffered transport), so apps that configure it are unaffected

simolus3 · 2026-07-16T07:39:53Z

+   * @returns Promise resolving to the created attachment record
+   * @throws Error if the local storage adapter does not support moving files
+   */
+  async saveFileFromUri({


Can we move common options into a shared interface and introduce a private method to implement saveFile and saveFileFromUri given that they're nearly the same?

Made the functions more DRY by extracting the common logic.

simolus3 · 2026-07-16T07:44:31Z

+    updateHook?: (transaction: Transaction, attachment: AttachmentRecord) => Promise<void>;
+  }): Promise<AttachmentRecord> {
+    if (!this.localStorage.moveFile) {
+      throw new Error('The configured local storage adapter does not support moveFile, required by saveFileFromUri.');


I would ideally also like this to be a static error.

I wonder if there's a way to make support for streaming downloads a subinterface of LocalStorageAdapter, and add a type parameter to AttachmentQueue to declare the type of local storage adapter used. Then we could use conditional types to declare that saveFileFromUri is only available when a stream-aware local adapter is used.

Additionally, it would also help with the option case above (we could require a RemoteAdapter if the local storage adapter is non-streaming only). Essentially, AttachmentTransportAdapter would then be a subinterface of LocalStorageAdapter and we'd add moveFile to it only.

This was addressed by introducing a new class StreamingLocalStorageAdapter which extends the LocalStorageAdapter and implements the moveFile function.

Now Node/Expo/RN all implement that class.

Model attachment byte-transfer as a single operation via AttachmentTransportAdapter, with a default BufferedAttachmentTransport that composes the existing local/remote adapters. Add AttachmentQueue.saveFileFromUri and an optional LocalStorageAdapter.moveFile for buffer-free registration of files already on disk, a native Expo ExpoFileSystemTransportAdapter, and moveFile in the Node and React Native local adapters. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Tag LocatedAttachmentRecord as @Alpha, use a qualified @link to AttachmentQueue.saveFile, and update etc/common.api.md for the new attachment transport API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…pter Native streaming transport for @dr.pogodin/react-native-fs (raw binary PUT via uploadFiles + downloadFile), mirroring ExpoFileSystemTransportAdapter, so non-Expo apps can transfer large attachments without buffering them in JS memory. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…pter Enhance AttachmentTransportAdapter to support file deletion operations. This includes updates to ExpoFileSystemTransportAdapter and ReactNativeFSTransportAdapter to implement the new deleteFile method. Additionally, adjustments were made to AttachmentQueue and BufferedAttachmentTransport to integrate the delete functionality into the attachment synchronization process.

…vulnerability audit This commit updates the websocket-driver dependency to version 0.7.5 in both pnpm-lock.yaml and pnpm-workspace.yaml to address security advisories. Additionally, the README.md for attachments-storage-react-native has been enhanced to clarify the functionality of local storage and streaming transport adapters, including usage examples and API details. # Conflicts: # pnpm-lock.yaml # pnpm-workspace.yaml

…er interfaces This commit refines the AttachmentQueue and transport adapter interfaces by consolidating configuration options and enhancing type safety. The `AttachmentQueueOptions` now requires either a `remoteStorage` or a `transportAdapter`, ensuring clarity in usage. Additionally, the `saveFile` and `saveFileFromUri` methods have been updated to utilize a unified `SaveAttachmentOptions` type, improving consistency across attachment operations. Documentation in the README.md has also been updated to reflect these changes.

…port streaming This commit refines the local storage adapter interfaces by replacing the `LocalStorageAdapter` with `StreamingLocalStorageAdapter` in multiple components, including `ExpoFileSystemStorageAdapter`, `ReactNativeFileSystemStorageAdapter`, and `NodeFileSystemAdapter`. The `AttachmentQueue` and related types have been updated to leverage the new streaming capabilities, ensuring that the `saveFileFromUri` method is only available when a streaming-capable adapter is used. Documentation has been adjusted to reflect these changes.

This commit updates the type definition for the `queue` variable in the attachments test file to explicitly use `AttachmentQueue<NodeFileSystemAdapter>`. This change enhances type safety and clarity in the test setup, aligning with recent refinements in the local storage adapter interfaces.

khawarizmus · 2026-07-23T01:01:22Z

@simolus3 Tests that are outside the scope of the PR are failing. But the PR is ready.

khawarizmus marked this pull request as draft July 14, 2026 15:54

khawarizmus force-pushed the attachment-transport branch from 509d298 to bf2d8cc Compare July 14, 2026 23:21

khawarizmus marked this pull request as ready for review July 16, 2026 06:20

khawarizmus requested review from joshuabrink, simolus3 and stevensJourney July 16, 2026 06:20

simolus3 reviewed Jul 16, 2026

View reviewed changes

khawarizmus and others added 8 commits July 22, 2026 21:03

fix(common): resolve api-extractor report and doc warnings

bcc9900

Tag LocatedAttachmentRecord as @Alpha, use a qualified @link to AttachmentQueue.saveFile, and update etc/common.api.md for the new attachment transport API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

khawarizmus force-pushed the attachment-transport branch from a56ffa6 to 037d1fc Compare July 23, 2026 00:47

khawarizmus requested a review from simolus3 July 23, 2026 00:51

		* @experimental
		* @alpha This is currently experimental and may change without a major version bump.

		/** Adapter for remote file storage operations. Undefined when a custom `transportAdapter` is used. */
		readonly remoteStorage?: RemoteStorageAdapter;

Uh oh!

Conversation

khawarizmus commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's included

Backward compatibility

Uh oh!

changeset-bot Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

simolus3 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

khawarizmus commented Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

khawarizmus commented Jul 14, 2026 •

edited

Loading

changeset-bot Bot commented Jul 14, 2026 •

edited

Loading