Skip to content

[Feature][High] Add streaming file upload support and async Python client #16

@numbers-official

Description

@numbers-official

Feature Improvements — High Priority

1. Streaming File Upload Support for Large Assets

Files:

  • ts/src/client.ts lines 73-123 (normalizeFile), lines 233-237
  • python/numbersprotocol_capture/client.py lines 68-106 (_normalize_file)

Description:
Both SDKs read the entire file contents into memory before uploading. In the TypeScript SDK, the data is then copied a second time into a new ArrayBuffer (lines 235-236) to avoid SharedArrayBuffer issues, meaning peak memory usage is roughly 2x the file size. For large media files (high-resolution video at hundreds of megabytes), this creates significant memory pressure and can cause out-of-memory crashes.

Expected impact:
Enables registration of large media files (100MB+) without crashing or excessive memory usage. Critical for video-heavy use cases.

Suggested implementation:

  • TypeScript: Accept ReadableStream as a FileInput variant. Use streaming SHA-256 via crypto.subtle.digest with incremental updates, and stream directly into FormData using a Blob constructed from the stream.
  • Python: Accept file-like objects (IO[bytes]). Use hashlib.sha256() with chunked update() calls. Pass the file handle directly to httpx as a streaming upload via files={"asset_file": (name, file_handle, mime_type)}.

2. Python SDK Async Support

File: python/numbersprotocol_capture/client.py (entire file)

Description:
The Python Capture class uses httpx.Client (synchronous) exclusively. There is no AsyncCapture or async variant. This is a significant limitation for modern Python applications using asyncio (FastAPI, aiohttp servers, etc.). The httpx library already provides httpx.AsyncClient with an identical API surface, making this a natural extension.

The TypeScript SDK is inherently async (all methods return Promise), so there is already a feature gap between the two SDKs in terms of concurrency support.

Expected impact:
Unblocks adoption in async Python applications and enables concurrent API operations (e.g., registering multiple assets in parallel).

Suggested implementation:

  • Create an AsyncCapture class that mirrors the Capture class but uses httpx.AsyncClient internally, with all methods as async def.
  • Alternatively, parameterize the existing Capture class to accept either sync or async client.
  • Add AsyncCapture to __init__.py exports and the feature parity checker.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions