Every object-store read in the mounts domain opens a brand-new aiohttp.ClientSession — and therefore a fresh TCP (and, on remote S3, TLS) connection with zero keep-alive. ObjectStore.get_object (api/oss/src/core/store/storage.py) does this per call: async with aiohttp.ClientSession() as session: .... The download-all archive path reads every file in a mount through get_object, so archiving a 5000-file mount opens ~5000 fresh connections. Against a remote TLS S3 endpoint that is on the order of 47 seconds of pure handshake time at 8-way concurrency — connection setup dominates the actual download for small-file mounts. The cost is negligible against the bundled local SeaweedFS (plain HTTP on a local socket); this bites on remote cloud stores.
The goal is to stop paying a fresh TCP/TLS handshake per object read on the archive path.
Follow-up from the #5400 backend review (fixes stacked in #5411/#5412), which added the download-all archive endpoint that makes this pattern first-order. Cited finding: review finding 3.6 (verified against miniopy-async 1.21.3 source — get_object requires a caller-supplied session and the current code creates a fresh one per read; the fresh session is a fresh connection, not just cheap object construction). Prioritize before a real multi-thousand-file mount hits a cloud store. The review report is not in the repo, so this summary stands alone.
Every object-store read in the mounts domain opens a brand-new
aiohttp.ClientSession— and therefore a fresh TCP (and, on remote S3, TLS) connection with zero keep-alive.ObjectStore.get_object(api/oss/src/core/store/storage.py) does this per call:async with aiohttp.ClientSession() as session: .... The download-all archive path reads every file in a mount throughget_object, so archiving a 5000-file mount opens ~5000 fresh connections. Against a remote TLS S3 endpoint that is on the order of 47 seconds of pure handshake time at 8-way concurrency — connection setup dominates the actual download for small-file mounts. The cost is negligible against the bundled local SeaweedFS (plain HTTP on a local socket); this bites on remote cloud stores.The goal is to stop paying a fresh TCP/TLS handshake per object read on the archive path.
aiohttp.ClientSessiononObjectStoreand use it forget_object. This is a small,storage.py-only change. The listing paths cannot reuse a session without patching miniopy-async (it hardwires a session per result page), so leave those as-is.Follow-up from the #5400 backend review (fixes stacked in #5411/#5412), which added the download-all archive endpoint that makes this pattern first-order. Cited finding: review finding 3.6 (verified against miniopy-async 1.21.3 source —
get_objectrequires a caller-supplied session and the current code creates a fresh one per read; the fresh session is a fresh connection, not just cheap object construction). Prioritize before a real multi-thousand-file mount hits a cloud store. The review report is not in the repo, so this summary stands alone.