Skip to content

Fix: --upload-blobs without storage connections (#5)#6

Merged
jensens merged 1 commit into
mainfrom
fix/upload-blobs-standalone-s3
Mar 31, 2026
Merged

Fix: --upload-blobs without storage connections (#5)#6
jensens merged 1 commit into
mainfrom
fix/upload-blobs-standalone-s3

Conversation

@jensens
Copy link
Copy Markdown
Member

@jensens jensens commented Mar 31, 2026

Summary

  • Add standalone S3 CLI flags (--s3-bucket, --s3-endpoint-url, --s3-region, --s3-access-key, --s3-secret-key)
  • --upload-blobs creates an S3Client directly from flags — no source or destination storage connections opened
  • Can now run in parallel with an ongoing --deferred-blobs import (no PG contention)
  • Falls back to opening destination storage if S3 flags not provided

Fixes #5

Usage

zodb-convert --upload-blobs /tmp/manifest.tsv \
  --s3-bucket my-bucket \
  --s3-endpoint-url https://fsn1.your-objectstorage.com \
  --s3-access-key $KEY \
  --s3-secret-key $SECRET \
  -w 16

Test plan

  • 87 tests pass, 94.3% coverage

🤖 Generated with Claude Code

Add standalone S3 CLI flags (--s3-bucket, --s3-endpoint-url, etc.)
so --upload-blobs can run without opening source or destination
storages. This allows running blob uploads in parallel with an
ongoing --deferred-blobs import.

Falls back to opening the destination storage if S3 flags are not
provided (existing behavior).

Closes #5
@jensens jensens merged commit 3d5bd0e into main Mar 31, 2026
10 checks passed
@jensens jensens deleted the fix/upload-blobs-standalone-s3 branch March 31, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--upload-blobs opens source and destination storages unnecessarily

1 participant