|
11 | 11 | <a href="https://github.com/constructive-io/constructive/blob/main/LICENSE"><img height="20" src="https://img.shields.io/badge/license-MIT-blue.svg"/></a> |
12 | 12 | <a href="https://www.npmjs.com/package/@constructive-io/s3-utils"><img height="20" src="https://img.shields.io/github/package-json/v/constructive-io/constructive?filename=uploads%2Fs3-utils%2Fpackage.json"/></a> |
13 | 13 | </p> |
| 14 | + |
| 15 | +Unified S3 utilities for the Constructive ecosystem — client factory, file operations, presigned URLs, and bucket management. Built on AWS SDK v3 with support for multiple S3-compatible providers. |
| 16 | + |
| 17 | +## Features |
| 18 | + |
| 19 | +- **Multi-provider support** — AWS S3, MinIO, Cloudflare R2, Google Cloud Storage, DigitalOcean Spaces |
| 20 | +- **Presigned URLs** — generate secure PUT and GET URLs with configurable expiry |
| 21 | +- **File operations** — streaming upload, download, existence checks, and metadata retrieval |
| 22 | +- **Bucket management** — create buckets with provider-appropriate policies and CORS |
| 23 | +- **AWS SDK v3** — modern, modular SDK with tree-shaking support |
| 24 | + |
| 25 | +## Installation |
| 26 | + |
| 27 | +```sh |
| 28 | +npm install @constructive-io/s3-utils |
| 29 | +``` |
| 30 | + |
| 31 | +## Quick Start |
| 32 | + |
| 33 | +```typescript |
| 34 | +import { |
| 35 | + createS3Client, |
| 36 | + presignPutUrl, |
| 37 | + presignGetUrl, |
| 38 | + upload, |
| 39 | + download, |
| 40 | +} from '@constructive-io/s3-utils'; |
| 41 | + |
| 42 | +// 1. Create a client |
| 43 | +const client = createS3Client({ |
| 44 | + provider: 'minio', |
| 45 | + region: 'us-east-1', |
| 46 | + endpoint: 'http://minio:9000', |
| 47 | + accessKeyId: 'minioadmin', |
| 48 | + secretAccessKey: 'minioadmin', |
| 49 | +}); |
| 50 | + |
| 51 | +// 2. Upload a file via stream |
| 52 | +import { createReadStream } from 'fs'; |
| 53 | + |
| 54 | +const result = await upload({ |
| 55 | + client, |
| 56 | + bucket: 'my-bucket', |
| 57 | + key: 'docs/report.pdf', |
| 58 | + contentType: 'application/pdf', |
| 59 | + readStream: createReadStream('/path/to/report.pdf'), |
| 60 | +}); |
| 61 | + |
| 62 | +// 3. Generate a presigned download URL |
| 63 | +const url = await presignGetUrl(client, { |
| 64 | + bucket: 'my-bucket', |
| 65 | + key: 'docs/report.pdf', |
| 66 | +}); |
| 67 | +``` |
| 68 | + |
| 69 | +## API |
| 70 | + |
| 71 | +### `createS3Client(config)` |
| 72 | + |
| 73 | +Creates an `S3Client` with provider-specific defaults. |
| 74 | + |
| 75 | +```typescript |
| 76 | +import { createS3Client } from '@constructive-io/s3-utils'; |
| 77 | + |
| 78 | +const client = createS3Client({ |
| 79 | + provider: 'r2', |
| 80 | + region: 'auto', |
| 81 | + endpoint: 'https://account.r2.cloudflarestorage.com', |
| 82 | + accessKeyId: 'KEY', |
| 83 | + secretAccessKey: 'SECRET', |
| 84 | +}); |
| 85 | +``` |
| 86 | + |
| 87 | +**`StorageConnectionConfig`** |
| 88 | + |
| 89 | +| Field | Type | Required | Description | |
| 90 | +|---|---|---|---| |
| 91 | +| `provider` | `'s3' \| 'minio' \| 'r2' \| 'gcs' \| 'spaces'` | yes | Storage provider | |
| 92 | +| `region` | `string` | yes | S3 region (e.g. `"us-east-1"`) | |
| 93 | +| `endpoint` | `string` | non-AWS | Endpoint URL (required for all providers except `s3`) | |
| 94 | +| `accessKeyId` | `string` | yes | Access key ID | |
| 95 | +| `secretAccessKey` | `string` | yes | Secret access key | |
| 96 | +| `forcePathStyle` | `boolean` | no | Override path-style URL behavior | |
| 97 | + |
| 98 | +Path-style URLs are enabled automatically for `minio`, `r2`, and `gcs`. Virtual-hosted style is used for `s3` and `spaces`. |
| 99 | + |
| 100 | +Throws `S3ConfigError` on invalid configuration. |
| 101 | + |
| 102 | +### `presignPutUrl(client, options)` |
| 103 | + |
| 104 | +Generates a presigned PUT URL for uploading. The URL is locked to the specified key, content type, and content length. |
| 105 | + |
| 106 | +```typescript |
| 107 | +const url = await presignPutUrl(client, { |
| 108 | + bucket: 'my-bucket', |
| 109 | + key: 'uploads/photo.jpg', |
| 110 | + contentType: 'image/jpeg', |
| 111 | + contentLength: 102400, |
| 112 | + expiresIn: 900, // default: 900 (15 min) |
| 113 | +}); |
| 114 | +``` |
| 115 | + |
| 116 | +### `presignGetUrl(client, options)` |
| 117 | + |
| 118 | +Generates a presigned GET URL for downloading. Supports an optional `filename` for `Content-Disposition`. |
| 119 | + |
| 120 | +```typescript |
| 121 | +const url = await presignGetUrl(client, { |
| 122 | + bucket: 'my-bucket', |
| 123 | + key: 'uploads/photo.jpg', |
| 124 | + expiresIn: 3600, // default: 3600 (1 hour) |
| 125 | + filename: 'photo.jpg', // optional, triggers download prompt |
| 126 | +}); |
| 127 | +``` |
| 128 | + |
| 129 | +### `headObject(client, bucket, key)` |
| 130 | + |
| 131 | +Checks if an object exists and returns its metadata. Returns `null` if the object is not found. |
| 132 | + |
| 133 | +```typescript |
| 134 | +const meta = await headObject(client, 'my-bucket', 'uploads/photo.jpg'); |
| 135 | +// { contentType: 'image/jpeg', contentLength: 102400 } or null |
| 136 | +``` |
| 137 | + |
| 138 | +### `fileExists({ client, bucket, key })` |
| 139 | + |
| 140 | +Returns `true` if the object exists, `false` otherwise. |
| 141 | + |
| 142 | +```typescript |
| 143 | +const exists = await fileExists({ client, bucket: 'my-bucket', key: 'doc.pdf' }); |
| 144 | +``` |
| 145 | + |
| 146 | +### `upload({ client, bucket, key, readStream, contentType })` |
| 147 | + |
| 148 | +Streams a file to S3 using multipart upload. |
| 149 | + |
| 150 | +```typescript |
| 151 | +import { createReadStream } from 'fs'; |
| 152 | + |
| 153 | +const result = await upload({ |
| 154 | + client, |
| 155 | + bucket: 'my-bucket', |
| 156 | + key: 'uploads/video.mp4', |
| 157 | + contentType: 'video/mp4', |
| 158 | + readStream: createReadStream('/path/to/video.mp4'), |
| 159 | +}); |
| 160 | +// { Location, ETag, Bucket, Key } |
| 161 | +``` |
| 162 | + |
| 163 | +### `download({ client, bucket, key, writeStream })` |
| 164 | + |
| 165 | +Downloads an object from S3 and pipes it to a writable stream. |
| 166 | + |
| 167 | +```typescript |
| 168 | +import { createWriteStream } from 'fs'; |
| 169 | + |
| 170 | +await download({ |
| 171 | + client, |
| 172 | + bucket: 'my-bucket', |
| 173 | + key: 'uploads/video.mp4', |
| 174 | + writeStream: createWriteStream('/tmp/video.mp4'), |
| 175 | +}); |
| 176 | +``` |
| 177 | + |
| 178 | +### `uploadThrough({ client, bucket, key, contentType })` |
| 179 | + |
| 180 | +Returns a `PassThrough` stream that uploads to S3 as data is written. Emits an `'upload'` event with the result when complete. |
| 181 | + |
| 182 | +```typescript |
| 183 | +const pass = uploadThrough({ |
| 184 | + client, |
| 185 | + bucket: 'my-bucket', |
| 186 | + key: 'stream-data.csv', |
| 187 | + contentType: 'text/csv', |
| 188 | +}); |
| 189 | + |
| 190 | +pass.on('upload', (result) => { |
| 191 | + console.log('Upload complete:', result.Location); |
| 192 | +}); |
| 193 | + |
| 194 | +someReadable.pipe(pass); |
| 195 | +``` |
| 196 | + |
| 197 | +### `createS3Bucket(client, bucket, options)` |
| 198 | + |
| 199 | +Creates a bucket with provider-appropriate policies and CORS configuration. |
| 200 | + |
| 201 | +- **MinIO**: read-only public access policy (list + get) |
| 202 | +- **S3/GCS**: full-access policy with CORS rules for browser uploads |
| 203 | + |
| 204 | +```typescript |
| 205 | +const { success } = await createS3Bucket(client, 'my-bucket', { |
| 206 | + provider: 'minio', |
| 207 | +}); |
| 208 | +``` |
| 209 | + |
| 210 | +Returns `{ success: true }` if the bucket was created or already exists. |
| 211 | + |
| 212 | +## Related Packages |
| 213 | + |
| 214 | +| Package | Description | |
| 215 | +|---|---| |
| 216 | +| [`@constructive-io/s3-streamer`](../s3-streamer) | Streaming uploads with automatic content-type detection and metadata extraction | |
| 217 | +| [`@constructive-io/etag-stream`](../etag-stream) | ETag generation from streams | |
| 218 | +| [`@constructive-io/content-type-stream`](../content-type-stream) | Magic-byte content-type detection | |
| 219 | +| [`@constructive-io/uuid-hash`](../uuid-hash) | Deterministic UUID generation from file content | |
| 220 | + |
| 221 | +## License |
| 222 | + |
| 223 | +MIT |
0 commit comments