s3store: Allow customizing upload ID, object ID and bucket

[Acconut]

Previously, the upload ID included the multipart ID, which made fully customizing the upload ID (and thus upload URL) impossible. With this PR, we hope to allow full customization of the upload ID and even the destination object key.

Overall changes:

• Removes the multipart ID from the upload ID and instead loads it from the info file
• Allow customization of upload ID
• Allow customization of destination object key via Storage.Key
• Allow customization of destination bucket via Storage.Bucket

TODO:

• Manual testing
• Allow customization of S3 metadata
• Allow customization of S3 storage class
• Allow customization of Content-Type, Content-Encoding, Content-Language, Content-Disposition and Cache-Control

[mosi-kha]

Hi, I'm excited when this PR will merge.
the custom bucket name is what I need

[tgoyal63]

Hi @Acconut,
Really excited and waiting for so long for this to be merged.
Please let us know if there's any ETA for this.

Thanks ❤️

[mbrimmer83]

Is this work stalled? One thing that would be nice is to allow choosing the uuid version. I'd be happy to let tusd generate id's if I could set it to generate uuid v7.

[mbrimmer83]

case Image.create_image(conn, attrs) do
  {:ok, image} ->
     mime_type = Map.get(metadata, "filetype")
     ext = mime_to_ext(mime_type)

       json(conn, %{
           "ChangeFileInfo" => %{
               "ID" => "images/#{image.id}.#{ext}"
            }
       })

  {:error, msg} ->
    send_resp(conn, 500, "Failed to create image")
 end

When I do this the in the pre-create hook, file is uploaded to S3 inside the image folder with the id as name and extension. Is this functionality already available?

[Acconut]

When I do this the in the pre-create hook, file is uploaded to S3 inside the image folder with the id as name and extension. Is this functionality already available?

Partly, yes. The s3store respected values for upload ID from the pre-create for a long time already, but it currently always appends the multipart upload ID to form the upload URL. That allows you to control the location where the file is saved on S3 right now without waiting for this PR as long as you are fine with exposing that location in the upload URL.

[ryab001]

Hi @Acconut ,
We have this branch running for a few months already in our system. Works fine.
This looks like a pretty interesting feature for users.
Does it makes sense to get TODO customizations out of scope of this PR and move ready changes to Release?

Thanks.

[Maousuki]

I’m really excited about this feature! I can’t wait to start using it!

[zhangjiaxin281]

I urgently need this improvement and am looking forward to its release.