S3 put_object returns {:error, :timeout} but uploads file correctly

[pulzzedavid]

When putting larger files taking more than few seconds to upload, AWS.S3.put_object call returns {:error, :timeout} even with timeout and recv_timeout configured

AWS.S3.put_object(client, bucket, file_name, %{"Body" => file, "ContentMD5" => md5}, [timeout: 50_000, recv_timeout: 50_000])
What is the correct way of executing request that takes longer to complete?

[onno-vos-dev]

@pulzzedavid Ill try and poke around the Elixir codebase over the weekend to see if I can get a more detailed answer to your question but depending on how large of a file you're dealing with the recommendation from AWS would be to use multipart uploads. How big is the file you're trying to upload as AWS recommends multipart for anything more than 100GB.

[pulzzedavid]

Thanks @onno-vos-dev. I am using files between 5MB - 70MB in size

[onno-vos-dev]

Sorry for dropping this... Could you try with something along the lines of: #168 (comment)

Would that resolve your issue? 🤔

[pulzzedavid]

I don't have the environment to try this right now. Will drop you a line if I am able to try this

…

On Fri, Dec 15, 2023 at 4:27 AM Onno Vos ***@***.***> wrote: Sorry for dropping this... Could you try with something along the lines of: #168 (comment) <#168 (comment)> Would that resolve your issue? 🤔 — Reply to this email directly, view it on GitHub <#154 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACQJUMWDENHSUNMVX6Y5EELYJQ62LAVCNFSM6AAAAAAVN3VEWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJXHAYDKMZZHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

--

------- David Jung CEO E: ***@***.*** ***@***.***>* P: +1 408.484.7780 C: +1 408.781.9384 Visit us at interactor.com *CONFIDENTIALITY NOTICE*: This e-mail and any documents attached to it may contain confidential or proprietary information or content. The transmission is intended solely for the information or use of the individuals addressed, or copied, as intended recipients. If you are not a named recipient, or you were otherwise sent this by mistake, you are hereby notified that any disclosure, copying, distribution or taking of any action as a result of or in reliance on the contents of this e-mail is strictly prohibited. If this message has been received in error, please delete it immediately and notify the sender by return e-mail. Please consider the environment before printing this e-mail.

[grant-zukowski-xumo]

I'm inconsistently seeing {:error, :timeout} as well but during "regular" executions. For example, listing distributions.

For creating a Medialive for example, the medialive gets created, but I get a {:error, :timeout} back, even though the resource was created.

[grant-zukowski-xumo]

@pulzzedavid I ended up switching our client to Finch and I've not seen a "timeout" since. We even ran 40 calls in a row that sometime produce timeouts and never got a single one. For anyone else having this problem I recommend changing the underlying client.