Skip to content

[Bug]: ERROR:services.s3_toolkit:Error uploading file to S3: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header. #188

@suyashrai

Description

@suyashrai

Describe the bug

Hey team,

I’m encountering a problem when using the /v1/image/convert/video endpoint with S3 storage.

The video is created successfully, but upload fails with:

ERROR:services.s3_toolkit:Error uploading file to S3: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

This was working as expected until about a week ago. Nothing has changed in my S3 credentials or config. This was running via n8n, and currently full automation has stopped. Tried via Postman too, throwing same issue.

Logs show:

INFO:services.v1.image.convert.image_to_video:Downloaded image to /tmp/0a1eca09-e1ef-4f93-99c8-abb8d29b0ae5.png

INFO:services.v1.image.convert.image_to_video:Original image dimensions: 576x1024

INFO:services.v1.image.convert.image_to_video:Using scale dimensions: 4320:7680, output dimensions: 1080x1920

INFO:services.v1.image.convert.image_to_video:Video length: 10s, Frame rate: 25fps, Total frames: 250

INFO:services.v1.image.convert.image_to_video:Zoom speed: 0.03/s, Final zoom factor: 1.3

INFO:services.v1.image.convert.image_to_video:Running FFmpeg command: ffmpeg -framerate 25 -loop 1 -i /tmp/0a1eca09-e1ef-4f93-99c8-abb8d29b0ae5.png -vf scale=4320:7680,zoompan=z='min(1+(0.03*10)*on/250, 1.3)':d=250:x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=1080x1920,fps=25 -c:v libx264 -r 25 -t 10 -pix_fmt yuv420p /tmp/47380e0c-e98b-4357-9b3d-7eace8e2d6bb.mp4

INFO:services.v1.image.convert.image_to_video:Video created successfully: /tmp/47380e0c-e98b-4357-9b3d-7eace8e2d6bb.mp4

INFO:services.cloud_storage:Uploading file to cloud storage: /tmp/47380e0c-e98b-4357-9b3d-7eace8e2d6bb.mp4

ERROR:services.s3_toolkit:Error uploading file to S3: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

ERROR:services.cloud_storage:Error uploading file to cloud storage: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

ERROR:routes.v1.image.convert.image_to_video:Job 47380e0c-e98b-4357-9b3d-7eace8e2d6bb: Error processing image to video: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

Traceback (most recent call last):

  File "/app/routes/v1/image/convert/image_to_video.py", line 63, in image_to_video

    cloud_url = upload_file(output_filename)

  File "/app/services/cloud_storage.py", line 112, in upload_file

    url = provider.upload_file(file_path)

  File "/app/services/cloud_storage.py", line 87, in upload_file

    return upload_to_s3(file_path, self.endpoint_url, self.access_key, self.secret_key, self.bucket_name, self.region)

  File "/app/services/s3_toolkit.py", line 41, in upload_to_s3

    client.upload_fileobj(data, bucket_name, os.path.basename(file_path), ExtraArgs={'ACL': 'public-read'})

  File "/usr/local/lib/python3.9/site-packages/botocore/context.py", line 123, in wrapper

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.9/site-packages/boto3/s3/inject.py", line 675, in upload_fileobj

    return future.result()

  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 111, in result

    return self._coordinator.result()

  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 287, in result

    raise self._exception

  File "/usr/local/lib/python3.9/site-packages/s3transfer/tasks.py", line 142, in __call__

    return self._execute_main(kwargs)

  File "/usr/local/lib/python3.9/site-packages/s3transfer/tasks.py", line 165, in _execute_main

    return_value = self._main(**kwargs)

  File "/usr/local/lib/python3.9/site-packages/s3transfer/upload.py", line 796, in _main

    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)

  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 595, in _api_call

    return self._make_api_call(operation_name, kwargs)

  File "/usr/local/lib/python3.9/site-packages/botocore/context.py", line 123, in wrapper

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 1058, in _make_api_call

    raise error_class(parsed_response, operation_name)

botocore.exceptions.ClientError: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

INFO:routes.v1.image.convert.image_to_video:Job 12fdaf1d-d755-44e1-9961-2be8df0ab22f: Received image to video request for https://baserow.suyashrai.com/media/user_files/FPjLmdgcSNKIA9vcvJfZLrMTgiTxVgF5_7932f83b305c2206ea61195f8252f09af27a656ea528d44b1125cb17bfe0f38d.png

INFO:services.v1.image.convert.image_to_video:Downloaded image to /tmp/2b87e39d-8915-4e92-b193-7f570f9bc293.png

INFO:services.v1.image.convert.image_to_video:Original image dimensions: 576x1024

INFO:services.v1.image.convert.image_to_video:Using scale dimensions: 4320:7680, output dimensions: 1080x1920

INFO:services.v1.image.convert.image_to_video:Video length: 10s, Frame rate: 25fps, Total frames: 250

INFO:services.v1.image.convert.image_to_video:Zoom speed: 0.03/s, Final zoom factor: 1.3

INFO:services.v1.image.convert.image_to_video:Running FFmpeg command: ffmpeg -framerate 25 -loop 1 -i /tmp/2b87e39d-8915-4e92-b193-7f570f9bc293.png -vf scale=4320:7680,zoompan=z='min(1+(0.03*10)*on/250, 1.3)':d=250:x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=1080x1920,fps=25 -c:v libx264 -r 25 -t 10 -pix_fmt yuv420p /tmp/12fdaf1d-d755-44e1-9961-2be8df0ab22f.mp4

INFO:services.v1.image.convert.image_to_video:Video created successfully: /tmp/12fdaf1d-d755-44e1-9961-2be8df0ab22f.mp4

INFO:services.cloud_storage:Uploading file to cloud storage: /tmp/12fdaf1d-d755-44e1-9961-2be8df0ab22f.mp4

ERROR:services.s3_toolkit:Error uploading file to S3: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

ERROR:services.cloud_storage:Error uploading file to cloud storage: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

ERROR:routes.v1.image.convert.image_to_video:Job 12fdaf1d-d755-44e1-9961-2be8df0ab22f: Error processing image to video: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.

Traceback (most recent call last):

  File "/app/routes/v1/image/convert/image_to_video.py", line 63, in image_to_video

    cloud_url = upload_file(output_filename)

  File "/app/services/cloud_storage.py", line 112, in upload_file

    url = provider.upload_file(file_path)

  File "/app/services/cloud_storage.py", line 87, in upload_file

    return upload_to_s3(file_path, self.endpoint_url, self.access_key, self.secret_key, self.bucket_name, self.region)

  File "/app/services/s3_toolkit.py", line 41, in upload_to_s3

    client.upload_fileobj(data, bucket_name, os.path.basename(file_path), ExtraArgs={'ACL': 'public-read'})

  File "/usr/local/lib/python3.9/site-packages/botocore/context.py", line 123, in wrapper

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.9/site-packages/boto3/s3/inject.py", line 675, in upload_fileobj

    return future.result()

  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 111, in result

    return self._coordinator.result()

  File "/usr/local/lib/python3.9/site-packages/s3transfer/futures.py", line 287, in result

    raise self._exception

  File "/usr/local/lib/python3.9/site-packages/s3transfer/tasks.py", line 142, in __call__

    return self._execute_main(kwargs)

  File "/usr/local/lib/python3.9/site-packages/s3transfer/tasks.py", line 165, in _execute_main

    return_value = self._main(**kwargs)

  File "/usr/local/lib/python3.9/site-packages/s3transfer/upload.py", line 796, in _main

    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)

  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 595, in _api_call

    return self._make_api_call(operation_name, kwargs)

  File "/usr/local/lib/python3.9/site-packages/botocore/context.py", line 123, in wrapper

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 1058, in _make_api_call

    raise error_class(parsed_response, operation_name)

botocore.exceptions.ClientError: An error occurred (IncompleteBody) when calling the PutObject operation: You did not provide the number of bytes specified by the Content-Length HTTP header.


Reproduction Steps

Tried adding header manually in n8n, but I don't think that's related since the error is imploding from the python part.

Expected behavior

No response

Screenshots and relevant files

No response

Platform

Local

Would you like to work on this issue?

Yes

Checklist

  • I have searched the open issues for duplicates.
  • I have shown the entire traceback, if possible.

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions