Skip to content

Supabase Storage S3 API returns transformed HTML body, causing Content-Length mismatch in rclone #45584

@Pietrex

Description

@Pietrex

Bug report

When copying an HTML file from Supabase Storage using the S3-compatible API, the object body returned by GET appears to be transformed before being returned to the client.

This causes tools like rclone to fail, because the object metadata reports one size, but the body returned by the S3 API has a different size.

In my case, the object metadata reports:

[
  {
    "Path": "dev.html",
    "Name": "dev.html",
    "Size": 37322
    ...
  }
]

But reading the same object through rclone returns a larger body:

rclone cat supabase:dev.html --s3-list-version 2 | wc -c
# 37473

The expected size is:

37322

This causes rclone to fail during copy/sync:

net/http: HTTP/1.x transport connection broken: http: ContentLength=37322 with Body length 37473

Steps to reproduce

  1. Create an HTML file with <a href="mailto:test@example.com">test</a> inside.
  2. Upload a file to Supabase Storage
  3. Configure rclone <=> supabase connection
  4. Try to clone file:
rclone copyto supabase:dev.html ./dev-copy.html --s3-list-version 2

Command should fail with an error:

net/http: HTTP/1.x transport connection broken: http: ContentLength...
  1. Download file contents with cat
rclone cat supabase:dev.html > ./dev-copy.html --s3-list-version 2
  1. Inspecting the returned HTML shows that the original mailto: link has been transformed into something like:
href="/cdn-cgi/l/email-protection#..."

This looks like Cloudflare Email Address Obfuscation being applied to the object body.

  1. Using Cloudflare’s email_off comments prevents the mailto: link from being rewritten, but the comments themselves are removed from the response, so the returned body still differs from the stored object.

Example:

<!--email_off-->
<a href="mailto:test@example.com">test@example.com</a>
<!--/email_off-->

Expected behavior

The S3-compatible Storage API should return the exact raw object bytes that were uploaded.

For object storage, especially through the S3 API, GET should be byte-for-byte consistent with HEAD / listing metadata.

If the object metadata says Size: 37322, then the body returned by GET should contain exactly 37322 bytes.

Question

Is there a way to disable this kind of HTML/body transformation for Supabase Storage S3-compatible API responses?

If not, could the S3-compatible API be adjusted so that it always returns raw object bytes without Cloudflare/Scrape Shield transformations?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions