net.http_post doesn't send bodies over 65536 bytes (64 KB)

[alxndrsn]

Bug report

When calling net.http_post() with a body over 65536 bytes, the first 65536 bytes are sent, and the request then hangs.

• I confirm this is a bug with Supabase, not with my own application.
• I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

When calling net.http_post() with a body over 65536 bytes, the first 65536 bytes are sent, and the request then hangs.

To Reproduce

1. run an HTTP server locally on port 1337 to accept POST requests at /
2. run this script:

docker exec -it --env PAGER='' supabase_db_server-v2 psql -U postgres \
-c "SELECT net.http_post('http://$(hostname -I | cut -d' ' -f1):1337', ('{\"val\":\"' || (SELECT string_agg(lpad(to_hex(n%256),2,'0') ,'') FROM generate_series(1, 65520/2) AS n)||'\"}')::JSONB);"

3. observe that it works ok

4. run this script (note the different series length):

docker exec -it --env PAGER='' supabase_db_server-v2 psql -U postgres \
-c "SELECT net.http_post('http://$(hostname -I | cut -d' ' -f1):1337', ('{\"val\":\"' || (SELECT string_agg(lpad(to_hex(n%256),2,'0') ,'') FROM generate_series(1, 65540/2) AS n)||'\"}')::JSONB);"

Note that the HTTP server will not receive the full request.

Expected behavior

Full request body should be sent.

Screenshots

If applicable, add screenshots to help explain your problem.

System information

• OS: [e.g. macOS, Windows]
• Browser (if applies) N/A
• Version of supabase-js: 2.34.3
• Version of Node.js: N/A

Additional context

Running locally with npx supabase start

[steve-chavez]

When calling net.http_post() with a body over 65536 bytes, the first 65536 bytes are sent,

@alxndrsn SFLR here. Can you provide a reproduction case for the above?

---

Might be related to #174

[alxndrsn]

When calling net.http_post() with a body over 65536 bytes, the first 65536 bytes are sent,

@alxndrsn SFLR here.

Hi @steve-chavez 👋 What's SFLR?

Can you provide a reproduction case for the above?

Is there something missing or unclear in the To Reproduce section above?

Might be related to #174

That looks likely - thanks for linking 👍

[steve-chavez]

Hi @steve-chavez 👋 What's SFLR?

@alxndrsn Sorry for late reply 😅

Is there something missing or unclear in the To Reproduce section above?

Ah, I missed the repro there (long command and I only saw the docker part). Will try it.

[nettum]

I can confirm this is a problem.

I've created several webhooks that listen to INSERT and UPDATE from the supabase dashboard.
This became a problem for me on UPDATE webhooks as the payload from the webhooks here have both record and old_record fields which will double the size of the webhook payload.
When the hook payload is over 65536(?) bytes it will not call the edge function and will fail silently.

I did some tests on my database to mock the trigger created by the supabase webhook:

This payload will go through and the edge function will be called:

SELECT length(
  jsonb_build_object(
    'type', 'INSERT',
    'table', '<myTable>',
    'schema', 'public',
    'record', to_jsonb(t)
  )::text
) as payload_size_bytes_insert
FROM <myTable> t
WHERE id = 1;

payload_size_bytes_insert | 12676

This payload will go through and the edge function will be called:

SELECT length(
  jsonb_build_object(
    'type', 'UPDATE',
    'table', '<myTable>',
    'schema', 'public',
    'record', to_jsonb(t),
    'old_record', to_jsonb(t)
  )::text
) as payload_size_bytes_update
FROM <myTable> t
WHERE id = 1;

payload_size_bytes_update | 25298

This payload will go through and the edge function will be called:

SELECT length(
  jsonb_build_object(
    'type', 'INSERT',
    'table', '<myTable>',
    'schema', 'public',
    'record', to_jsonb(t)
  )::text
) as payload_size_bytes_insert
FROM <myTable> t
WHERE id = 8;

payload_size_bytes_insert | 60470

This payload will NOT go through and it will fail silently without calling my edge function:

SELECT length(
  jsonb_build_object(
    'type', 'UPDATE',
    'table', '<myTable>',
    'schema', 'public',
    'record', to_jsonb(t),
    'old_record', to_jsonb(t)
  )::text
) as payload_size_bytes_update
FROM <myTable> t
WHERE id = 8;

payload_size_bytes_update | 120886

Since we're using the built in webhooks functionality, this fails silently, and we're not able to debug this ourself as something stops / fails before our edge function code is reached I would say this is something that should be looked into.

I think the workaround for now would be to not use supabase webhooks but create our own trigger that does not send that much data in the payload.

[alxndrsn]

Ah, I missed the repro there (long command and I only saw the docker part). Will try it.

Any luck?

[nettum]

After doing some more testing, for me this seems to only be the case when running supabase locally.
Testing with the same data in production, the request seems to go through.
Not sure if this is because of diff in versions I run locally vs. hosted supabase or if this is always the case. Anyone else tested on hosted supabase?

[alxndrsn]

After doing some more testing, for me this seems to only be the case when running supabase locally. Testing with the same data in production, the request seems to go through. Not sure if this is because of diff in versions I run locally vs. hosted supabase or if this is always the case. Anyone else tested on hosted supabase?

👍 Definitely a good idea to check version numbers. I couldn't find a way to ensure that the local version of supabase was in sync with the production version, and there were definitely differences.

[nettum]

For the record, I added this as a support ticket to Supabase. They confirmed the bug and verified that it only occured on their local test too - not when testing on a production env.
So hopefully this is just something that happen locally, regardless of supabase versions (but I can't be sure about this).

Here is the feedback from Supabase support:

Our internal engineering team acknowledges the issue and will be working on a fix. I don't have an ETA for when the fix will be released, but you can keep following the GitHub thread at #174.