Title: Strange internal_redirect / buffer interactions
Description:
I have an envoy config with:
envoy.filters.http.buffer with max_request_bytes: 4194304- A route with
internal_redirect_action and internal_redirect_policy enabled
- A route to handle the internal redirect via dynamic_forward_proxy, and buffering disabled
When a request with body size <= 2MiB is internall redirected, everything works just fine.
Request bodies > 2MiB and <= 4MiB get the redirect passed back to the customer. The following appears in the logs with a 3MB request body:
envoy-1 | [2026-03-26 15:23:45.009][28][debug][router] [source/common/router/router.cc:1025] The request payload has at least 6000000 bytes data which exceeds buffer limit 4194304. Marking request as buffer overflowed to cancel internal redirects.
Request bodies over 4MiB result in a HTTP413 as expected.
It appears the buffer is getting filled twice, even though buffering is disabled on the internal redirect route.
My current workaround is to set per_request_buffer_limit_bytes on the vhost to 8MiB. It feels like this shouldn't be required, since buffering isn't enabled (or required by the backend) on the sub-request.
Repro steps:
Include sample requests, environment, etc. All data and inputs
required to reproduce the bug.
Use the config below, and some toy backends:
- server1 - redirect everything to server2
- server2 - just return the size of the request body
Send a request with body size 3MB
Config:
admin:
address:
socket_address: { address: 0.0.0.0, port_value: 9901 }
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 0.0.0.0, port_value: 10000 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
internal_only_headers:
- "X-Internal-Redirect"
virtual_hosts:
- name: local_service
domains: ["*"]
# Uncommenting this line fixes the problem
# per_request_buffer_limit_bytes: 8388608
routes:
- match:
prefix: "/"
headers:
- name: X-Internal-Redirect
present_match: false
route:
cluster: server1
internal_redirect_action: 'HANDLE_INTERNAL_REDIRECT'
internal_redirect_policy:
max_internal_redirects: 1
redirect_response_codes: [301]
response_headers_to_copy:
- 'X-Internal-Redirect'
- match:
prefix: /
headers:
- name: X-Internal-Redirect
present_match: True
route:
cluster: dynamic_forward_proxy_cluster
typed_per_filter_config:
envoy.filters.http.buffer:
"@type": type.googleapis.com/envoy.extensions.filters.http.buffer.v3.BufferPerRoute
disabled: true
http_filters:
- name: envoy.filters.http.dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.dynamic_forward_proxy.v3.FilterConfig
- name: envoy.filters.http.buffer
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.buffer.v3.Buffer
max_request_bytes: 4194304
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: dynamic_forward_proxy_cluster
lb_policy: CLUSTER_PROVIDED
cluster_type:
name: envoy.clusters.dynamic_forward_proxy
typed_config:
"@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfig
- name: server1
connect_timeout: 0.25s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: server1
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: server1
port_value: 12345Logs:
[2026-03-26 15:31:32.776][26][debug][conn_handler] [source/common/listener_manager/active_tcp_listener.cc:160] [Tags: "ConnectionId":"13"] new connection from 192.168.65.1:30028
[2026-03-26 15:31:32.776][26][debug][http] [source/common/http/conn_manager_impl.cc:394] [Tags: "ConnectionId":"13"] new stream
[2026-03-26 15:31:32.777][26][debug][http] [source/common/http/conn_manager_impl.cc:1291] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] request headers complete (end_stream=false):
':authority', 'localhost:10000'
':path', '/redirect_with_header'
':method', 'POST'
'user-agent', 'python-requests/2.32.5'
'accept-encoding', 'gzip, deflate, zstd'
'accept', '*/*'
'connection', 'keep-alive'
'content-length', '3000000'
[2026-03-26 15:31:32.777][26][debug][connection] [./source/common/network/connection_impl.h:101] [Tags: "ConnectionId":"13"] current connecting state: false
[2026-03-26 15:31:32.777][26][debug][http] [source/common/http/filter_manager.cc:1648] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] setting buffer limit to 4194304
[2026-03-26 15:31:32.808][26][debug][http] [source/common/http/conn_manager_impl.cc:1274] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] request end stream timestamp recorded
[2026-03-26 15:31:32.808][26][debug][router] [source/common/router/router.cc:537] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] cluster 'server1' match for URL '/redirect_with_header'
[2026-03-26 15:31:32.808][26][debug][router] [source/common/router/router.cc:809] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] router decoding headers:
':authority', 'localhost:10000'
':path', '/redirect_with_header'
':method', 'POST'
':scheme', 'http'
'user-agent', 'python-requests/2.32.5'
'accept-encoding', 'gzip, deflate, zstd'
'accept', '*/*'
'content-length', '3000000'
'x-forwarded-proto', 'http'
'x-request-id', '56247295-476a-436c-a492-07fa1101a0a3'
'x-envoy-expected-rq-timeout-ms', '15000'
[2026-03-26 15:31:32.808][26][debug][pool] [source/common/conn_pool/conn_pool_base.cc:330] [Tags: "ConnectionId":"6"] using existing fully connected connection
[2026-03-26 15:31:32.808][26][debug][pool] [source/common/conn_pool/conn_pool_base.cc:247] [Tags: "ConnectionId":"6"] creating stream
[2026-03-26 15:31:32.808][26][debug][router] [source/common/router/upstream_request.cc:599] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] pool ready
[2026-03-26 15:31:32.808][26][debug][router] [source/common/router/router.cc:1025] The request payload has at least 6000000 bytes data which exceeds buffer limit 4194304. Marking request as buffer overflowed to cancel internal redirects.
[2026-03-26 15:31:32.809][26][debug][http] [source/common/http/conn_manager_impl.cc:2004] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] Read-disabling downstream stream due to filter callbacks.
[2026-03-26 15:31:32.809][26][debug][http] [source/common/http/conn_manager_impl.cc:1994] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] Read-enabling downstream stream due to filter callbacks.
[2026-03-26 15:31:32.809][26][debug][connection] [source/common/network/connection_impl.cc:638] [Tags: "ConnectionId":"6"] onAboveWriteBufferHighWatermark
[2026-03-26 15:31:32.809][26][debug][http] [source/common/http/conn_manager_impl.cc:2004] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] Read-disabling downstream stream due to filter callbacks.
[2026-03-26 15:31:32.809][26][debug][client] [source/common/http/codec_client.cc:150] [Tags: "ConnectionId":"6"] encode complete
[2026-03-26 15:31:32.810][26][debug][connection] [source/common/network/connection_impl.cc:627] [Tags: "ConnectionId":"6"] onBelowWriteBufferLowWatermark
[2026-03-26 15:31:32.810][26][debug][http] [source/common/http/conn_manager_impl.cc:1994] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] Read-enabling downstream stream due to filter callbacks.
[2026-03-26 15:31:32.811][26][debug][client] [source/common/http/codec_client.cc:137] [Tags: "ConnectionId":"6"] response complete
[2026-03-26 15:31:32.811][26][debug][router] [source/common/router/router.cc:1687] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] upstream headers complete: end_stream=true
[2026-03-26 15:31:32.811][26][debug][router] [source/common/router/router.cc:1974] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] attempting internal redirect
[2026-03-26 15:31:32.811][26][debug][router] [source/common/router/router.cc:1133] Executing cleanup(): resetting retry_state_ and disabling timers
[2026-03-26 15:31:32.811][26][debug][http] [source/common/http/conn_manager_impl.cc:1948] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] encoding headers via codec (end_stream=true):
':status', '301'
'date', 'Thu, 26 Mar 2026 15:31:32 GMT'
'server', 'envoy'
'content-length', '0'
'location', 'http://server2:12346/redirect_with_header'
'x-internal-redirect', 'true'
'x-envoy-upstream-service-time', '1'
[2026-03-26 15:31:32.811][26][debug][http] [source/common/http/conn_manager_impl.cc:2063] [Tags: "ConnectionId":"13","StreamId":"282741079585230665"] Codec completed encoding stream.
[2026-03-26 15:31:32.811][26][debug][router] [source/common/router/router.cc:1133] Executing cleanup(): resetting retry_state_ and disabling timers
[2026-03-26 15:31:32.811][26][debug][pool] [source/common/http/http1/conn_pool.cc:61] [Tags: "ConnectionId":"6"] response complete
[2026-03-26 15:31:32.811][26][debug][pool] [source/common/conn_pool/conn_pool_base.cc:282] [Tags: "ConnectionId":"6"] destroying stream: 0 active remaining, readyForStream false, currentUnusedCapacity 1
[2026-03-26 15:31:32.812][26][debug][connection] [source/common/network/connection_impl.cc:777] [Tags: "ConnectionId":"13"] remote close
[2026-03-26 15:31:32.812][26][debug][connection] [source/common/network/connection_impl.cc:317] [Tags: "ConnectionId":"13"] closing socket: 0
Call Stack:
N/A
Title: Strange internal_redirect / buffer interactions
Description:
I have an envoy config with:
envoy.filters.http.bufferwithmax_request_bytes: 4194304internal_redirect_actionandinternal_redirect_policyenabledWhen a request with body size <= 2MiB is internall redirected, everything works just fine.
Request bodies > 2MiB and <= 4MiB get the redirect passed back to the customer. The following appears in the logs with a 3MB request body:
Request bodies over 4MiB result in a HTTP413 as expected.
It appears the buffer is getting filled twice, even though buffering is disabled on the internal redirect route.
My current workaround is to set
per_request_buffer_limit_byteson the vhost to 8MiB. It feels like this shouldn't be required, since buffering isn't enabled (or required by the backend) on the sub-request.Repro steps:
Use the config below, and some toy backends:
Send a request with body size 3MB
Config:
Logs:
Call Stack:
N/A