Skip to content

linkerd-proxy mangles error code / message / trailer forwarding for sufficiently large messages #15199

Description

@n-oden

What is the issue?

In some release after edge-25.2.1, linkerd-proxy began failing to forward gRPC status codes, error messages and trailers in cases where the total gRPC response trailer size was at or over roughly 12 kilobytes. The error message generated by the server is replaced by code 13, the message is replaced by stream terminated by RST_STREAM with error code: INTERNAL_ERROR, and no trailer information is forwarded.

This is tested on server version edge-26.4.2.

How can it be reproduced?

I have put together a full demonstration of this effect as a client-server combo and a helm chart:

https://github.com/odenio/echoerror

To install on your cluster, run:

helm upgrade --install echoerror ./chart/echoerror

To vary the amount of requested data in the trailers, add --set client.padMessageKb=<int>.

This is a toy server implementation that implements a single gRPC API, EchoError, which takes an EchoRequest message that lets the client specify the kind of result it wants:

message EchoRequest {
  int32 code = 1;
  string message = 2;
  int32 pad_message_kb = 3;
}

The client code continuously sends messages to the server, iterating over codes 0 through 16, generating a random message per request, and setting a configured padding size. The response from the server is then compared to the request and any mismatch is logged.

On our cluster, if pad_message_kb is set to 11 or below, no mismatch errors are noted. But if it is set to 12 or above, all requests produce mismatches:

2026/04/22 15:31:49 MISMATCH TRAILER: no x-echo-message trailer received for code=16
2026/04/22 15:31:49 MISMATCH PAD: no x-echo-pad trailer received for code=16
2026/04/22 15:31:49 MISMATCH CODE: sent=16 (Unauthenticated) got=13 (Internal) message="test error code 16 at 2026-04-22T15:31:49.23084851Z"
2026/04/22 15:31:49 MISMATCH MESSAGE: sent="test error code 16 at 2026-04-22T15:31:49.23084851Z" got="stream terminated by RST_STREAM with error code: INTERNAL_ERROR" code=16

Logs, error output, etc

[   158.165469s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.165506s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]
[   158.167073s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.167116s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]
[   158.168402s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.168451s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]
[   158.169892s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.169930s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]
[   158.171079s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.171121s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]
[   158.172517s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.172587s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]
[   158.173839s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.173877s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]
[   158.175163s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=http2 error error.sources=[stream error received: unexpected internal error encountered]
[   158.175201s]  INFO ThreadId(01) inbound:server{port=4143}:rescue{client.addr=10.4.26.79:49978}: linkerd_app_core::errors::respond: gRPC request failed error=client 10.4.26.79:49978: server: 10.4.26.68:4143: server 10.4.26.68:9090: service echoerror:9090: http2 error error.sources=[server 10.4.26.68:9090: service echoerror:9090: http2 error, http2 error, stream error received: unexpected internal error encountered]

output of linkerd check -o short

linkerd-version
---------------
‼ cli is up-to-date
    unsupported version channel: stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-version-cli for hints

control-plane-version
---------------------
‼ control plane is up-to-date
    is running version 26.4.2 but the latest edge version is 26.4.3
    see https://linkerd.io/2.14/checks/#l5d-version-control for hints
‼ control plane and cli versions match
    control plane running edge-26.4.2 but cli running stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-version-control for hints

linkerd-control-plane-proxy
---------------------------
‼ control plane proxies are up-to-date
    some proxies are not running the current version:
	* linkerd-destination-5c48449b76-pl8pd (edge-26.4.2)
	* linkerd-destination-5c48449b76-qg6pc (edge-26.4.2)
	* linkerd-destination-5c48449b76-s48x8 (edge-26.4.2)
	* linkerd-identity-649c455559-4xbgf (edge-26.4.2)
	* linkerd-identity-649c455559-m6klc (edge-26.4.2)
	* linkerd-identity-649c455559-ng57w (edge-26.4.2)
	* linkerd-proxy-injector-59f54fd677-6f9jc (edge-26.4.2)
	* linkerd-proxy-injector-59f54fd677-g5s66 (edge-26.4.2)
	* linkerd-proxy-injector-59f54fd677-q246s (edge-26.4.2)
    see https://linkerd.io/2.14/checks/#l5d-cp-proxy-version for hints
‼ control plane proxies and cli versions match
    linkerd-destination-5c48449b76-pl8pd running edge-26.4.2 but cli running stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-cp-proxy-cli-version for hints

linkerd-ha-checks
-----------------
‼ pod injection disabled on kube-system
    kube-system namespace needs to have the label config.linkerd.io/admission-webhooks: disabled if injector webhook failure policy is Fail
    see https://linkerd.io/2.14/checks/#l5d-injection-disabled for hints

linkerd-viz
-----------
‼ viz extension proxies are up-to-date
    some proxies are not running the current version:
	* grafana-65db7cc7d9-25v8q (edge-25.11.3)
	* metrics-api-5c4b8db764-n2dqj (edge-26.4.2)
	* tap-847dc9cd6b-6mm5t (edge-26.4.2)
	* tap-injector-7f445877b9-rmxqm (edge-26.4.2)
	* web-7f4b65b674-wfdj6 (edge-26.4.2)
    see https://linkerd.io/2.14/checks/#l5d-viz-proxy-cp-version for hints
‼ viz extension proxies and cli versions match
    grafana-65db7cc7d9-25v8q running edge-25.11.3 but cli running stable-2.14.10
    see https://linkerd.io/2.14/checks/#l5d-viz-proxy-cli-version for hints

Status check results are √

Environment

Kubernetes version 1.33.8-gke.1026000 on GKE

Possible solution

No response

Additional context

No response

Would you like to work on fixing this bug?

None

Metadata

Metadata

Assignees

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions