Failed to produce message to Kafka

[arkadicolson]

Environment

self-hosted (https://develop.sentry.dev/self-hosted/)

Steps to Reproduce

Issues always coming in

Expected Result

After a few days issues stop coming in

Actual Result

After some time, issues stop coming in. When I check the logs, I see the following on the relay:

relay-1  | 2026-04-28T06:57:15.746845Z ERROR relay_kafka::producer::utils: failed to produce message to Kafka (delivery callback) error=Message production error: MessageTimedOut (Local: Message timed out) error.sources=[MessageTimedOut (Local: Message timed out)] payload_len=25185 tags.topic="ingest-transactions"
relay-1  | 2026-04-28T06:57:15.746904Z ERROR relay_kafka::producer::utils: failed to produce message to Kafka (delivery callback) error=Message production error: MessageTimedOut (Local: Message timed out) error.sources=[MessageTimedOut (Local: Message timed out)] payload_len=280 tags.topic="ingest-metrics"
relay-1  | 2026-04-28T06:57:15.746923Z ERROR relay_kafka::producer::utils: failed to produce message to Kafka (delivery callback) error=Message production error: MessageTimedOut (Local: Message timed out) error.sources=[MessageTimedOut (Local: Message timed out)] payload_len=234 tags.topic="ingest-performance-metrics"

After restarting the Kafka and relay containers, everything works again for a few days. Then the same errors return.

Does anyone have an idea what might be causing this?

Product Area

Issues

Link

No response

DSN

No response

Version

26.4.1

[linear-code]

ENG-7537

[getsantry]

Assigning to @getsentry/support for routing ⏲️

[73VW]

We're experiencing the same issue. Restarting Kafka and the relays temporarily resolves the problem. The errors reappear within 24 hours.

This issue occures only since our last upgrade from 26.3.1 to 26.4.1.

[aldy505]

Hi! Can you share your kafka container logs as well?

I would guess, if this is happening between kafka and relay, there is a possibility it affects other containers as well. Most sentry/snuba consumer connects to Kafka.