Skip to content

Add retry logging for replication task send attempts#9801

Merged
jiechenz merged 3 commits intomainfrom
jiechenz/add-retry-logs
Apr 8, 2026
Merged

Add retry logging for replication task send attempts#9801
jiechenz merged 3 commits intomainfrom
jiechenz/add-retry-logs

Conversation

@jiechenz
Copy link
Copy Markdown
Contributor

@jiechenz jiechenz commented Apr 3, 2026

What changed?

Added throttled warn logging inside the replication task send retry loop in stream_sender.go. Error messages are now prefixed with the step that failed (convert:, rate_limit:, send:).

Sample log output:

  {
    "level": "warn",
    "msg": "Replication task send retry",
    "wf-namespace-id": "abc123",
    "wf-id": "my-workflow",
    "task-id": 54321,
    "counter": 2,
    "error": "convert: context deadline exceeded"
  }

Why?

The replication_task_send_attempt inner retry loop was completely silent — no logs. The only logging happened after all retries were exhausted, making it impossible to diagnose transient retry.

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

Potential risks

n/a

@jiechenz jiechenz marked this pull request as ready for review April 3, 2026 18:04
@jiechenz jiechenz requested a review from a team as a code owner April 3, 2026 18:04
@jiechenz jiechenz merged commit 5afb8e4 into main Apr 8, 2026
46 checks passed
@jiechenz jiechenz deleted the jiechenz/add-retry-logs branch April 8, 2026 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants