Commit ac60243
committed
cluster_link/replication: retry deferred prefix truncation on commit
When a source partition is prefix-trimmed, the shadow partition has to
truncate to the same start offset. maybe_synchronize_start_offset()
defers that while the shadow hasn't replicated up to the source start
offset yet and (before this commit) relied on the next fetch_next() to
retry.
It's possible that after deferring the prefix-trim, we replicate up to
the source HWM so there isn't a next fetch_next(), and so our start
offset never proceeds.
This commit retries the synchronization when a batch commits, which is
exactly when the shadow high watermark advances past the offset the
truncation was waiting on, so it no longer depends on new source data
arriving.
This attempts to fix
ShadowLinkingReplicationTests.test_auto_prefix_trimming which is quite
flaky.1 parent d013af2 commit ac60243
2 files changed
Lines changed: 21 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
124 | 127 | | |
125 | 128 | | |
126 | 129 | | |
| |||
Lines changed: 18 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
408 | 408 | | |
409 | 409 | | |
410 | 410 | | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
411 | 429 | | |
0 commit comments