Commit 0e2c06c
fix(cdk): emit interim heartbeat state for global-cursor substreams with empty parent state
During a long parent walk with mostly-empty children, a
global_substream_cursor substream without incremental_dependency emitted no
RECORD and no STATE for the whole walk, tripping the Airbyte source heartbeat
(86400s). ConcurrentPerPartitionCursor._emit_state_message suppressed the
throttled checkpoint STATE whenever the parent state was empty, so the only
keepalive signal was withheld exactly when it was needed.
Remove the empty-parent guard so the existing 600s-throttled checkpoint STATE
is emitted even when parent state is empty. The interim state carries the
stable (unadvanced) global cursor, so it is inert on resume (no data loss, no
re-read regression). The 600s throttle is unchanged.
Co-Authored-By: bot_apk <apk@cognition.ai>1 parent d828c57 commit 0e2c06c
2 files changed
Lines changed: 46 additions & 4 deletions
File tree
- airbyte_cdk/sources/declarative/incremental
- unit_tests/sources/declarative/incremental
Lines changed: 0 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
298 | 298 | | |
299 | 299 | | |
300 | 300 | | |
301 | | - | |
302 | | - | |
303 | | - | |
304 | 301 | | |
305 | 302 | | |
306 | 303 | | |
| |||
Lines changed: 46 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1021 | 1021 | | |
1022 | 1022 | | |
1023 | 1023 | | |
1024 | | - | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
1025 | 1029 | | |
1026 | 1030 | | |
1027 | 1031 | | |
| |||
3609 | 3613 | | |
3610 | 3614 | | |
3611 | 3615 | | |
| 3616 | + | |
| 3617 | + | |
| 3618 | + | |
| 3619 | + | |
| 3620 | + | |
| 3621 | + | |
| 3622 | + | |
| 3623 | + | |
| 3624 | + | |
| 3625 | + | |
| 3626 | + | |
| 3627 | + | |
| 3628 | + | |
| 3629 | + | |
| 3630 | + | |
| 3631 | + | |
| 3632 | + | |
| 3633 | + | |
| 3634 | + | |
| 3635 | + | |
| 3636 | + | |
| 3637 | + | |
| 3638 | + | |
| 3639 | + | |
| 3640 | + | |
| 3641 | + | |
| 3642 | + | |
| 3643 | + | |
| 3644 | + | |
| 3645 | + | |
| 3646 | + | |
| 3647 | + | |
| 3648 | + | |
| 3649 | + | |
| 3650 | + | |
| 3651 | + | |
| 3652 | + | |
| 3653 | + | |
| 3654 | + | |
| 3655 | + | |
| 3656 | + | |
3612 | 3657 | | |
3613 | 3658 | | |
3614 | 3659 | | |
| |||
0 commit comments