Skip to content

Commit 367f291

Browse files
refactor(cdk): fix emit_slice logic - default True when parent_record exists, False on KeyError
- emit_slice = parent_record is not None (default True when we have a record) - Set emit_slice = False only when KeyError occurs during dpath.get - This makes the logic clearer: we want to emit slices by default, skip only on error Co-Authored-By: Daryna Ishchenko <darina.ishchenko17@gmail.com>
1 parent aa99d1b commit 367f291

1 file changed

Lines changed: 2 additions & 3 deletions

File tree

airbyte_cdk/sources/declarative/partition_routers/substream_partition_router.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,7 @@ def stream_slices(self) -> Iterable[StreamSlice]:
212212
for parent_record, is_last_record_in_slice in iterate_with_last_flag(
213213
partition.read()
214214
):
215-
emit_slice = False
215+
emit_slice = parent_record is not None
216216
if parent_record is not None:
217217
# In the previous CDK implementation, state management was done internally by the stream.
218218
# However, this could cause issues when doing availability check for example as the availability
@@ -234,10 +234,9 @@ def stream_slices(self) -> Iterable[StreamSlice]:
234234
record_data, # type: ignore [arg-type]
235235
parent_field,
236236
)
237-
emit_slice = True
238237
except KeyError:
239238
# FIXME a log here would go a long way for debugging
240-
pass
239+
emit_slice = False
241240

242241
if emit_slice:
243242
# Add extra fields

0 commit comments

Comments
 (0)