Skip to content

fix: aggregator cache flush on shutdown#735

Merged
smrz2001 merged 2 commits into
mainfrom
m0ar/aggregator-flush-on-shutdown
Aug 27, 2025
Merged

fix: aggregator cache flush on shutdown#735
smrz2001 merged 2 commits into
mainfrom
m0ar/aggregator-flush-on-shutdown

Conversation

@m0ar

@m0ar m0ar commented Aug 27, 2025

Copy link
Copy Markdown
Collaborator

Adds an optional shutdown actor trait that is used in the aggregator to flush the inmemory tables to the on-disk parquet storage, using the same logic as when it happens automatically when the inmemory tables goes over the 10k event limit.

Resolves #728

@m0ar m0ar requested review from dav1do and smrz2001 August 27, 2025 09:49
@m0ar m0ar added the bug Something isn't working label Aug 27, 2025
@m0ar m0ar requested a review from a team as a code owner August 27, 2025 09:49
@m0ar m0ar requested review from stbrody and removed request for a team August 27, 2025 09:49
@m0ar m0ar temporarily deployed to tnet-prod-2024 August 27, 2025 10:10 — with GitHub Actions Inactive

@smrz2001 smrz2001 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@smrz2001 smrz2001 added this pull request to the merge queue Aug 27, 2025
Merged via the queue into main with commit 9e0e0c0 Aug 27, 2025
20 checks passed
@smrz2001 smrz2001 deleted the m0ar/aggregator-flush-on-shutdown branch August 27, 2025 23:41
@smrz2001 smrz2001 mentioned this pull request Aug 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: sporadic pipeline data loss

2 participants