fix: use mem::forget to avoid fjall Drop deadlock (#86)#87
Open
mwiewior wants to merge 1 commit into
Open
Conversation
…ds (#86) fjall 3.1.x DatabaseInner::drop() deadlocks when background worker threads are still compacting — the bounded(1000) flume channel fills up and send(Close) blocks permanently. After persist + major_compact all data is safely on disk, so skipping Drop via mem::forget is safe. Applies to both build_variation_fjall_from_parquet() and build_sift_fjall(). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
build_cachehangs indefinitely at "closing database..." on full-sized Ensembl VEP cachesstd::mem::forgetto skip fjall's brokenDropimpl after all data is persisted and compactedbuild_variation_fjall_from_parquet()andbuild_sift_fjall()Root cause
fjall 3.1.x
DatabaseInner::drop()sendsClosemessages on a bounded(1000) flume channel in a spin loop. When background workers are still in long-runningcompact()(which doesn't checkstop_signal), the channel fills up andsend()blocks permanently → deadlock.Why this is safe
Both code paths call
persist()+major_compact()before the forget — all data is on disk and fully compacted. TheDroponly needs to join worker threads, but it deadlocks trying.mem::forgetis safe Rust and the standard workaround for brokenDropimplementations.Test plan
cargo clippy -- -D warningspasses🤖 Generated with Claude Code