FUSE client hot-upgrade: controlled drain, READY/NACK handover, teardown fixes#957
Merged
Merged
Conversation
6295f7a to
a256bdd
Compare
a256bdd to
78134a1
Compare
…ix teardown deadlock
…xecutors + idempotent VFSHub/meta Stop
…/NACK handover handshake.
78134a1 to
f4f1224
Compare
5209ece to
3e02123
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds graceful hot-upgrade support for the DingoFS FUSE client.
The new process takes over the existing kernel FUSE connection by receiving the old process's
/dev/fusefd over UDS, replays the saved FUSE INIT request, and loads the dumped VFS state. The old process uses the libfuse controlled-drain APIs before checkpointing, so requests that have already been read by the old daemon are handled before the fd is handed over.This is a graceful handover path, not a fully transactional hot-upgrade with rollback at every failure point. Failures before checkpoint can NACK/resume on the old process. After
Stop(true) + Dump, the old VFS has already been torn down, so a later new-side mount/load failure still requires remount or manual recovery.Main Changes
/tmpand publish it atomically with temp file + fsync + rename + directory fsync./dev/fusefd.kPreparevalidation.VFSWrapper::Stop(true)and state dump/load.kReadyToExit; drain/checkpoint-precondition failure sendskNackand resumes the old process.HandleGuardso Stop waits for active handle users and releases reader/writer resources outside the handle lock.kReadyToExita one-way final notification. The new process no longer sendskAck, and the old process no longer waits for it after checkpoint.kNack/ResumeReceive().Commit Layout
free leaked FUSE INIT receive buffermove VFS state file off /tmp and publish atomicallycontrolled drain + flush checkpoint + ACK/NACK handover handshakeRelease handle resources in Stop() via HandleGuardHot-upgrade teardown: dedicated read-cleanup/write-bg executors + idempotent VFSHub/meta Stopmake READY one-way and remove ACK waitdetach statfs wakeup thread on drain timeoutValidation
cmake --build build --target test_fuse_upgrade -j2build/bin/test/test_fuse_upgradeCurrent result: 16 tests passed.
Notes
This PR depends on the libfuse controlled-drain APIs used by
HandoverSessionImpl:fuse_session_pause_receivefuse_session_resume_receivefuse_session_wait_drained