Skip to content

Filestore] add SIGUSR1 retries to FUSE loop thread shutdown to mitigate single lost SIGUSR1 wakeup#6426

Merged
neihar merged 2 commits into
mainfrom
users/ihar/fix-unmount
Jul 7, 2026
Merged

Filestore] add SIGUSR1 retries to FUSE loop thread shutdown to mitigate single lost SIGUSR1 wakeup#6426
neihar merged 2 commits into
mainfrom
users/ihar/fix-unmount

Conversation

@neihar

@neihar neihar commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Problem

https://github.com/ydb-platform/nbs/actions/runs/27998974260/job/82867193543

Notes

TFuseLoopThread::Stop() sends one pthread_kill(SIGUSR1), then joins. The loop thread runs while(!fuse_session_exited(se)) read(/dev/fuse). A signal only breaks read() with EINTR if the thread is already sleeping inside it. There is a short window between the exit-flag check and the start of read(). If the signal arrives in that window, the no-op handler eats it. The thread then enters read() with nothing pending and blocks forever. No more FUSE requests come. No more signals are sent. The unmount that would break read() with ENODEV runs only after the join, and the join is already stuck on this thread. This window widens under host load. The bug predates the multi-queue refactor (#4045).

@neihar neihar added large-tests Launch large tests for PR filestore Add this label to run only cloud/filestore build and tests on PR tsan Launch builds with thread sanitizer along with regular build ubsan Launch builds with undefined behaviour sanitizer along with regular build labels Jul 5, 2026
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-tsan have completed.

Tip

Planned checks for linux-x86_64-release-tsan.

🔴 linux-x86_64-release-tsan target: cloud/filestore/ (test time: 5275s): some tests FAILED for commit f3036c7.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
3500 3499 0 1 0 0 0

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 6792s): all tests PASSED for commit f3036c7.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4144 4144 0 0 0 0 0

@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-ubsan have completed.

Tip

Planned checks for linux-x86_64-release-ubsan.

🟢 linux-x86_64-release-ubsan target: cloud/filestore/ (test time: 4561s): all tests PASSED for commit f3036c7.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
3560 3560 0 0 0 0 0

@neihar
neihar requested review from SvartMetal, budevg and qkrorlqr July 5, 2026 11:34
@neihar neihar changed the title Filestore] fix FUSE loop thread shutdown hang caused by a lost SIGUSR1 wakeup Filestore] add SIGUSR1 retires to FUSE loop thread shutdown to mitigate single lost SIGUSR1 wakeup Jul 5, 2026
@neihar neihar changed the title Filestore] add SIGUSR1 retires to FUSE loop thread shutdown to mitigate single lost SIGUSR1 wakeup Filestore] add SIGUSR1 retries to FUSE loop thread shutdown to mitigate single lost SIGUSR1 wakeup Jul 5, 2026
Comment thread cloud/filestore/libs/vfs_fuse/loop.cpp Outdated
qkrorlqr
qkrorlqr previously approved these changes Jul 6, 2026
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-ubsan have completed.

Tip

Planned checks for linux-x86_64-release-ubsan.

🟢 linux-x86_64-release-ubsan target: cloud/filestore/ (test time: 4711s): all tests PASSED for commit 4063480.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
3560 3560 0 0 0 0 0

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-tsan have completed.

Tip

Planned checks for linux-x86_64-release-tsan.

🔴 linux-x86_64-release-tsan target: cloud/filestore/ (test time: 5129s): some tests FAILED for commit 4063480.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
3500 3498 0 2 0 0 0

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 6609s): some tests FAILED for commit 4063480.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4144 4143 0 1 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 65s): all tests PASSED for commit 4063480.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 2 0 0 0 0 0

@neihar
neihar merged commit 15651d2 into main Jul 7, 2026
32 of 35 checks passed
@neihar
neihar deleted the users/ihar/fix-unmount branch July 7, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

filestore Add this label to run only cloud/filestore build and tests on PR large-tests Launch large tests for PR tsan Launch builds with thread sanitizer along with regular build ubsan Launch builds with undefined behaviour sanitizer along with regular build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants