Skip to content

feat(install): add file locking to prevent concurrent installation issues#289

Merged
fengmk2 merged 1 commit intomainfrom
11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations
Nov 4, 2025
Merged

feat(install): add file locking to prevent concurrent installation issues#289
fengmk2 merged 1 commit intomainfrom
11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations

Conversation

@fengmk2
Copy link
Copy Markdown
Member

@fengmk2 fengmk2 commented Nov 3, 2025

Add file locking to prevent race conditions in package manager installation

Added file-based locking to ensure atomic operations during package manager downloads, preventing concurrent processes from conflicting when installing the same package manager version.

close #140

Changes

  • Added file-based locking using fs4 crate to synchronize critical operations
  • Added NFS filesystem detection to skip locking on NFS where it may be unreliable
  • Improved error logging in remove_dir_all_force to provide better debugging information
  • Added RUST_BACKTRACE=1 to CLI E2E tests for better error diagnostics
  • Added RUST_* to passthrough environment variables in tools utils

Why

This change prevents race conditions when multiple processes attempt to install the same package manager version simultaneously. The file lock ensures that the remove and rename operations are performed atomically, avoiding conflicts between concurrent installation attempts.

Copy link
Copy Markdown
Member Author

fengmk2 commented Nov 3, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@fengmk2 fengmk2 changed the title fix: lock to ensure atomicity of remove + rename operations fix(pm): add directory locking for concurrent package manager installations Nov 3, 2025
@fengmk2 fengmk2 self-assigned this Nov 3, 2025
@fengmk2 fengmk2 marked this pull request as ready for review November 3, 2025 05:56
Copilot AI review requested due to automatic review settings November 3, 2025 05:56
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a concurrency-safe mechanism for downloading and installing package managers by introducing per-directory locks to prevent race conditions when multiple threads attempt to install the same package manager version simultaneously.

  • Adds a get_directory_lock function that maintains a global map of per-directory locks using OnceLock and Arc<TokioMutex<()>>
  • Protects the critical section where directories are removed and renamed with an async lock
  • Enhances the file existence check after lock acquisition to verify all required shim files

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/vite_install/src/package_manager.rs Outdated
Comment thread crates/vite_install/src/package_manager.rs Outdated
Comment thread crates/vite_install/src/package_manager.rs Outdated
Comment thread crates/vite_install/src/package_manager.rs Outdated
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 19076a6 to 2888f33 Compare November 3, 2025 07:08
@fengmk2 fengmk2 changed the title fix(pm): add directory locking for concurrent package manager installations fix(pm): add mutex lock for concurrent package manager downloads Nov 3, 2025
@fengmk2 fengmk2 requested a review from Copilot November 3, 2025 07:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/vite_install/src/package_manager.rs Outdated
Comment thread crates/vite_install/src/package_manager.rs Outdated
Copy link
Copy Markdown
Member Author

fengmk2 commented Nov 3, 2025

image.png

https://github.com/voidzero-dev/vite-plus/runs/54331917061

Let's set RUST_BACKTRACE=1 on CI running.

@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 2888f33 to 7ddf40b Compare November 3, 2025 07:27
Copilot AI review requested due to automatic review settings November 3, 2025 08:44
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 7ddf40b to d14a57a Compare November 3, 2025 08:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Member Author

fengmk2 commented Nov 3, 2025

image.png

https://github.com/voidzero-dev/vite-plus/actions/runs/19028678443/job/54337451055

The issue is still caused by concurrent deletions of the same directory. Snap-test is a multi-process operation, so thread locks cannot resolve concurrency conflicts and need to be changed to file locks.

@fengmk2 fengmk2 marked this pull request as draft November 3, 2025 08:50
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from d14a57a to 201a5aa Compare November 3, 2025 13:35
Comment thread crates/vite_install/src/package_manager.rs Outdated
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 201a5aa to c5cd576 Compare November 3, 2025 15:12
@fengmk2 fengmk2 changed the title fix(pm): add mutex lock for concurrent package manager downloads feat(install): add file locking to prevent concurrent installation issues Nov 3, 2025
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch 4 times, most recently from 9f4372e to a73f6bc Compare November 3, 2025 15:49
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from a73f6bc to ca1cfa3 Compare November 3, 2025 16:17
@fengmk2 fengmk2 marked this pull request as ready for review November 3, 2025 16:21
Copilot AI review requested due to automatic review settings November 3, 2025 16:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/vite_install/src/package_manager.rs
Comment thread crates/vite_install/src/package_manager.rs Outdated
Comment thread Cargo.toml Outdated
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from ca1cfa3 to 1d5153a Compare November 3, 2025 16:32
Copilot AI review requested due to automatic review settings November 3, 2025 16:49
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 1d5153a to 41df960 Compare November 3, 2025 16:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/tools/src/snap-test.ts Outdated
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 41df960 to 0c1be78 Compare November 3, 2025 16:54
Copilot AI review requested due to automatic review settings November 3, 2025 17:01
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 0c1be78 to 57e4322 Compare November 3, 2025 17:01
Comment thread packages/tools/src/snap-test.ts Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/vite_install/src/package_manager.rs
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 57e4322 to 3d1aecf Compare November 3, 2025 17:06
@fengmk2 fengmk2 mentioned this pull request Nov 3, 2025
@fengmk2 fengmk2 requested a review from branchseer November 4, 2025 01:20
Copy link
Copy Markdown
Member Author

fengmk2 commented Nov 4, 2025

Run it a few more times today; if no errors occur, it can be merged.

Copilot AI review requested due to automatic review settings November 4, 2025 03:37
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from 3d1aecf to c6a415b Compare November 4, 2025 03:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/tools/src/snap-test.ts
@fengmk2 fengmk2 force-pushed the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch from c6a415b to c91191b Compare November 4, 2025 05:41
Copy link
Copy Markdown
Member Author

fengmk2 commented Nov 4, 2025

@branchseer Can you take another look? I retried enough times today and the directory error hasn't reappeared

@fengmk2 fengmk2 merged commit 3cc727b into main Nov 4, 2025
37 checks passed
Copy link
Copy Markdown
Member Author

fengmk2 commented Nov 4, 2025

Merge activity

@fengmk2 fengmk2 deleted the 11-03-fix_lock_to_ensure_atomicity_of_remove_rename_operations branch November 4, 2025 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

unstable snap-test: packages/global/snap-tests/command-add-yarn4-with-workspace fix package_manager unstable tests

3 participants