fix(build): replace fs.rmSync with recursive directory removal for CI reliability#526
fix(build): replace fs.rmSync with recursive directory removal for CI reliability#526Copilot wants to merge 3 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The fs.rmSync() call was failing in CI with ENOTEMPTY errors when trying to clean the json-schema directory. This implements a more robust removeDirRecursive() function that: - Manually walks the directory tree - Removes files and subdirectories with retry logic - Handles ENOTEMPTY errors more reliably in CI environments - Uses exponential backoff for retries Tested locally with multiple consecutive builds to verify directory cleaning works correctly. Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
- Add comprehensive JSDoc with @param and @throws - Add error context for failing subdirectories - Keep fs.rmdirSync for directory removal (most reliable approach) Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR addresses CI build failures caused by ENOTEMPTY errors when using fs.rmSync() to clean the json-schema directory. The solution replaces the bulk directory removal with a custom depth-first traversal implementation that provides per-file retry logic with exponential backoff.
Changes:
- Replaced
fs.rmSync()with a newremoveDirRecursive()function that manually traverses directory trees - Implemented file-level retry logic with exponential backoff to handle filesystem timing issues
- Added error context for failed subdirectory removals
| if (entry.isDirectory()) { | ||
| // Recursively remove subdirectory | ||
| try { | ||
| removeDirRecursive(fullPath, retries); |
There was a problem hiding this comment.
The recursive call passes the same retries parameter to subdirectories, which means the outer retry loop (line 86) can retry the entire directory removal multiple times, and each subdirectory removal will also retry multiple times. This creates a multiplicative retry effect (e.g., with MAX_RETRIES=3, a nested directory could see up to 333=27 attempts for deeply nested structures). This can significantly increase build times in CI.
Consider passing retries: 1 to recursive calls or removing the outer retry loop, since the inner operations already have retry logic.
| removeDirRecursive(fullPath, retries); | |
| removeDirRecursive(fullPath, 1); |
| if (entry.isDirectory()) { | ||
| // Recursively remove subdirectory | ||
| try { | ||
| removeDirRecursive(fullPath, retries); | ||
| } catch (error) { | ||
| throw new Error(`Failed to remove subdirectory ${fullPath}: ${error}`); | ||
| } | ||
| } else { | ||
| // Remove file with retry | ||
| for (let fileRetry = 0; fileRetry < retries; fileRetry++) { | ||
| try { | ||
| fs.unlinkSync(fullPath); | ||
| break; | ||
| } catch (error) { | ||
| if (fileRetry === retries - 1) { | ||
| throw error; | ||
| } | ||
| sleepSync(RETRY_DELAY_BASE_MS * (fileRetry + 1)); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The function doesn't handle symbolic links, which could cause issues if the json-schema directory contains symlinks. Attempting to recurse into a symlink to a directory or remove a symlink as a file could fail. The original fs.rmSync handles symlinks correctly.
Add a check for symlinks using entry.isSymbolicLink() and handle them appropriately (e.g., using fs.unlinkSync() regardless of the target type).
| try { | ||
| removeDirRecursive(fullPath, retries); | ||
| } catch (error) { | ||
| throw new Error(`Failed to remove subdirectory ${fullPath}: ${error}`); |
There was a problem hiding this comment.
Wrapping the subdirectory removal error in a new Error object loses the original error's stack trace and type information. This makes debugging CI failures more difficult, which contradicts the PR's goal of improving reliability.
Instead of wrapping, re-throw the original error or use error cause chaining (available in Node.js 16.9.0+): throw new Error(Failed to remove subdirectory ${fullPath}, { cause: error }).
| throw new Error(`Failed to remove subdirectory ${fullPath}: ${error}`); | |
| throw new Error(`Failed to remove subdirectory ${fullPath}`, { cause: error }); |
| for (let attempt = 0; attempt < retries; attempt++) { | ||
| try { | ||
| if (!fs.existsSync(dirPath)) { | ||
| return; // Already removed | ||
| } | ||
|
|
||
| // Read all entries in the directory | ||
| const entries = fs.readdirSync(dirPath, { withFileTypes: true }); | ||
|
|
||
| // Process each entry | ||
| for (const entry of entries) { | ||
| const fullPath = path.join(dirPath, entry.name); | ||
|
|
||
| if (entry.isDirectory()) { | ||
| // Recursively remove subdirectory | ||
| try { | ||
| removeDirRecursive(fullPath, retries); | ||
| } catch (error) { | ||
| throw new Error(`Failed to remove subdirectory ${fullPath}: ${error}`); | ||
| } | ||
| } else { | ||
| // Remove file with retry | ||
| for (let fileRetry = 0; fileRetry < retries; fileRetry++) { | ||
| try { | ||
| fs.unlinkSync(fullPath); | ||
| break; | ||
| } catch (error) { | ||
| if (fileRetry === retries - 1) { | ||
| throw error; | ||
| } | ||
| sleepSync(RETRY_DELAY_BASE_MS * (fileRetry + 1)); | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Now remove the empty directory | ||
| // Using rmdirSync for removing empty directories (works reliably across Node versions) | ||
| fs.rmdirSync(dirPath); | ||
| return; // Success | ||
|
|
||
| } catch (error) { | ||
| if (attempt === retries - 1) { | ||
| throw new Error(`Failed to remove directory ${dirPath} after ${retries} attempts: ${error}`); | ||
| } | ||
| // Wait before retrying with exponential backoff | ||
| const delay = RETRY_DELAY_BASE_MS * (attempt + 1); | ||
| sleepSync(delay); | ||
| } | ||
| } |
There was a problem hiding this comment.
The outer try-catch block's retry logic (lines 86-135) may not work as intended because once the directory traversal begins (line 93), any error thrown from within the loop (lines 96-120) will be caught by the outer catch block. However, if the error occurred partway through processing entries, retrying from the beginning could encounter already-deleted files or directories, potentially causing new errors.
Consider whether the outer retry loop adds value given that individual file operations already have retry logic. If ENOTEMPTY errors occur at the rmdirSync call (line 124), the outer retry would help, but the current error handling doesn't distinguish between different failure points.
| // Recursively remove subdirectory | ||
| try { | ||
| removeDirRecursive(fullPath, retries); | ||
| } catch (error) { | ||
| throw new Error(`Failed to remove subdirectory ${fullPath}: ${error}`); |
There was a problem hiding this comment.
The nested try-catch for subdirectory removal (lines 101-105) will catch errors and re-throw them, bypassing the file-level retry logic. This means if a subdirectory removal fails on the first attempt, it won't be retried by the outer loop at line 86 because the error is thrown immediately, preventing the loop from continuing to process remaining entries.
This inconsistency means subdirectories get different error handling than the main directory, potentially making the function less reliable for the nested structures it's meant to handle.
| // Recursively remove subdirectory | |
| try { | |
| removeDirRecursive(fullPath, retries); | |
| } catch (error) { | |
| throw new Error(`Failed to remove subdirectory ${fullPath}: ${error}`); | |
| // Recursively remove subdirectory with retry | |
| for (let dirRetry = 0; dirRetry < retries; dirRetry++) { | |
| try { | |
| removeDirRecursive(fullPath, retries); | |
| break; | |
| } catch (error) { | |
| if (dirRetry === retries - 1) { | |
| throw error; | |
| } | |
| sleepSync(RETRY_DELAY_BASE_MS * (dirRetry + 1)); | |
| } |
| // Now remove the empty directory | ||
| // Using rmdirSync for removing empty directories (works reliably across Node versions) | ||
| fs.rmdirSync(dirPath); | ||
| return; // Success | ||
|
|
There was a problem hiding this comment.
fs.rmdirSync() has been deprecated since Node.js 14.14.0 in favor of fs.rmSync() with the recursive option (which is being replaced in this PR). While it still works in Node 18+ (per the package.json engines requirement), using deprecated APIs is a best practice violation.
Consider using fs.rmSync(dirPath) instead of fs.rmdirSync(dirPath) for removing the empty directory, as it's the modern, non-deprecated alternative.
| // Now remove the empty directory | |
| // Using rmdirSync for removing empty directories (works reliably across Node versions) | |
| fs.rmdirSync(dirPath); | |
| return; // Success | |
| // Now remove the empty directory using modern API (non-deprecated) | |
| fs.rmSync(dirPath, { recursive: false, force: false }); | |
| return; // Success | |
CI builds fail with
ENOTEMPTYerrors whenfs.rmSync()attempts to clean thejson-schemadirectory. The native retry options (maxRetries,retryDelay) are insufficient for filesystem timing issues in CI environments.Changes
Replaced bulk
fs.rmSync()with manual depth-first traversal:Key differences:
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.