Skip to content

Fix Node process crash in compressFileWithGZIP on unreadable PUT source#1454

Open
kartikgupta2607 wants to merge 1 commit into
snowflakedb:masterfrom
kartikgupta2607:fix/put-compress-stream-error-handling
Open

Fix Node process crash in compressFileWithGZIP on unreadable PUT source#1454
kartikgupta2607 wants to merge 1 commit into
snowflakedb:masterfrom
kartikgupta2607:fix/put-compress-stream-error-handling

Conversation

@kartikgupta2607

Copy link
Copy Markdown
Contributor

Description

FileUtil.compressFileWithGZIP (lib/file_util.js) pipes the source file through gzip into a temp file and awaits a promise that only listens for the writable's finish event — none of the read/gzip/write streams has an error handler. If the source file is missing or unreadable when a PUT ... AUTO_COMPRESS=TRUE runs, the read stream emits an error with no listener, which Node escalates to an uncaughtException that crashes the whole process; the wrapping promise also never settles.

This is reachable whenever a file disappears between when the PUT file list is resolved and when a given file is compressed (an external cleanup process, a race, or a path that no longer exists).

Fix: use stream.pipeline from node:stream/promises. It propagates an error from any stream in the chain into a single rejection and destroys every stream, so a missing/unreadable source now rejects compressFileWithGZIP — which propagates through the existing try/catch in uploadOneFile and surfaces as a failed PUT — instead of terminating the process. No behavior change on the success path.

Minimal repro (crashes on current master):

const os = require('os');
const { FileUtil } = require('snowflake-sdk/lib/file_util');
await new FileUtil().compressFileWithGZIP('/path/does-not-exist.csv', os.tmpdir());
// -> uncaughtException: ENOENT (process crash); the promise never settles

Checklist

  • Create tests which fail without the change (added FileUtil.compressFileWithGZIP() tests: gzip round-trip + rejection when the source file is missing; the latter crashes the runner without this fix)
  • Unit tests pass (npm run test:unit); prettier, oxlint and check-ts pass. Integration tests not run locally (require a live account)
  • Extend the types in index.d.ts file (not necessary — no public API change)
  • Updated CHANGELOG.md
  • Provide JIRA/GitHub issue id (external contribution; no JIRA id — happy to link a GitHub issue if preferred)

Draft: opening for early feedback.

compressFileWithGZIP piped the source through gzip with an 'error'-less
promise that only resolved on 'finish', so a missing or unreadable source
raised an unhandled stream 'error' that crashed the process while the
promise never settled. Use stream.pipeline so a failure on any stream
rejects the call and destroys every stream, surfacing as a failed PUT
instead of terminating the process.

Add unit tests: gzip round-trip (happy path) and rejection when the
source file is missing.

Signed-off-by: Kartik Gupta <kartikgupta2607@gmail.com>
@kartikgupta2607
kartikgupta2607 marked this pull request as ready for review July 15, 2026 07:45
@kartikgupta2607
kartikgupta2607 requested a review from a team as a code owner July 15, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant