Skip to content

[Feature] enable gz compression by default for StreamLoad#648

Merged
JNSimba merged 6 commits intoapache:masterfrom
JNSimba:enablegz
Apr 7, 2026
Merged

[Feature] enable gz compression by default for StreamLoad#648
JNSimba merged 6 commits intoapache:masterfrom
JNSimba:enablegz

Conversation

@JNSimba
Copy link
Copy Markdown
Member

@JNSimba JNSimba commented Apr 3, 2026

Summary

  • Enable gz compression by default for StreamLoad writes to reduce network transfer and improve write performance
  • Users can disable compression by explicitly setting 'sink.properties.compress_type' = ''
  • Covers all write paths: Flink SQL, DataStream API, CDC DatabaseSync, and Catalog
  • Added IT cases for default gz compression and explicit no-compression scenarios

Test plan

  • Verify default gz compression works (testTableDefaultGzFormat)
  • Verify explicit no-compression works (testTableNoCompressFormat)
  • Verify existing testTableGzFormat still passes
  • Verify all other sink IT cases pass with implicit default compression

🤖 Generated with Claude Code

JNSimba and others added 6 commits April 3, 2026 16:12
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Doris 2.1 does not support json compression, so explicitly set
compress_type to empty for json format test cases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…fix checkstyle CI

Doris 2.1 does not support json compression, so disable default gz
compression for json format test cases. Also fix checkstyle CI to
install dependencies before running checkstyle:check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@JNSimba JNSimba merged commit 06edd0c into apache:master Apr 7, 2026
8 of 9 checks passed
@JNSimba JNSimba mentioned this pull request Apr 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants