Skip to content

Commit bb6e3c0

Browse files
committed
address feedback
1 parent f63118b commit bb6e3c0

2 files changed

Lines changed: 5 additions & 1 deletion

File tree

common/src/main/scala/org/apache/comet/CometConf.scala

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -400,6 +400,9 @@ object CometConf extends ShimCometConf {
400400
"which can improve performance for wide tables while still providing " +
401401
"reasonable distribution.")
402402
.intConf
403+
.checkValue(
404+
v => v >= 0,
405+
"The maximum number of columns to hash for round robin partitioning must be non-negative.")
403406
.createWithDefault(0)
404407

405408
val COMET_EXEC_SHUFFLE_COMPRESSION_CODEC: ConfigEntry[String] =

docs/source/user-guide/latest/compatibility.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,8 @@ impossible to match Spark's exact partition assignments.
8787
Instead of true round-robin assignment, Comet implements round-robin as hash partitioning on ALL columns. This
8888
achieves the same semantic goals:
8989

90-
- **Even distribution**: Rows are distributed evenly across partitions
90+
- **Even distribution**: Rows are distributed evenly across partitions (as long as the hash varies sufficiently -
91+
in some cases there could be skew)
9192
- **Deterministic**: Same input always produces the same partition assignments (important for fault tolerance)
9293
- **No semantic grouping**: Unlike hash partitioning on specific columns, this doesn't group related rows together
9394

0 commit comments

Comments
 (0)