Skip to content

Commit a8bf7ea

Browse files
authored
[MINOR] Fix typo in shuffleInputOutput config name (#12239)
This PR fixes a typo in the Velox backend configuration key: spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInputOuptut.minSize is renamed to: spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInputOutput.minSize
1 parent ca7d68f commit a8bf7ea

2 files changed

Lines changed: 2 additions & 2 deletions

File tree

backends-velox/src/main/scala/org/apache/gluten/config/VeloxConfig.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -372,7 +372,7 @@ object VeloxConfig extends ConfigRegistry {
372372
.createOptional
373373

374374
val COLUMNAR_VELOX_RESIZE_BATCHES_SHUFFLE_INPUT_OUTPUT_MIN_SIZE =
375-
buildConf("spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInputOuptut.minSize")
375+
buildConf("spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInputOutput.minSize")
376376
.doc(
377377
s"The minimum batch size for shuffle input and output. " +
378378
s"If size of an input batch is " +

docs/velox-configuration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ nav_order: 16
6363
| spark.gluten.sql.columnar.backend.velox.resizeBatches.copyRanges.enabled | 🔄 Dynamic | true | Enables a VeloxResizeBatchesExec fast path that combines eligible batches using Velox vector copyRanges instead of generic RowVector append. When possible, it collects the small input batches for one VeloxResizeBatchesExec output, allocates the output RowVector once, and bulk-copies child vector ranges. This is most useful for shuffle-read outputs where plain hash shuffle payloads are materialized as dense flat vectors. Complex vectors can also use copyRanges, but ARRAY and MAP still rebuild nested offsets and sizes while bulk-copying child ranges. Unsupported encodings such as dictionary and constant vectors fall back to the generic copy path. This option is enabled by default and complements the reader-side raw payload merge fast path: that path avoids materializing small plain payload batches, while this option optimizes VeloxResizeBatchesExec when that operator is enabled. |
6464
| spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInput | 🔄 Dynamic | true | If true, combine small columnar batches together before sending to shuffle. The default minimum output batch size is equal to 0.25 * spark.gluten.sql.columnar.maxBatchSize |
6565
| spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInput.minSize | 🔄 Dynamic | &lt;undefined&gt; | The minimum batch size for shuffle. If size of an input batch is smaller than the value, it will be combined with other batches before sending to shuffle. Only functions when spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInput is set to true. Default value: 0.25 * <max batch size> |
66-
| spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInputOuptut.minSize | 🔄 Dynamic | &lt;undefined&gt; | The minimum batch size for shuffle input and output. If size of an input batch is smaller than the value, it will be combined with other batches before sending to shuffle. The same applies for batches output by shuffle read. Only functions when spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInput or spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleOutput is set to true. Default value: 0.25 * <max batch size> |
66+
| spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInputOutput.minSize | 🔄 Dynamic | &lt;undefined&gt; | The minimum batch size for shuffle input and output. If size of an input batch is smaller than the value, it will be combined with other batches before sending to shuffle. The same applies for batches output by shuffle read. Only functions when spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleInput or spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleOutput is set to true. Default value: 0.25 * <max batch size> |
6767
| spark.gluten.sql.columnar.backend.velox.resizeBatches.shuffleOutput | 🔄 Dynamic | false | If true, combine small columnar batches together right after shuffle read. The default minimum output batch size is equal to 0.25 * spark.gluten.sql.columnar.maxBatchSize |
6868
| spark.gluten.sql.columnar.backend.velox.showTaskMetricsWhenFinished | 🔄 Dynamic | false | Show velox full task metrics when finished. |
6969
| spark.gluten.sql.columnar.backend.velox.spillFileSystem | 🔄 Dynamic | local | The filesystem used to store spill data. local: The local file system. heap-over-local: Write file to JVM heap if having extra heap space. Otherwise write to local file system. |

0 commit comments

Comments
 (0)