[GLUTEN-12238][VL] Translate Velox invalid bitmap position errors to Spark's INVALID_BITMAP_POSITION exception

[PerumalsamyR]

What changes are proposed in this pull request?

When bitmap_construct_agg is offloaded to Velox, an invalid bitmap position fails the query with a GlutenException wrapping VeloxUserError, instead of the SparkArrayIndexOutOfBoundsException with error condition INVALID_BITMAP_POSITION that vanilla Spark throws. This breaks Spark error API compatibility for callers that match on the error condition.

This PR translates the error on the JVM side, following the existing translation precedent in ColumnarBatchOutIterator.translateException (which already maps Velox type-conversion errors to SchemaColumnConvertNotSupportedException):

• ColumnarBatchOutIterator.translateException now detects Velox's Bitmap position out of bounds check failure, parses the failing position from the message, and rethrows Spark's INVALID_BITMAP_POSITION exception with the original error as the cause. The bitmap size parameter is Velox's fixed kBitmapNumBytes = 4096, which matches Spark's implementation, so the error parameters (bitPosition, bitmapNumBytes, bitmapNumBits) are identical to vanilla Spark's.
• A new SparkShims.invalidBitmapPositionError shim constructs the exception via QueryExecutionErrors.invalidBitmapPositionError on Spark 3.5/4.0/4.1 (through a small org.apache.spark.sql.errors bridge, since QueryExecutionErrors is private[sql]). Spark 3.3/3.4 have no bitmap aggregate functions and keep the default behavior.
• Re-enables the two previously excluded tests in VeloxTestSettings for Spark 3.5/4.0/4.1 (INVALID_BITMAP_POSITION: position out of bounds, INVALID_BITMAP_POSITION: negative position), which were excluded as a workaround in [GLUTEN-12143][VL] Route bitmap_construct_agg to native Velox execution #12142.

Fixes #12238

How was this patch tested?

Re-enabled the QueryExecutionErrorsSuite tests INVALID_BITMAP_POSITION: position out of bounds and INVALID_BITMAP_POSITION: negative position in the Velox backend test settings for Spark 3.5, 4.0 and 4.1; they are validated by CI.

Was this patch authored or co-authored using generative AI tooling?

Claude Code

[github-actions]

Run Gluten Clickhouse CI on x86

[Copilot]

Pull request overview

This pull request restores Spark error API compatibility for bitmap_construct_agg when executed via the Velox backend by translating Velox “bitmap position out of bounds” failures into Spark’s INVALID_BITMAP_POSITION exception type/condition.

Changes:

• Add JVM-side exception translation in ColumnarBatchOutIterator.translateException to detect Velox invalid bitmap position failures and rethrow Spark’s INVALID_BITMAP_POSITION exception (with the original error as the cause).
• Introduce Spark-version shims (Spark 3.5/4.0/4.1) to construct the Spark-native exception via a small org.apache.spark.sql.errors bridge to private[sql] QueryExecutionErrors.
• Re-enable previously excluded Velox backend tests for INVALID_BITMAP_POSITION on Spark 3.5/4.0/4.1.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
shims/spark41/src/main/scala/org/apache/spark/sql/errors/GlutenQueryExecutionErrors.scala	Adds Spark 4.1 bridge to QueryExecutionErrors.invalidBitmapPositionError.
shims/spark41/src/main/scala/org/apache/gluten/sql/shims/spark41/Spark41Shims.scala	Exposes invalidBitmapPositionError via the Spark 4.1 shims implementation.
shims/spark40/src/main/scala/org/apache/spark/sql/errors/GlutenQueryExecutionErrors.scala	Adds Spark 4.0 bridge to QueryExecutionErrors.invalidBitmapPositionError.
shims/spark40/src/main/scala/org/apache/gluten/sql/shims/spark40/Spark40Shims.scala	Exposes invalidBitmapPositionError via the Spark 4.0 shims implementation.
shims/spark35/src/main/scala/org/apache/spark/sql/errors/GlutenQueryExecutionErrors.scala	Adds Spark 3.5 bridge to QueryExecutionErrors.invalidBitmapPositionError.
shims/spark35/src/main/scala/org/apache/gluten/sql/shims/spark35/Spark35Shims.scala	Exposes invalidBitmapPositionError via the Spark 3.5 shims implementation.
shims/common/src/main/scala/org/apache/gluten/sql/shims/SparkShims.scala	Adds a shim hook for INVALID_BITMAP_POSITION exception creation (defaulting to null for older Spark).
gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala	Re-enables Spark 4.1 INVALID_BITMAP_POSITION tests previously excluded.
gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala	Re-enables Spark 4.0 INVALID_BITMAP_POSITION tests previously excluded.
gluten-ut/spark35/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala	Re-enables Spark 3.5 INVALID_BITMAP_POSITION tests previously excluded.
gluten-arrow/src/main/java/org/apache/gluten/vectorized/ColumnarBatchOutIterator.java	Implements detection/parsing of Velox bitmap OOB failures and maps them to Spark’s INVALID_BITMAP_POSITION.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[PerumalsamyR]

The four failing jobs all appear to be environment issues unrelated to this change:

• spark-test-spark41 and spark-test-spark41-slow: the surefire fork for backends-velox aborts with exit code 134 (SIGABRT) right after the Velox backend initializes, before the gluten-ut modules run — same signature as [VL] Spark 4.1 CI intermittently aborts during Velox backend initialization #12534.
• spark-test-spark40 and spark-test-spark35-slow (second workflow run): install-spark-resources.sh failed to download the Spark tarball from the mirror (Connection timed out / Unable to establish SSL connection). The same jobs passed in the other workflow run.

The re-enabled QueryExecutionErrorsSuite tests passed where they ran (spark35 and spark40 jobs), with the expected translated error:

org.apache.spark.SparkArrayIndexOutOfBoundsException: [INVALID_BITMAP_POSITION] The 0-indexed bitmap position 32768 is out of bounds. The bitmap has 32768 bits (4096 bytes). SQLSTATE: 22003
- INVALID_BITMAP_POSITION: position out of bounds
- INVALID_BITMAP_POSITION: negative position

Could a committer please re-run the failed jobs? Thanks!

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

[github-actions]

Run Gluten Clickhouse CI on x86

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.