feat: enable JVM Scala UDF codegen dispatch by default

Flip `spark.comet.exec.scalaUDF.codegen.enabled` to default `true` so that
eligible Spark `ScalaUDF` expressions are routed through Comet's Arrow-direct
codegen dispatcher without requiring opt-in. The feature is no longer marked
experimental.

Update the Scala/Java UDF and Iceberg user guides to reflect that the
dispatcher is on by default and document how to disable it.

Author: andygrove

docs/source/user-guide/latest/iceberg.md

@@ -157,12 +157,12 @@ Iceberg ships several `ScalaUDF`s that surface in user queries and maintenance a
   (`INT_ORDERED_BYTES`, `LONG_ORDERED_BYTES`, ..., `INTERLEAVE_BYTES`) over the sort key columns
   during compaction.
 
-By default these UDFs cause the enclosing operator to fall back to Spark, which forces a
-columnar-to-row roundtrip and demotes the surrounding shuffle from `CometExchange` to
-`CometColumnarExchange`. Enabling the experimental
-[Scala UDF and Java UDF Support](scala_java_udfs.md) feature
-(`spark.comet.exec.scalaUDF.codegen.enabled=true`) routes these UDFs through native execution so
-the project, exchange, and sort operators around them stay on the Comet path end-to-end.
+[Scala UDF and Java UDF Support](scala_java_udfs.md) is enabled by default
+(`spark.comet.exec.scalaUDF.codegen.enabled=true`), so these UDFs run through native execution and
+the project, exchange, and sort operators around them stay on the Comet path end-to-end. Setting
+`spark.comet.exec.scalaUDF.codegen.enabled=false` causes the enclosing operator to fall back to
+Spark, which forces a columnar-to-row roundtrip and demotes the surrounding shuffle from
+`CometExchange` to `CometColumnarExchange`.
 
 ### Task input metrics
 

docs/source/user-guide/latest/scala_java_udfs.md

@@ -23,13 +23,13 @@ Comet executes Spark's Scala and Java [scalar user-defined functions (UDFs)](htt
 
 This page covers Spark's `ScalaUDF` (Scala `udf(...)`, `spark.udf.register(...)` over Scala or Java functional interfaces, and SQL `CREATE FUNCTION ... AS 'com.example.MyUDF'`). Other UDF kinds (Python / Pandas, Hive, aggregate) are out of scope and continue to fall back to Spark.
 
-This feature is experimental and disabled by default.
+This feature is enabled by default. Set `spark.comet.exec.scalaUDF.codegen.enabled` to `false` to route plans containing a `ScalaUDF` back to Spark for the enclosing operator.
 
 ## Configuration
 
 | Key                                         | Default | Description                                                                                                        |
 | ------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------ |
-| `spark.comet.exec.scalaUDF.codegen.enabled` | `false` | When `true`, eligible `ScalaUDF`s run on the Comet path. When `false`, the enclosing operator falls back to Spark. |
+| `spark.comet.exec.scalaUDF.codegen.enabled` | `true` | When `true`, eligible `ScalaUDF`s run on the Comet path. When `false`, the enclosing operator falls back to Spark. |
 
 ## Supported
 

spark/src/main/scala/org/apache/comet/CometConf.scala

@@ -365,13 +365,13 @@ object CometConf extends ShimCometConf {
   val COMET_SCALA_UDF_CODEGEN_ENABLED: ConfigEntry[Boolean] =
     conf("spark.comet.exec.scalaUDF.codegen.enabled")
       .category(CATEGORY_EXEC)
-      .doc("Experimental. Whether to route Spark `ScalaUDF` expressions through Comet's " +
+      .doc("Whether to route Spark `ScalaUDF` expressions through Comet's " +
         "Arrow-direct codegen dispatcher. When enabled, a supported ScalaUDF is compiled into " +
         "a per-batch kernel that reads and writes Arrow vectors directly from native " +
         "execution. When disabled, plans containing a ScalaUDF fall back to Spark for the " +
         "enclosing operator.")
       .booleanConf
-      .createWithDefault(false)
+      .createWithDefault(true)
 
   val COMET_EXEC_SHUFFLE_WITH_HASH_PARTITIONING_ENABLED: ConfigEntry[Boolean] =
     conf("spark.comet.native.shuffle.partitioning.hash.enabled")