feat: contrib Delta driver-side Rust (log replay + predicate pushdown + JNI) [Delta contrib split, part 3a]

Part 3a of the Delta Lake contrib PR breakup (tracking: apache#4366). Replaces the build-gate
stub crate's deps with the real driver-side modules and the delta-kernel-rs dependency,
while the executor-side read path stays deferred (the `planner` stub still returns
NotImplemented until part 3b adds `kernel_scan`/`dv_reader` + the real planner).

Driver side (open table, replay log, push predicates, return a DeltaScanTaskList over JNI):
- `error.rs`  - DeltaError / DeltaResult.
- `engine.rs` - delta-kernel engine + object_store config (S3/Azure/GCS/local) for log replay.
- `predicate.rs` - Catalyst -> kernel predicate translation for file skipping.
- `scan.rs`   - log replay -> DeltaFileEntry/DeltaScanPlan; scan-task assembly.
- `jni.rs`    - `Native_planDeltaScan` / `Native_planDeltaReadSchemas` JNI entry points
  (the JVM `Native.scala` that calls them lands in part 4b).
- `lib.rs`    - declares the driver modules + keeps the `planner` stub; drops the
  `dv_reader`/`kernel_scan` module decls (part 3b).
- `Cargo.toml` - only the deps the driver set uses (delta_kernel, object_store, arrow,
  jni, prost, serde_json, url, thiserror, log + jni-bridge); executor deps arrive in 3b.

The driver set is self-contained (`jni -> scan -> engine -> error`, `predicate` standalone);
nothing references the deferred modules. Core is untouched -- the dispatch shim still calls
`planner::plan_delta_scan` (the stub) so a Delta read falls back to vanilla Spark until 3b.
The gate-verify cargo-tree assertion is re-tightened to require `delta_kernel` (now real).

Verification: gated native build, 54 in-crate unit tests (cargo test), default native build
unchanged, clippy (both feature states), gate-verify script, cargo fmt -- all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: schenksj

contrib/delta/native/Cargo.lock

@@ -20,16 +20,15 @@
 # `{ workspace = true }` inheritance. Versions are kept in sync with the rest of the
 # repo by convention; the path deps anchor against the same on-disk crates.
 #
-# BUILD-GATE STUB: this is the inert wiring unit. The crate exists only so that a
-# `--features contrib-delta` build of core compiles and links, and so the gate-verify
-# script can prove the default build carries zero Delta surface. The real kernel-read
-# implementation (delta_kernel, object_store, the scan/engine/dv modules, ...) lands in
-# later units; here `planner::plan_delta_scan` returns a clean "not implemented" error so
-# any Delta read that reaches native falls back to vanilla Spark.
+# This unit ships the DRIVER side (log replay, predicate pushdown, scan-task assembly,
+# JNI). It declares only the deps the driver modules (error/engine/predicate/scan/jni)
+# actually use; the executor-side read path lands in the next unit and brings its own
+# deps (parquet, roaring, datafusion-datasource, futures, chrono*, comet-common, ...).
+# `planner` is still the build-gate stub until then.
 
 [package]
 name = "comet-contrib-delta"
-description = "delta-kernel-rs integration for Comet (build-gate stub; native read path lands in a later unit). Linked into libcomet via core's `contrib-delta` Cargo feature flag."
+description = "delta-kernel-rs integration for Comet. Reads Delta tables via kernel-rs's log replay and DataFusion's parquet scan. Linked into libcomet via core's `contrib-delta` Cargo feature flag."
 version = "0.18.0"
 edition = "2021"
 rust-version = "1.86.0"
@@ -43,8 +42,34 @@ crate-type = ["rlib"]
 
 [dependencies]
 # Typed Delta proto messages live in core's proto crate (alongside IcebergScan, ...) so the
-# dispatcher arm has direct access; the stub planner names `DeltaScan` / `DeltaScanCommon` from here.
+# dispatcher arm has direct access. Re-exported as `crate::proto::*`.
 datafusion-comet-proto = { path = "../../../native/proto" }
-# For `ExecutionPlan` (return type), `SchemaRef` (the shim hands these in), and `DataFusionError`
-# (the not-implemented error). Versions/features mirror core so the shared lockfile resolves cleanly.
-datafusion = { version = "53.1.0", default-features = false, features = ["parquet"] }
+# JNI helpers (CometError, CometResult, try_unwrap_or_throw) for the driver-side JNI entry
+# points. jni-bridge is a leaf crate -- depending on it drags no Comet logic into the contrib.
+datafusion-comet-jni-bridge = { path = "../../../native/jni-bridge" }
+# Heavy Delta dep -- intentionally lives ONLY in this contrib, never in core. delta_kernel
+# 0.24 pins the SAME arrow (58) as Comet via the `arrow-58` feature, so kernel RecordBatches
+# flow straight into Comet plans with no arrow-version bridge.
+delta_kernel = { version = "0.24", default-features = false, features = ["default-engine-rustls", "arrow-58"] }
+# DataFusion / Arrow versions chosen to match core's pinned values. The driver only needs
+# `ExecutionPlan` / `SchemaRef` / `DataFusionError` (the planner stub) -- no `parquet` feature
+# here; the executor unit, which actually scans parquet, adds it back.
+datafusion = { version = "53.1.0", default-features = false }
+arrow = { version = "58.1.0", features = ["prettyprint", "ffi", "chrono-tz"] }
+# Match core Comet's object_store features (native/Cargo.toml) so the Azure/GCS builders
+# reachable via `object_store::parse_url` mirror core's non-S3 read path.
+object_store = { version = "0.13.1", features = ["aws", "azure", "gcp", "http"] }
+url = "2.5.4"
+thiserror = "2"
+prost = "0.14.3"
+# Parse the analysis-time Delta schema JSON (shipped from the JVM) into a kernel `StructType`
+# for `ScanBuilder::with_schema`, so kernel resolves column-mapping physical names against the
+# schema the query was PLANNED with (correct for schema-change-since-analysis).
+serde_json = "1"
+jni = "0.22.4"
+log = "0.4"
+
+[dev-dependencies]
+# Used by unit tests under #[cfg(test)] in scan.rs to materialise a Delta `_delta_log`
+# in a tempdir without polluting the real filesystem.
+tempfile = "3"