fix(gso): stop 9-judge eval SIGSEGV from concurrent Spark Connect

[hiydavid]

Problem

GSO's baseline_eval task crashed the Python kernel with a fatal SIGSEGV
(exit code 139, "The Python kernel is unresponsive") during Step 2b: Run
9-Judge Evaluation. It was reproducible across serverless and dedicated
compute, and reducing target_benchmark_count did not help.

Root cause

The 9-judge evaluation runs scorers across an 8-worker thread pool
(scorer_workers=8). Of the 9 judges, only syntax_validity touches
Spark — it called spark.sql("USE CATALOG …") / spark.sql("EXPLAIN …")
directly inside its scoring closure. Under the thread pool, up to 8 threads
drove the shared Spark Connect session concurrently, mutating session
state and issuing EXPLAINs over the same gRPC channel. Spark Connect's client
and the underlying gRPC / pyarrow C extensions are not safe under that
concurrency, producing a native segfault.

Evidence:

• Preflight passes (UC metadata + data profiling use Spark single-threaded);
  only Step 2b (8-way concurrent Spark) crashes.
• Reproducible on any compute — a code-level concurrency bug, not infra.
• Reducing benchmark count doesn't help — each row still fans out 8 scorers.
• The faulthandler dump shows the main thread merely parked in the watchdog
  join; the watchdog did not time out, so the crash came from inside the eval.

Fix

1. Route the scorer's EXPLAIN through the thread-safe SQL Warehouse Statement
   Execution API (_execute_sql_via_warehouse) when a warehouse_id is
   available — each call is an independent HTTP request with no shared session.
   This mirrors the existing benchmark precheck, and surfaces planning errors
   from the returned plan column (the warehouse EXPLAIN returns a plan rather
   than throwing).
2. Add a process-wide spark_serialized() lock (common/spark_concurrency.py)
   as defense-in-depth for the no-warehouse fallback, so the shared Spark Connect
   session is never driven by two threads at once.

EXPLAIN's full catalog-aware validation (unresolved columns/tables/functions,
not just syntax) is preserved — the warehouse path runs the same EXPLAIN.

Changes

• common/spark_concurrency.py (new) — module-level lock + spark_serialized().
• scorers/syntax_validity.py — new _explain_sql() helper (warehouse-first,
  serialized-Spark fallback); factory gains w + warehouse_id.
• scorers/__init__.py — make_all_scorers(...) gains warehouse_id, threaded
  into the syntax scorer.
• harness.py — all 3 make_all_scorers call sites pass
  warehouse_id=resolve_warehouse_id("").

Verification

No local dev server — verify by deploying (./scripts/deploy.sh --update) and
running an Auto-Optimize pass against a test Genie Space:

1. baseline_eval → Step 2b completes without the exit-139 SIGSEGV.
2. syntax_validity still emits yes/no verdicts — valid SQL → yes, SQL with a
   bad column/function → no with the right failure type (proves the
   warehouse-routed EXPLAIN does real catalog-aware validation).
3. Job log shows no concurrent Spark Connect EXPLAINs from the scorer pool.