Skip to content

Add OptionVal for JSON import fast path#1086

Open
He-Pin wants to merge 1 commit into
databricks:masterfrom
He-Pin:perf/option-val
Open

Add OptionVal for JSON import fast path#1086
He-Pin wants to merge 1 commit into
databricks:masterfrom
He-Pin:perf/option-val

Conversation

@He-Pin

@He-Pin He-Pin commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Motivation

CachedResolver.parseJsonImport returns an optional result so callers can use the strict .json parser when it succeeds and fall back to the Jsonnet parser otherwise. The old Option[(Expr, FileScope)] return allocated a scala.Some on every successful JSON fast-path import.

Modification

  • Add a small internal OptionVal[A <: AnyRef] value class backed by a nullable reference.
  • Use a singleton OptionVal.None sentinel for empty results.
  • Change CachedResolver.parseJsonImport and the Preloader call site from Option to OptionVal.
  • Keep the scope narrow to allocation-sensitive internal code; no public API changes.

Result

Successful strict .json imports avoid the scala.Some wrapper allocation while keeping the same behavior for:

  • non-.json imports
  • invalid strict JSON falling back to the Jsonnet parser
  • duplicate-key and non-finite-number fallback/error behavior
  • preloaded JSON import caching

@He-Pin He-Pin marked this pull request as ready for review July 4, 2026 14:17
Motivation:
Strict .json imports return an optional parse result so callers can fall back to the Jsonnet parser. The previous Option[(Expr, FileScope)] result allocated a scala.Some on each successful JSON fast-path parse.

Modification:
Add a small internal OptionVal value class for optional non-null reference values and use it for CachedResolver.parseJsonImport and the Preloader call site. Reject null payloads, add direct OptionVal regression tests, and avoid tuple-pattern destructuring in the preloader fast path so the generated bytecode does not introduce an extra Tuple2 allocation.

Result:
Successful strict .json imports no longer allocate a scala.Some wrapper while preserving fallback behavior for non-JSON or invalid JSON inputs across evaluator and preloader paths. OptionVal's core semantics are covered by tests, and the preloader JSON fast path keeps the intended lower-allocation shape.

References:
https://github.com/apache/pekko/blob/main/actor/src/main/scala/org/apache/pekko/util/OptionVal.scala
@He-Pin He-Pin force-pushed the perf/option-val branch from 05f6525 to 2971331 Compare July 5, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant