perf: optimize the json newline scanning

This is an alternative approach to
#19687

Instead of reading the entire range in the json FileOpener, implement an
AlignedBoundaryStream which scans the range for newlines as the
FileStream requests data from the stream, by wrapping the original
stream returned by the ObjectStore.

This eliminated the overhead of the extra two get_opts requests needed
by calculate_range and more importantly, it allows for efficient
read-ahead implementations by the underlying ObjectStore. Previously
this was inefficient because the streams opened by calculate_range
included a stream from (start - 1) to file_size and another one from
(end - 1) to end_of_file, just to find the two relevant newlines.

Author: ariel-miculas

Cargo.lock

@@ -185,7 +185,7 @@ strum = "0.28.0"
 strum_macros = "0.28.0"
 tempfile = "3"
 testcontainers-modules = { version = "0.15" }
-tokio = { version = "1.48", features = ["macros", "rt", "sync"] }
+tokio = { version = "1.48", features = ["macros", "rt", "sync", "fs"] }
 tokio-stream = "0.1"
 tokio-util = "0.7"
 url = "2.5.7"

Cargo.toml

@@ -46,6 +46,7 @@ futures = { workspace = true }
 object_store = { workspace = true }
 tokio = { workspace = true }
 tokio-stream = { workspace = true, features = ["sync"] }
+tokio-util = { workspace = true, features = ["io"] }
 
 # Note: add additional linter rules in lib.rs.
 # Rust does not support workspace + new linter rules in subcrates yet