Skip to content

Commit 797b650

Browse files
branczclaude
andauthored
fix(vortex-bench): map gs:// scheme to gcs storage label (#8630)
## Rationale for this change Running the benchmark harness (`datafusion-bench` / `query_bench`) against a remote dataset on Google Cloud Storage (`--opt remote-data-dir=gs://…`) fails immediately during benchmark setup with: ``` Error: unknown URL scheme: gs ``` `vortex-bench`'s `url_scheme_to_storage` helper — which maps a data-dir URL scheme to a `storage` label used for result reporting — only handled `s3` and `file`, so any `gs://` run bailed before a single query executed. S3 remote runs work because `s3` is handled; GCS was simply never covered. `make_object_store` already supports `gs://` for the actual reads, so the only gap was this reporting helper. ## What changes are included in this PR? - Add a `STORAGE_GCS = "gcs"` constant. - Add a `"gs"` arm to `url_scheme_to_storage` returning that label. Verified by running TPC-H SF1 from a GCS bucket end-to-end (DataFusion + Vortex, 22/22 queries executing against `gs://…`, results tagged `storage=gcs`). ## What APIs are changed? Are there any user-facing changes? None. This only affects the benchmark harness's storage-label reporting; no public API, format, or behavior change outside `vortex-bench`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 5d3be01 commit 797b650

2 files changed

Lines changed: 3 additions & 0 deletions

File tree

vortex-bench/src/utils/constants.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@
44
/// Storage type constants
55
pub const STORAGE_NVME: &str = "nvme";
66
pub const STORAGE_S3: &str = "s3";
7+
pub const STORAGE_GCS: &str = "gcs";

vortex-bench/src/utils/file.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,11 +161,13 @@ pub fn resolve_data_url(remote_data_dir: Option<&str>, local_subdir: &str) -> Re
161161
/// - A storage type string ("s3", "nvme")
162162
/// - Or an error if the scheme is unknown
163163
pub fn url_scheme_to_storage(url: &Url) -> Result<String> {
164+
use super::constants::STORAGE_GCS;
164165
use super::constants::STORAGE_NVME;
165166
use super::constants::STORAGE_S3;
166167

167168
match url.scheme() {
168169
STORAGE_S3 => Ok(STORAGE_S3.to_owned()),
170+
"gs" => Ok(STORAGE_GCS.to_owned()),
169171
"file" => Ok(STORAGE_NVME.to_owned()),
170172
otherwise => {
171173
bail!("unknown URL scheme: {}", otherwise)

0 commit comments

Comments
 (0)