Skip to content

Commit 95a9395

Browse files
yeyu-nvidiaclaude
andcommitted
fix: catch ValueError in parquet fallback for HF datasets compat
HF datasets raises ValueError (not just TypeError) when it encounters unknown feature types in embedded parquet metadata. Catch both so the PyArrow fallback triggers correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ye Yu <yeyu@nvidia.com>
1 parent 455759c commit 95a9395

File tree

1 file changed

+1
-1
lines changed
  • examples/specdec_bench/specdec_bench/datasets

1 file changed

+1
-1
lines changed

examples/specdec_bench/specdec_bench/datasets/speed.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -718,7 +718,7 @@ def _load_dataset(self, config_name_or_dataset_path: config_type | str) -> "Data
718718
data_files = {"test": [str(config_name_or_dataset_path_path)]}
719719
try:
720720
dataset = load_dataset("parquet", data_files=data_files, split="test")
721-
except TypeError:
721+
except (TypeError, ValueError):
722722
# Fallback: parquet metadata may be incompatible with the installed
723723
# ``datasets`` version. Read via PyArrow and convert directly.
724724
import pyarrow

0 commit comments

Comments
 (0)