Skip to content

Commit 4731a12

Browse files
yeyu-nvidiaclaude
andcommitted
fix: catch ValueError in parquet fallback for HF datasets compat
HF datasets raises ValueError (not just TypeError) when it encounters unknown feature types in embedded parquet metadata. Catch both so the PyArrow fallback triggers correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ye Yu <yeyu@nvidia.com>
1 parent 94fadbd commit 4731a12

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • examples/specdec_bench/specdec_bench/datasets

examples/specdec_bench/specdec_bench/datasets/speed.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -718,7 +718,7 @@ def _load_dataset(self, config_name_or_dataset_path: config_type | str) -> "Data
718718
data_files = {"test": [str(config_name_or_dataset_path_path)]}
719719
try:
720720
dataset = load_dataset("parquet", data_files=data_files, split="test")
721-
except TypeError:
721+
except (TypeError, ValueError):
722722
# Fallback: parquet metadata may be incompatible with the installed
723723
# ``datasets`` version. Read via PyArrow and convert directly.
724724
import pyarrow

0 commit comments

Comments
 (0)