Skip to content

Commit c7bd12f

Browse files
committed
updated to latest code changes in grain
1 parent 0d7c747 commit c7bd12f

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

src/maxtext/input_pipeline/data_processing_utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626

2727
def parse_and_keep_features(dataset, config, data_columns, tokenize):
2828
"""Parse arrayrecord features or keep specified columns for other formats."""
29-
if config.grain_file_type == "arrayrecord":
29+
if config.grain_file_type in ("arrayrecord", "tfrecord"):
3030
dataset = dataset.map(input_pipeline_utils.ParseFeatures(data_columns, tokenize))
3131
dataset = dataset.map(input_pipeline_utils.NormalizeFeatures(data_columns, tokenize))
3232
else:

0 commit comments

Comments
 (0)