Skip to content

Commit 2bd994a

Browse files
text input that could be read as NA/None is now read as a string
1 parent 7473f01 commit 2bd994a

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

src/cnlpt/cnlp_processors.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@ def __init__(self, data_dir: str, tasks: set[str] = None, max_train_items=-1):
167167
else:
168168
sep = "\t"
169169

170-
self.dataset = load_dataset("csv", sep=sep, data_files=data_files)
170+
self.dataset = load_dataset("csv", sep=sep, data_files=data_files, keep_default_na=False)
171171

172172
## find out what tasks are available to this dataset, and see the overlap with what the
173173
## user specified at the cli, remove those tasks so we don't also get them from other datasets

0 commit comments

Comments
 (0)