Skip to content

Commit 78159d0

Browse files
committed
fix small bug
1 parent 868a124 commit 78159d0

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

src/lmflow/pipeline/finetuner.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,8 @@ def group_text(self, tokenized_datasets, model_max_length):
149149
f"({model_max_length})."
150150
f"Using block_size={data_args.block_size}.")
151151
block_size = data_args.block_size
152-
152+
else:
153+
block_size = data_args.block_size
153154
# Main data processing function that will concatenate all texts from
154155
# our dataset and generate chunks of block_size.
155156
def group_texts(examples):

0 commit comments

Comments
 (0)