Use metadata-backed document boundaries for SFT datasets#653

taivu1998 wants to merge 1 commit into

allenai:mainallenai/OLMo-core:mainfrom

taivu1998:tdv/issue-594-boundary-metadatataivu1998/OLMo-core:tdv/issue-594-boundary-metadataCopy head branch name to clipboard