Merge pull request #1692 from AI-Hypercomputer:bos_jetstream

maxtext authors · maxtext authors · commit 0add45d734a7 · 2025-05-07T16:09:32.000-07:00
PiperOrigin-RevId: 756044581
diff --git a/MaxText/generate_distillation_data.py b/MaxText/generate_distillation_data.py
@@ -80,6 +80,7 @@ async def send_request(config, request, stub, tokenizer, progress_bar):  # pylin
       token_content=jetstream_pb2.DecodeRequest.TokenContent(token_ids=prompt_token_ids),
       max_tokens=request.max_output_tokens,
       num_samples=config.num_generations,  # number of responses to generate for each request
+      has_bos=True,
   )
 
   response = stub.Decode(decode_request)

Original file line number	Diff line number	Diff line change
`@@ -80,6 +80,7 @@ async def send_request(config, request, stub, tokenizer, progress_bar): # pylin`
`80`	`80`	`token_content=jetstream_pb2.DecodeRequest.TokenContent(token_ids=prompt_token_ids),`
`81`	`81`	`max_tokens=request.max_output_tokens,`
`82`	`82`	`num_samples=config.num_generations, # number of responses to generate for each request`
	`83`	`+ has_bos=True,`
`83`	`84`	`)`
`84`	`85`
`85`	`86`	`response = stub.Decode(decode_request)`