does the budget forcing works?

[maziye-azad]

Hi, thanks for your great work.
i am reproducing the evaluations. i want to know, does the budget forcing works in eval/commands.sh?it seems as if no final answer is appended.
thanks

[Muennighoff]

yeah max_tokens_thinking activates budget forcing

[maziye-azad]

yeah max_tokens_thinking activates budget forcing

thank you for your answer.
that's true, but it cuts the reasoning chain, and after that there is no final answer, so i do not get the final answer. what is the reason?

[Muennighoff]

it should insert this to force a final answer:

s1/eval/lm-evaluation-harness/lm_eval/models/vllm_causallms.py

Line 254 in 77272c6

thinking_end_max = thinking_end + "\nFinal Answer:"

[maziye-azad]

aha,
thank you for your answer.