ValueError: Logits processors are not supported in multi-step decoding

[christopherzc]

When I try to run one of the example notebooks with the latest vllm version, I get this error. Is there a specific vllm version I need to run this? My vllm version is 0.10.1.1

[christopherzc]

This went away when I downgraded to 0.10.0 but maybe worth pinning the version?

[jwkirchenbauer]

Two relevant other pointers on the vllm repo side:

vllm-project/vllm#15636 (comment)

vllm-project/vllm#23656 (comment)

In summary, as part of the migration to the V1 engine there is some breaking refactoring of the logits processor machinery. Currently a fallback to V0 is required, as well as reverting to the earlier vllm release 0.10.0 @ 6d8d0a2 as suggested above.

However, my hope is that this actually gets "fixed" in a more elegant way, there's just a lot of work happening over there that is touching this at the moment I think.

My cross link to try and keep this on the vllm team's radar:
vllm-project/vllm#23656 (comment)