Summary and motivation
ChatGenerator Motivation:
from #1958
When using OpenAIChatGenerator with OpenAI-compatible APIs that return additional custom parameters in the response delta (such as reasoning_content, thinking_content, or other provider-specific fields), these parameters are currently ignored and lost during the streaming chunk conversion process.
For example, when using APIs that provide reasoning capabilities or additional metadata, the current implementation only extracts standard OpenAI fields (content, tool_calls, etc.) and discards any custom fields that might be present in the choice.delta object.
Reranker Motivation:
vLLM supports hosting of models like Qwen3-Reranker which are a new style of powerful rerankers based on the Qwen3ForCausalLM architecture in HuggingFace. Supporting causal LLMs for reranking has not yet been standardized so often times specific preprocessing and post-processing is needed to get these models to work. For example check out the example code here for the Qwen3-Reranker.
We have clients who would like to use this model for reranking via vLLM hosting so it would be nice to add a component in this integration to support that.
Checklist
If the request is accepted, ensure the following checklist is complete before closing this issue.
Tasks
Summary and motivation
ChatGenerator Motivation:
from #1958
Reranker Motivation:
vLLM supports hosting of models like Qwen3-Reranker which are a new style of powerful rerankers based on the
Qwen3ForCausalLMarchitecture in HuggingFace. Supporting causal LLMs for reranking has not yet been standardized so often times specific preprocessing and post-processing is needed to get these models to work. For example check out the example code here for the Qwen3-Reranker.We have clients who would like to use this model for reranking via vLLM hosting so it would be nice to add a component in this integration to support that.
Checklist
If the request is accepted, ensure the following checklist is complete before closing this issue.
Tasks
mainbranchintegration:<your integration name>has been added to the list of labels for this repository