feat(charts): Add vllm-omni container support in API deployment

[Banh-Canh]

Signed-off-by: Victor Hang vhvictorhang@gmail.com

[Banh-Canh]

https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/quickstart/#online-serving-with-openai-completions-api

i believe we need to run this with --omni flag. I did test running an instance.

[Alex-Welsh]

Are you sure this is required? When I originally tested vllm-omni, this was the default entrypoint (vllm server --omni). Though that was a pre-release. Has that default changed?

[Banh-Canh]

Are you sure this is required? When I originally tested vllm-omni, this was the default entrypoint (vllm server --omni). Though that was a pre-release. Has that default changed?

I tested with image: vllm/vllm-omni:v0.18.0

else the container doesn't start with:

ess: exec: "mistralai/Voxtral-4B-TTS-2603": stat mistralai/Voxtral-4B-TTS-2603: no such file or directory: unknown 

[Alex-Welsh]

LGTM, just one question. I'm happy with this either way though

[Banh-Canh]

LGTM, just one question. I'm happy with this either way though

I think it's fine as it is, so the upstream provider can just do whatever it wants and it won't have to update the entrypoint here (at least, hopefully most of the time)

For omni we have to.. but well that's how omni is packaged so..

thanks for the review

[Alex-Welsh]

I believe these are the default entrypoints already

[Banh-Canh]

I believe these are the default entrypoints already

Yes so that's why we want to let upstream decide what is best for its app, just in case they change the entrypoint, we won't have to update our override

Which is why I would vote to not override forcibly for the base image, the entrypoint !

[Alex-Welsh]

Fix for CI not triggering is in progress #159
Once it's merged, rebase and it should work