feat: model name Filter#172
Conversation
Signed-off-by: Arad Halevy <arad.halevy@ibm.com>
Signed-off-by: Arad Halevy <arad.halevy@ibm.com>
There was a problem hiding this comment.
not sure I understood the intention in this PR.
when a model is specified in the body, we shouldn't run model selector at all.
(when it is a single model).
this should be done by creating a profile picker that checks whether model is set in the body or not, and if model is already set, then select a profile without model selector.
regarding the use case of multiple models in an array alike value.. this is an invented syntax, no cli or client library for openai (or other formats) would work that way.
I would suggest to do some changes here:
- if your intention is to restrict to a single model - that should be done using new ProfilePicker.
- if your intention is to restrict to a subset of models, I'd be happy to get more details on this. assuming we have models a,b,c,d,e and this filter allows only b,c. then why do we need models a,d,e? will those ever be used? why are they deployed?
- if it's still a valid case - I think it's legit to specify a subset of allowed models in a header, not inside the body. (same is done in epp btw, using subset header).
Signed-off-by: Arad Halevy <arad.halevy@ibm.com>
Signed-off-by: Arad Halevy <arad.halevy@ibm.com>
Signed-off-by: Arad Halevy <arad.halevy@ibm.com>
|
@nirrozenbaum Changed the Filter to allow Filter the model name exactly, else an empty field and then we let all the possible models work |
@aradhalevy thanks. this looks better. wdyt? |
What type of PR is this?
What this PR does / why we need it:
Adds a
model-name-filterthat restricts the candidate models to the model name requested in the body.The configured request-body fieldis read as a string:
"[]"→ no constraint, all candidates pass through.Which issue(s) this PR fixes:
Fixes #171
Release note (write
NONEif no user-facing change):