Skip to content

fix: VLLM Supplier Dialogue Verification #2412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shaohuzhang1 merged 1 commit into from
Feb 26, 2025
Merged

fix: VLLM Supplier Dialogue Verification #2412

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions apps/setting/models_provider/impl/vllm_model_provider/credential/llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,11 @@ def is_valid(self, model_type: str, model_name, model_credential: Dict[str, obje
model = provider.get_model(model_type, model_name, model_credential, **model_params)
try:
res = model.invoke([HumanMessage(content=gettext('Hello'))])
print(res)
except Exception as e:
print(e)
raise AppApiException(ValidCode.valid_error.value,
gettext(
'Verification failed, please check whether the parameters are correct: {error}').format(
error=str(e)))
return True

def encryption_dict(self, model_info: Dict[str, object]):
Copy link
Copy Markdown
Contributor Author

@shaohuzhang1 shaohuzhang1 Feb 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks mostly well-written, but there's one issue that needs improvement: the exception handling for invalid calls to get_model might not be sufficient for all cases.

Improvement Suggestions:

  1. Ensure Proper Error Handling: Add checks after calling get_model() to verify if model is not null or returns an error value before attempting to invoke it. This can help prevent unexpected failures from unhandled errors. 
    def is_valid(self, model_type: str, model_name, model_credential: Dict[str, object], model_params):
    # Ensure proper model retrieval and initial validation
    model = provider.get_model(model_type, model_name, model_credential, **model_params)
    if model is None:  # Handle case where model retrieval fails
        raise AppApiException(ValidCode.model_not_found.value,
                             gettext('Model not found'))

    try:
        res = model.invoke([HumanMessage(content=gettext('Hello'))])
       # Print response for debugging purposes (optional)
    except Exception as e:
        raise AppApiException(ValidCode.valid_error.value,
                            gettext('Verification failed, please check whether the parameters are correct: {error}')
                                .format(error=str(e)))

    return True

By adding these modifications, you improve robustness by checking if the model was successfully retrieved before proceeding with other operations, which can reduce runtime exceptions due to missing models. Also note that using None here is more precise than relying on truthiness for provider.get_model() responses.

Expand Down