Skip to content

fix: Ollama maximum output token field #2973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shaohuzhang1 merged 1 commit into from
Apr 24, 2025
Merged

fix: Ollama maximum output token field #2973

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/setting/models_provider/impl/ollama_model_provider/credential/llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ class OllamaLLMModelParams(BaseForm):
_step=0.01,
precision=2)

max_tokens = forms.SliderField(
num_predict = forms.SliderField(
TooltipLabel(_('Output the maximum Tokens'),
_('Specify the maximum number of tokens that the model can generate')),
required=True, default_value=1024,
Copy link
Copy Markdown
Contributor Author

@shaohuzhang1 shaohuzhang1 Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code provided in the comment about optimizing OllamaLLMModelParams appears to be describing changes made to an existing form schema using some UI widget classes. Specifically:

  1. Parameter Changes:

    • An attribute named max_tokens (slider) was changed to a new parameter num_predict.

  2. Additional Fields:

    • No additional fields were added.

  3. General Considerations:
    The change from one field name (max_tokens) to another (num_predict) suggests that these might represent conceptually similar parameters but with slightly different meanings or purposes within the context of the application.

    • It's important to ensure that both parameters serve a consistent purpose throughout the entire system.
    • If num_predict is meant to replace max_tokens, there should likely be a note in comments explaining this replacement and confirming its equivalence meaningfully.

Here are some general steps to consider for further verification and improvement:

General Verification Steps:

  1. Contextual Understanding:
    Ensure you have a clear understanding of the roles that "output the maximum Tokens" and "generate the expected number of tokens" play in the larger project architecture.

  2. Consistency Check:
    Confirm that the use of num_predict accurately reflects its intended functionality, especially if other parts of the code also rely on it under a previous identifier.

  3. Documentation Update:
    Double-check if any documentation needs updating to reflect the renaming of the field from max_tokens to num_predict.

  4. Unit Tests:
    Write unit tests to validate behavior around the interaction between this slider and other related logic.

  5. Review by Collaborators:
    Share updates with team members to gather feedback and address concerns before deploying significant changes.

Optimizations and Suggestions:

  • Type Consistency:
    Maintain consistency in type definitions. While not strictly necessary for a simple SliderField, ensuring uniformity across types can prevent runtime errors.
# Example: Replacing 'max_tokens' with 'num_predict'
class OllamaLLMModelParams(BaseForm):
    # ...
    num_predict = forms.SliderField(  # Use pluralize the term as per semantic correctness
        TooltipLabel(_('Output the Maximum Number of Predictions/Tokens')),  # Clarify the role clearly
        _('Specify how many predictions/tokens the model will make'),  # Specify what the model actually does
        required=True, default_value=1024, min_value=0,
        step=1, max_value=4096  # Adjust values based on specific requirements/supported limits
    )

This approach ensures clarity, enhances maintainability, and prevents unintended side effects from subtle changes like renaming parameters.

Expand Down