feat: Specify the directory for calculating tokens models#4331
feat: Specify the directory for calculating tokens models#4331shaohuzhang1 merged 1 commit intov2from
Conversation
|
Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
56a22e0 to
49b8fd9
Compare
| os.environ['TIKTOKEN_CACHE_DIR'] = '/opt/maxkb-app/model/tokenizer/openai-tiktoken-cl100k-base' | ||
| application = get_wsgi_application() | ||
|
|
||
|
|
There was a problem hiding this comment.
The given code looks mostly correct, but there is one suggestion for improvement:
# Change import line to avoid redefining __import__
builtins.__import__ = TorchBlocker()
# Ensure you set DJANGO_SETTINGS_MODULE correctly before starting the WSGI app
if not Django.get_settings_module():
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'maxkb.settings')
application = get_wsgi_application()Explanation of Improvements:
- Avoid Redefinition: The first change ensures that
builtins.__import__remains unchanged after being reassigned withTorchBlocker(). This avoids any unintended side effects on other parts of the codebase. - Set Environment Variable Early: The second condition checks if Django's settings module has already been initialized (
Django.get_settings_module()), and sets it only if needed. This prevents unnecessary calls tosetdefault, especially if the environment variable was already set elsewhere.
These changes might seem minor, but they maintain clean and consistent coding practices.
feat: Specify the directory for calculating tokens models