added einops for embedding models and simplified accuracy description#4207
Open
dtrawins wants to merge 1 commit into
Open
added einops for embedding models and simplified accuracy description#4207dtrawins wants to merge 1 commit into
dtrawins wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Updates demo documentation around accuracy evaluation and model export, and adds a missing Python dependency (einops) needed by some embedding/export workflows.
Changes:
- Simplifies continuous batching accuracy demo instructions by linking to other deployment demos and updates the VLM evaluation command.
- Adds
einopsto the export-models demo Python requirements. - Replaces a long CLI help “Expected Output” block in the export-models README with a short compatibility note about
transformersversions.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
demos/continuous_batching/accuracy/README.md |
Simplifies server startup guidance (links to other demos) and adjusts VLM eval command; retains example outputs. |
demos/common/export_models/requirements.txt |
Adds einops dependency to export-model requirements. |
demos/common/export_models/README.md |
Removes verbose help output and adds a note about potential transformers version requirements. |
Comment on lines
17
to
+22
| ## Starting the model server | ||
|
|
||
| ### With Docker | ||
| ```bash | ||
| docker run -d --rm -p 8000:8000 -v $(pwd)/models:/workspace:ro openvino/model_server:latest --rest_port 8000 --config_path /workspace/config.json | ||
| ``` | ||
|
|
||
| ### On Baremetal | ||
| ```bash | ||
| ovms --rest_port 8000 --config_path ./models/config.json | ||
| ``` | ||
| Example of LLM and VLM models deployment is documented in other demos like | ||
| [Agentic usage for LLM models](../agentic_ai/README.md) | ||
| [Using VLM models](../vlm/README.md) |
Comment on lines
70
to
74
| python -m lmms_eval \ | ||
| --model openai_compatible \ | ||
| --model_args model_version=OpenGVLab/InternVL2_5-8B,max_retries=1 \ | ||
| --model_args model_version=OpenVINO/InternVL2_5-8B_int4-ov,max_retries=1 \ | ||
| --tasks mme,mmmu_val \ | ||
| --batch_size 1 \ |
| --enable_tool_guided_generation | ||
| Enables enforcing tool schema during generation. Requires setting tool_parser | ||
| ``` | ||
| > Note: Exporting some models might require different transformers version than specified in requirements.txt Check [supported models](https://openvinotoolkit.github.io/openvino.genai/docs/supported-models/). If custom transformers version is required, install it afterwards via `pip install transformers==<version>` |
| --enable_tool_guided_generation | ||
| Enables enforcing tool schema during generation. Requires setting tool_parser | ||
| ``` | ||
| > Note: Exporting some models might require different transformers version than specified in requirements.txt Check [supported models](https://openvinotoolkit.github.io/openvino.genai/docs/supported-models/). If custom transformers version is required, install it afterwards via `pip install transformers==<version>` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🛠 Summary
CVS-186324
🧪 Checklist
``