Skip to content

chore: drop metrics from schema#277

Merged
viraatc merged 1 commit intomainfrom
feat/viraatc-drop-metrics-schema
Apr 15, 2026
Merged

chore: drop metrics from schema#277
viraatc merged 1 commit intomainfrom
feat/viraatc-drop-metrics-schema

Conversation

@viraatc
Copy link
Copy Markdown
Collaborator

@viraatc viraatc commented Apr 10, 2026

What does this PR do?

based on offline discussion, we want to drop metrics from schema (it was unused anyways).

Type of change

  • Bug fix
  • New feature
  • Documentation update
  • Refactor/cleanup

Related issues

Testing

  • Tests added/updated
  • All tests pass locally
  • Manual testing completed

Checklist

  • Code follows project style
  • Pre-commit hooks pass
  • Documentation updated (if needed)

@viraatc viraatc requested review from a team and Copilot April 10, 2026 01:13
@github-actions
Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes the unused metrics configuration section from the benchmark configuration schema and updates templates/examples/tests accordingly so configs no longer include or validate that field.

Changes:

  • Removed the Metrics Pydantic model and metrics field from BenchmarkConfig.
  • Updated config templates and example YAMLs to drop the metrics: section.
  • Updated unit tests to stop including/validating metrics in YAML/schema tests.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/unit/config/test_yaml_loader.py Removes metrics from a valid YAML config used in loader tests.
tests/unit/config/test_schema.py Removes Metrics import and the Metrics unit tests.
src/inference_endpoint/config/templates/submission_template.yaml Drops metrics section from submission template.
src/inference_endpoint/config/templates/online_template_full.yaml Drops metrics section from online full template.
src/inference_endpoint/config/templates/offline_template_full.yaml Drops metrics section from offline full template.
src/inference_endpoint/config/templates/eval_template.yaml Drops metrics section from eval template.
src/inference_endpoint/config/templates/concurrency_template_full.yaml Drops metrics section from concurrency full template.
src/inference_endpoint/config/schema.py Removes Metrics model and BenchmarkConfig.metrics field.
examples/08_Qwen3-VL-235B-A22B_Example/online_qwen3_vl_235b_a22b_shopify.yaml Drops metrics section from example config.
examples/08_Qwen3-VL-235B-A22B_Example/offline_qwen3_vl_235b_a22b_shopify.yaml Drops metrics section from example config.
examples/06_Llama2-70B_Example/online_llama2_70b_orca.yaml Drops metrics section from example config.
examples/05_Llama3.1-8B_Example/online_llama3_8b_cnn.yaml Drops metrics section from example config.
examples/05_Llama3.1-8B_Example/offline_llama3_8b_cnn.yaml Drops metrics section from example config.
examples/04_GPTOSS120B_Example/vllm_gptoss_120b_example.yaml Drops metrics section from example config.
examples/04_GPTOSS120B_Example/sglang_gptoss_120b_example.yaml Drops metrics section from example config.
examples/04_GPTOSS120B_Example/gptoss_120b_example.yaml Drops metrics section from example config.
examples/02_ServerBenchmarking/online_llama2_70b_cnn.yaml Drops metrics section from example config.
examples/02_ServerBenchmarking/offline_llama3_8b_cnn.yaml Drops metrics section from example config.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes the metrics collection configuration and associated logic from the codebase, including the Metrics model, its helper methods, and all references in YAML configuration templates and unit tests. I have no feedback to provide as the changes are straightforward and clean.

Copy link
Copy Markdown
Collaborator

@arekay-nv arekay-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Can you also check if the templates were generated or edited - i don't see the generation script updated, so it might be broken if the templates weren't generated.

@viraatc
Copy link
Copy Markdown
Collaborator Author

viraatc commented Apr 15, 2026

confirmed - templates checked in by precommit, which was triggered by change to schema.py already.

@viraatc viraatc merged commit 40a11df into main Apr 15, 2026
12 checks passed
@viraatc viraatc deleted the feat/viraatc-drop-metrics-schema branch April 15, 2026 21:34
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants