Skip to content

feat: enrich /v1/models with context_length, pricing, and max_output_tokens#1781

Open
sanwzzz wants to merge 1 commit intoWei-Shaw:mainfrom
sanwzzz:feat/models-metadata-1777
Open

feat: enrich /v1/models with context_length, pricing, and max_output_tokens#1781
sanwzzz wants to merge 1 commit intoWei-Shaw:mainfrom
sanwzzz:feat/models-metadata-1777

Conversation

@sanwzzz
Copy link
Copy Markdown

@sanwzzz sanwzzz commented Apr 21, 2026

Summary

  • Enhance /v1/models to optionally include: context_length, max_output_tokens, pricing.
  • Metadata source priority: pricingService (LiteLLM pricing metadata) -> embedded fallback defaults.
  • Keep backwards compatibility: fields are omitted when unknown.

Why

  • Clients like Hermes rely on these fields to display correct context window and pricing.

Implementation notes

  • Added internal/pkg/modelmetadata with embedded default model metadata JSON for stable tests (no cwd dependency).
  • Extended pricing_service parsing to support max_input_tokens/max_output_tokens/max_tokens.

Tests

  • go test -tags=unit ./internal/handler ./internal/service -count=1

Closes #1777

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 21, 2026

All contributors have signed the CLA. ✅
Posted by the CLA Assistant Lite bot.

@sanwzzz sanwzzz force-pushed the feat/models-metadata-1777 branch from eca3b00 to a7db5a1 Compare April 21, 2026 09:43
@sanwzzz
Copy link
Copy Markdown
Author

sanwzzz commented Apr 21, 2026

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Apr 21, 2026
@sanwzzz
Copy link
Copy Markdown
Author

sanwzzz commented Apr 21, 2026

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: 为 /v1/models 增加 context_length、pricing、max_output_tokens 等可选字段

1 participant