Skip to content

feat: enhance image generation and tools use#115

Merged
chenyme merged 1 commit into
devfrom
imagine_and_tools
Jun 3, 2026
Merged

feat: enhance image generation and tools use#115
chenyme merged 1 commit into
devfrom
imagine_and_tools

Conversation

@chenyme

@chenyme chenyme commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

This change improves model request option configuration across the backend and admin/user UI.

The backend now centralizes official provider-native tool definitions in a catalog, uses model capability JSON (nativeToolKeys) to control which native tools may be canonicalized, and defaults supported image response protocols to base64 output where applicable. Gemini image generation parameter handling was aligned with the actual request shape (generationConfig.imageConfig) and model-specific constraints.

The frontend model capability editor now supports visual configuration for defaultOptions, optionControls, and nativeToolKeys, including validation, localized descriptions, protocol compatibility hints, and a clearer fixed-header/scrolling-list dialog layout. Official native tools are configured from model capability JSON instead of the Tools settings page. The admin Tools page remains focused on MCP tools.

Change type

  • Bug fix
  • Feature
  • Documentation
  • Refactor
  • Configuration / deployment
  • Security hardening
  • Other

Affected areas

  • Frontend / UI
  • Backend / API
  • Authentication / authorization
  • Conversations / streaming
  • Files / RAG / extraction
  • Model routing / providers
  • MCP / tools
  • Billing / payments
  • Admin console
  • Deployment / Docker / configuration
  • Documentation

Verification

  • cd backend && go test ./...
  • cd frontend && pnpm lint
  • cd frontend && pnpm build
  • Not run; reason:

Screenshots, API examples, or logs

UI behavior changed in the model capability quick configuration dialog and chat model configuration dialog:

  • parameter controls validate required fields before applying
  • new rows are inserted at the top
  • option values are displayed and parsed as comma-separated lists
  • tabs, help text, and action buttons remain fixed while the form list scrolls
  • native tools show protocol compatibility badges/tooltips

No secrets or personal data are included.

Configuration, migration, and compatibility notes

  • Official native tool enablement moved from global tool settings into model capability JSON via nativeToolKeys.
  • User-side JSON options remain the final request options; visual controls only write values after user interaction.
  • Backend canonicalizes only catalog-known official native tools allowed by the selected model’s nativeToolKeys.
  • MCP tools remain admin-configured and are not user-defined through JSON.
  • OpenAI/xAI image responses default to b64_json where the protocol supports response format.
  • Gemini image generation options now use generationConfig.imageConfig as the backend request standard.
  • Swagger docs were regenerated for updated settings/billing API contracts.

Documentation

  • Documentation is not needed for this change.
  • Documentation was updated.
  • Documentation still needs to be updated.

Security and privacy

  • No secrets, tokens, credentials, local config, or personal data are included.
  • User data access remains scoped by authenticated user context unless an admin-only path explicitly requires broader access.
  • Security-sensitive behavior was reviewed, including authentication, authorization, provider routing, file processing, billing, and admin APIs where relevant.

Checklist

  • I searched existing issues and pull requests.
  • Changes are focused and do not include unrelated refactors.
  • Tests or static verification were run where practical.
  • User-facing behavior, deployment steps, API contracts, or configuration changes are documented.
  • Generated artifacts are included only when this project explicitly requires them.
  • Caches, build output, .pyc files, .env files, and local storage data are not committed.

@chenyme chenyme linked an issue Jun 3, 2026 that may be closed by this pull request
3 tasks
@chenyme
chenyme merged commit a25bedb into dev Jun 3, 2026
4 checks passed
@chenyme
chenyme deleted the imagine_and_tools branch June 3, 2026 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: 可视化配置中的工具应该是可定义的

1 participant