You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: clarify provider registration, dot-notation, add non-OpenAI provider example
- Q1: Explain how setting env vars triggers auto-discovery and pool formation
- Q3: Explain how capability pools are formed from provider model tags
- Q5: Clarify relationship between shortcuts and dot-notation paths
- Q10: Add complete BaseProvider example for non-OpenAI-compatible APIs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
**How does this work?** Setting `OPENAI_API_KEY` triggers auto-discovery: ModelMesh finds the OpenAI provider, registers its models, and groups them into **capability pools** by what each model can do. `create("chat-completion")` returns a client wired to the pool containing all chat-capable models. The shortcut `"chat-completion"` resolves to the full dot-notation path `generation.text-generation.chat-completion` automatically (see [Q5](#5-what-does-request-capabilities-not-model-names-mean)).
29
+
28
30
When you need more control, add a YAML file or pass options programmatically. All three layers compose: env vars for secrets, YAML for topology, code for runtime overrides.
29
31
30
32
```python
@@ -109,6 +111,8 @@ for i in range(100):
109
111
110
112
Your code makes the same call every time. The library handles detection, pooling, and rotation internally.
111
113
114
+
**How are pools formed?** Each provider registers its models with capability tags (e.g. `generation.text-generation.chat-completion`). ModelMesh groups all models sharing a capability into a single pool. When you call `create("chat-completion")`, you get a client backed by every chat-capable model across all discovered providers. Adding a new API key adds that provider's models to the existing pools automatically.
115
+
112
116
See the [Free-Tier Aggregation](QuickStart.md) guide.
**Shortcuts vs dot-notation:** Every capability has a full dot-notation path reflecting its position in the hierarchy tree (e.g. `generation.text-generation.chat-completion`). Shortcuts like `"chat-completion"` are leaf-node aliases that resolve automatically. Both forms work everywhere: `create("chat-completion")` and `create("generation.text-generation.chat-completion")` are equivalent. Providers tag their models with full paths; you use whichever form is convenient.
183
+
178
184
When a new model launches or an old one is deprecated, update your config. Your application code stays the same.
179
185
180
186
See the [Capability Discovery](Capabilities.md) guide.
When your API doesn't follow the OpenAI format, inherit from `BaseProvider` and override four hook methods. BaseProvider handles HTTP transport, retries, and error classification; you only translate the request and response formats.
403
+
404
+
```python
405
+
from modelmesh.cdk import BaseProvider, BaseProviderConfig
Override only what differs: `_get_completion_endpoint()` for the URL path, `_build_headers()` for authentication, `_build_request_payload()` to translate the request format, and `_parse_response()` to translate the response back. For streaming, also override `_parse_sse_chunk()`.
461
+
394
462
Six connector types are extensible this way: providers, rotation policies, secret stores, storage backends, observability sinks, and discovery connectors.
395
463
396
464
See the [CDK](../ConnectorCatalogue.md) reference and [CDK Developer Guide](../cdk/DeveloperGuide.md).
0 commit comments