Skip to content

Commit 443dc4f

Browse files
apartsinclaude
apartsin
and
claude
committed
Add AI agent integration guide, Claude skills, fix TS build errors, and polish install paths
- Add docs/ForAIAgent.md: comprehensive guide for AI coding agents to install, configure, and integrate ModelMesh (Python, TypeScript, Docker) - Add ClaudeSkill/ folder with 5 skill definitions: install, configure, integrate, deploy-proxy, test - Add py.typed marker (PEP 561) for Python type checking support - Add exports field to TypeScript package.json for browser subpath (@modelmesh/core/browser) - Replace eslint lint script with tsc --noEmit (no eslint config needed) - Fix 3 TypeScript build errors: widen RUNTIME static property type on base classes to allow subclass overrides (RuntimeEnvironment enum) - Update README and docs/index.md with Docker install instructions - Update CoverageMatrix.md with ForAIAgent.md traceability All 1,366 tests pass (855 Python + 511 TypeScript). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent e31241b commit 443dc4f

20 files changed

+1383
-13
lines changed

‎ClaudeSkill/README.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# ModelMesh Claude Code Skills
2+
3+
This folder contains Claude Code skill definitions for working with ModelMesh.
4+
Each `.md` file is a self-contained skill that Claude Code can load and execute.
5+
6+
## Available Skills
7+
8+
| Skill | File | Description |
9+
|---|---|---|
10+
| **Install** | `install.md` | Install ModelMesh into any project (Python, TypeScript, or Docker) |
11+
| **Configure** | `configure.md` | Generate modelmesh.yaml config with providers, models, pools |
12+
| **Integrate** | `integrate.md` | Replace existing AI SDK calls with ModelMesh routing |
13+
| **Deploy Proxy** | `deploy-proxy.md` | Set up and deploy the Docker OpenAI proxy |
14+
| **Test** | `test.md` | Run ModelMesh test suite and verify integration |
15+
16+
## Usage with Claude Code
17+
18+
These skills can be loaded as custom commands in Claude Code. To use:
19+
20+
1. Copy the desired skill `.md` file into your project's `.claude/` directory
21+
2. Or reference them directly when asking Claude Code for help
22+
23+
## How Skills Work
24+
25+
Each skill file contains:
26+
- **Context**: What ModelMesh is and how it works
27+
- **Decision tree**: Questions to ask the user to determine the right approach
28+
- **Implementation steps**: Exact commands and code to execute
29+
- **Verification**: How to confirm the integration works

‎ClaudeSkill/configure.md‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# ModelMesh Configure Skill
2+
3+
## Purpose
4+
Generate a `modelmesh.yaml` configuration file tailored to the user's needs.
5+
6+
## Decision Steps
7+
8+
1. **Which providers?** Ask which API keys the user has available.
9+
2. **Which capabilities?** What does their app need?
10+
- Chat completion (text generation)
11+
- Text embeddings
12+
- Text-to-speech
13+
- Speech-to-text
14+
- Image generation
15+
- Code generation
16+
3. **Which rotation strategy?**
17+
- `stick-until-failure` (default, recommended) — stay with working model
18+
- `cost-first` — minimize spending
19+
- `round-robin` — spread load evenly
20+
- `latency-first` — fastest response
21+
- `rate-limit-aware` — avoid rate limits
22+
4. **Budget controls?** Does the user want daily spend limits per provider?
23+
5. **Secret store?** How are API keys managed?
24+
- `modelmesh.env.v1` (default) — environment variables
25+
- `modelmesh.dotenv.v1` — .env file
26+
- Cloud options: AWS Secrets Manager, Google Secret Manager, Azure Key Vault
27+
28+
## Configuration Template
29+
30+
```yaml
31+
# modelmesh.yaml — Generated configuration
32+
secrets:
33+
store: modelmesh.env.v1
34+
35+
providers:
36+
# Add providers based on user's available API keys
37+
openai.llm.v1:
38+
api_key: ${secrets:OPENAI_API_KEY}
39+
budget:
40+
daily_limit: 10.00 # optional
41+
42+
anthropic.claude.v1:
43+
api_key: ${secrets:ANTHROPIC_API_KEY}
44+
45+
groq.api.v1:
46+
api_key: ${secrets:GROQ_API_KEY}
47+
48+
models:
49+
# Add models for each provider
50+
gpt-4o-mini:
51+
provider: openai.llm.v1
52+
capabilities:
53+
- generation.text-generation.chat-completion
54+
features:
55+
tool_calling: true
56+
structured_output: true
57+
json_mode: true
58+
system_prompt: true
59+
constraints:
60+
context_window: 128000
61+
max_output_tokens: 16384
62+
63+
claude-3-5-haiku:
64+
provider: anthropic.claude.v1
65+
capabilities:
66+
- generation.text-generation.chat-completion
67+
features:
68+
tool_calling: true
69+
system_prompt: true
70+
constraints:
71+
context_window: 200000
72+
max_output_tokens: 8192
73+
74+
llama-3.3-70b:
75+
provider: groq.api.v1
76+
capabilities:
77+
- generation.text-generation.chat-completion
78+
features:
79+
tool_calling: true
80+
system_prompt: true
81+
constraints:
82+
context_window: 131072
83+
max_output_tokens: 32768
84+
85+
pools:
86+
text-generation:
87+
strategy: modelmesh.stick-until-failure.v1
88+
capability: generation.text-generation
89+
```
90+
91+
## Provider Reference
92+
93+
| Provider ID | Env Var | Models |
94+
|---|---|---|
95+
| `openai.llm.v1` | `OPENAI_API_KEY` | gpt-4o, gpt-4o-mini, gpt-4-turbo |
96+
| `anthropic.claude.v1` | `ANTHROPIC_API_KEY` | claude-sonnet-4, claude-3-5-haiku |
97+
| `google.gemini.v1` | `GOOGLE_API_KEY` | gemini-2.0-flash, gemini-1.5-pro |
98+
| `groq.api.v1` | `GROQ_API_KEY` | llama-3.3-70b, mixtral-8x7b |
99+
| `deepseek.api.v1` | `DEEPSEEK_API_KEY` | deepseek-chat, deepseek-coder |
100+
| `mistral.api.v1` | `MISTRAL_API_KEY` | mistral-large, mistral-medium |
101+
| `together.api.v1` | `TOGETHER_API_KEY` | Various open-source models |
102+
| `openrouter.gateway.v1` | `OPENROUTER_API_KEY` | Multi-provider gateway |
103+
| `xai.grok.v1` | `XAI_API_KEY` | grok-2, grok-2-mini |
104+
| `cohere.nlp.v1` | `COHERE_API_KEY` | command-r, command-r-plus |
105+
106+
## Capability Paths
107+
108+
| Short Name | Full Path | Use For |
109+
|---|---|---|
110+
| `chat-completion` | `generation.text-generation.chat-completion` | Chat, Q&A, text generation |
111+
| `text-generation` | `generation.text-generation` | Broader text generation pool |
112+
| `text-embeddings` | `representation.embeddings.text-embeddings` | Semantic search, RAG |
113+
| `text-to-speech` | `generation.audio.text-to-speech` | Voice synthesis |
114+
| `speech-to-text` | `understanding.audio.speech-to-text` | Transcription |
115+
| `text-to-image` | `generation.image.text-to-image` | Image generation |
116+
| `code-generation` | `generation.text-generation.code-generation` | Code completion |
117+
118+
## Strategy Reference
119+
120+
| Strategy ID | Best For |
121+
|---|---|
122+
| `modelmesh.stick-until-failure.v1` | General use (default) |
123+
| `modelmesh.round-robin.v1` | Even load distribution |
124+
| `modelmesh.cost-first.v1` | Budget-sensitive apps |
125+
| `modelmesh.latency-first.v1` | Real-time applications |
126+
| `modelmesh.priority-selection.v1` | Preferred model with fallback |
127+
| `modelmesh.rate-limit-aware.v1` | High-volume apps |
128+
| `modelmesh.load-balanced.v1` | Weighted traffic splitting |
129+
| `modelmesh.session-stickiness.v1` | Conversation continuity |
130+
131+
## Verification
132+
133+
After creating the config, verify:
134+
135+
```python
136+
from modelmesh.config.mesh_config import MeshConfig
137+
config = MeshConfig.from_yaml("modelmesh.yaml")
138+
print(f"Providers: {len(config.providers)}")
139+
print(f"Models: {len(config.models)}")
140+
print(f"Pools: {len(config.pools)}")
141+
```

‎ClaudeSkill/deploy-proxy.md‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# ModelMesh Deploy Proxy Skill
2+
3+
## Purpose
4+
Set up and deploy the ModelMesh Docker proxy as an OpenAI-compatible REST API server.
5+
6+
## Prerequisites
7+
8+
- Docker and Docker Compose installed
9+
- At least one AI provider API key
10+
11+
## Quick Deploy
12+
13+
### Step 1: Get the Project
14+
15+
```bash
16+
git clone https://github.com/ApartsinProjects/ModelMesh.git
17+
cd ModelMesh
18+
```
19+
20+
### Step 2: Configure API Keys
21+
22+
```bash
23+
cp .env.example .env
24+
```
25+
26+
Edit `.env` and add your API keys:
27+
```env
28+
OPENAI_API_KEY=sk-...
29+
ANTHROPIC_API_KEY=sk-ant-...
30+
GROQ_API_KEY=gsk_...
31+
```
32+
33+
### Step 3: Configure Models and Pools
34+
35+
Edit `modelmesh.yaml` to define:
36+
- Which providers to use
37+
- Which models to expose
38+
- Which pools to create (capability groupings)
39+
- Rotation strategy
40+
41+
See the `configure.md` skill for detailed configuration options.
42+
43+
### Step 4: Build and Start
44+
45+
```bash
46+
docker compose up --build
47+
```
48+
49+
Or use the automation script:
50+
```bash
51+
./scripts/proxy-up.sh
52+
```
53+
54+
For detached (background) mode:
55+
```bash
56+
./scripts/proxy-up.sh --detach
57+
```
58+
59+
### Step 5: Verify
60+
61+
```bash
62+
# Health check
63+
curl http://localhost:8080/health
64+
65+
# List models and pools
66+
curl http://localhost:8080/v1/models
67+
68+
# Test chat completion
69+
curl -X POST http://localhost:8080/v1/chat/completions \
70+
-H "Content-Type: application/json" \
71+
-d '{"model":"text-generation","messages":[{"role":"user","content":"Hello!"}]}'
72+
```
73+
74+
Or use the smoke test script:
75+
```bash
76+
./scripts/proxy-test.sh
77+
```
78+
79+
## Authentication
80+
81+
To require a bearer token:
82+
83+
```bash
84+
docker compose up --build
85+
# Add to docker-compose.yaml command:
86+
command: ["--config", "/app/modelmesh.yaml", "--host", "0.0.0.0", "--port", "8080", "--token", "my-secret-token"]
87+
```
88+
89+
Clients must then include: `Authorization: Bearer my-secret-token`
90+
91+
## Proxy CLI Reference
92+
93+
```
94+
python -m modelmesh.proxy [OPTIONS]
95+
96+
Options:
97+
--config PATH YAML configuration file (default: auto-detect)
98+
--host HOST Bind address (default: 0.0.0.0)
99+
--port PORT Listen port (default: 8080)
100+
--token TOKEN Optional bearer token for authentication
101+
--log-level LEVEL Logging level: DEBUG, INFO, WARNING, ERROR (default: INFO)
102+
```
103+
104+
## API Endpoints
105+
106+
| Method | Path | Description |
107+
|---|---|---|
108+
| `GET` | `/health` | Health check (returns status, uptime, model count) |
109+
| `GET` | `/v1/models` | List available models and pools |
110+
| `POST` | `/v1/chat/completions` | Chat completion (streaming + non-streaming) |
111+
| `POST` | `/v1/embeddings` | Text embeddings |
112+
| `POST` | `/v1/audio/speech` | Text-to-speech |
113+
| `POST` | `/v1/audio/transcriptions` | Speech-to-text |
114+
115+
## Docker Compose Configuration
116+
117+
```yaml
118+
services:
119+
modelmesh-proxy:
120+
build: .
121+
ports:
122+
- "8080:8080"
123+
env_file: .env
124+
volumes:
125+
- ./modelmesh.yaml:/app/modelmesh.yaml:ro
126+
command: ["--config", "/app/modelmesh.yaml", "--host", "0.0.0.0", "--port", "8080"]
127+
```
128+
129+
## Stopping
130+
131+
```bash
132+
docker compose down
133+
# Or:
134+
./scripts/proxy-down.sh
135+
# Clean up volumes and images:
136+
./scripts/proxy-down.sh --clean
137+
```
138+
139+
## Production Considerations
140+
141+
1. **Use `--token` for authentication** in production environments
142+
2. **Set daily budget limits** per provider in `modelmesh.yaml`
143+
3. **Use cloud secret stores** (AWS Secrets Manager, etc.) instead of env vars
144+
4. **Monitor** via the `/health` endpoint for uptime checks
145+
5. **Reverse proxy**: Place behind nginx/caddy for TLS and rate limiting
146+
6. **Logging**: Use `--log-level DEBUG` during setup, `INFO` in production

0 commit comments

Comments
 (0)