Skip to content

Commit 6b2f809

Browse files
apartsinclaude
andcommitted
Add TypeScript implementation, Azure Speech TTS connector, browser support, and updated docs
- Add complete TypeScript implementation mirroring Python: core, CDK, connectors, interfaces, proxy, client - Add Azure Speech TTS provider connector (azure.tts.v1) for both Python and TypeScript with SSML support - Add browser-compatible provider base class (BrowserBaseProvider) and browser entry point - Add secret store connectors: EncryptedFileSecretStore, MemorySecretStore, KeyringSecretStore - Add CORS proxy tool for browser development - Add browser samples (index.html + proxy-server.js) - Update all documentation: connector catalogue, coverage matrix, CDK guides, developer guide - Add new docs: BrowserUsage guide, SecretStores guide, Azure Speech TTS connector reference - Update .gitignore with Node.js exclusions (node_modules, *.tgz) - 1,084 tests passing (Python: 752, TypeScript: 332) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent a0ccee5 commit 6b2f809

209 files changed

Lines changed: 28176 additions & 3132 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,10 @@ Thumbs.db
3131
# Logs
3232
*.log
3333

34+
# Node.js
35+
node_modules/
36+
*.tgz
37+
3438
# Package
3539
*.whl
3640
*.tar.gz

README.md

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,10 @@
99

1010
<p align="center">
1111
<img src="https://img.shields.io/badge/python-3.11%2B-blue" alt="Python 3.11+">
12+
<img src="https://img.shields.io/badge/typescript-5.0%2B-blue" alt="TypeScript 5.0+">
13+
<img src="https://img.shields.io/badge/docker-supported-2496ED" alt="Docker">
1214
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="License"></a>
13-
<a href="https://github.com/ApartsinProjects/ModelMesh/actions"><img src="https://img.shields.io/badge/tests-579%20passed-brightgreen" alt="Tests"></a>
15+
<a href="https://github.com/ApartsinProjects/ModelMesh/actions"><img src="https://img.shields.io/badge/tests-808%20passed-brightgreen" alt="Tests"></a>
1416
<a href="https://apartsinprojects.github.io/ModelMesh/"><img src="https://img.shields.io/badge/docs-GitHub%20Pages-blue" alt="Documentation"></a>
1517
</p>
1618

@@ -20,10 +22,16 @@ Your application requests a **capability** (e.g. "chat completion"). ModelMesh p
2022

2123
## Install
2224

25+
**Python:**
2326
```bash
2427
pip install modelmesh-lite
2528
```
2629

30+
**TypeScript / Node.js:**
31+
```bash
32+
npm install @modelmesh/core
33+
```
34+
2735
## Quick Start
2836

2937
Set an API key and go:
@@ -49,11 +57,9 @@ print(response.choices[0].message.content)
4957
### TypeScript
5058

5159
```typescript
52-
import { ModelMesh } from "@modelmesh/core";
60+
import { create } from "@modelmesh/core";
5361

54-
const mesh = new ModelMesh();
55-
await mesh.initialize({ providers: ["openai"] });
56-
const client = mesh.getClient();
62+
const client = create("chat-completion");
5763

5864
const response = await client.chat.completions.create({
5965
model: "chat-completion",
@@ -187,13 +193,22 @@ client = modelmesh.create(config="modelmesh.yaml")
187193
## Development
188194

189195
```bash
190-
# Clone and install dev dependencies
196+
# Clone the repository
191197
git clone https://github.com/ApartsinProjects/ModelMesh.git
192198
cd ModelMesh
193199

194-
# Run tests
200+
# Run Python tests (640 tests)
195201
pip install pytest
196202
cd src/python && python -m pytest ../../tests/ -v
203+
204+
# Run TypeScript tests (168 tests)
205+
cd src/typescript && npm install && npm test
206+
```
207+
208+
## Docker
209+
210+
```bash
211+
docker run -e OPENAI_API_KEY="sk-..." ghcr.io/apartsinprojects/modelmesh
197212
```
198213

199214
## License

docs/ConnectorCatalogue.md

Lines changed: 61 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,7 @@ pools:
118118
| **Replicate** | `provider.replicate.inference.v1` | Run any open-source model via API. Pay-per-second billing. | Flux Schnell, SDXL, Llama 3, Whisper | Limited free predictions; no credit card required | [replicate.com/docs](https://replicate.com/docs) |
119119
| **ElevenLabs** | `provider.elevenlabs.tts.v1` | Leading voice AI. Realistic speech synthesis and voice cloning. | Multilingual v2, Turbo v2.5, Flash v2.5, Monolingual v1 | 10,000 chars/month (~20 min audio); 3 custom voices; non-commercial | [elevenlabs.io/docs](https://elevenlabs.io/docs) |
120120
| **AssemblyAI** | `provider.assemblyai.stt.v1` | Speech intelligence platform. Transcription with built-in NLU. | Universal, Nano | $50 credits (~185h transcription); one-time, non-recurring | [www.assemblyai.com/docs](https://www.assemblyai.com/docs) |
121+
| **Azure Speech** | `provider.azure.tts.v1` | Microsoft Azure Cognitive Services Speech. Neural TTS with 400+ voices in 140+ languages. | en-US-JennyNeural, en-US-AndrewNeural, and all Azure Neural voices | 0.5M chars/month free (Neural); 5M chars/month free (Standard) | [learn.microsoft.com/azure/ai-services/speech-service](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/rest-text-to-speech) |
121122

122123
### Aggregators and Inference Platforms
123124

@@ -184,10 +185,41 @@ Non-AI web services can be wrapped as provider connectors using the same interfa
184185
| **Stability AI** | - | yes | - | - | - | - | - | - | credits |
185186
| **ElevenLabs** | - | - | yes | - | - | - | - | - | 10k chars/mo |
186187
| **AssemblyAI** | - | - | yes | - | - | - | - | - | $50 credit |
188+
| **Azure Speech** | - | - | yes | - | - | - | - | - | 0.5M chars/mo |
187189
| **Perplexity** | yes | - | - | - | yes | yes | - | - | Pro only |
188190
| **AWS Bedrock** | yes | yes | - | yes | - | yes | yes | yes | AWS credits |
189191
| **Google Cloud APIs** | - | - | yes | - | - | - | - | - | generous |
190192

193+
### CDK Base Classes for Providers
194+
195+
| Base Class | Environment | Transport | Streaming | Use Case |
196+
| --- | --- | --- | --- | --- |
197+
| **BaseProvider** | Node.js | `http`/`https` | Node.js streams | Server-side applications, CLI tools, backend services |
198+
| **BrowserBaseProvider** | Browser, Deno, Bun, Workers | Fetch API | `ReadableStream` | Single-page apps, browser extensions, edge runtimes |
199+
200+
Both classes expose the same provider interface and the same protected hooks for subclassing. See [cdk/BaseClasses.md](cdk/BaseClasses.html#browserbAseprovider) for details and [guides/BrowserUsage.md](guides/BrowserUsage.html) for browser setup.
201+
202+
### Audio Namespace
203+
204+
ElevenLabs (`elevenlabs.tts.v1`), Azure Speech (`azure.tts.v1`), and AssemblyAI (`assemblyai.stt.v1`) are accessible through the MeshClient audio namespace, which follows the OpenAI SDK audio pattern:
205+
206+
```python
207+
# Text-to-speech
208+
audio_response = client.audio.speech.create(
209+
model="text-to-speech",
210+
input="Hello, world!",
211+
voice="alloy",
212+
)
213+
214+
# Speech-to-text
215+
transcript = client.audio.transcriptions.create(
216+
model="speech-to-text",
217+
file=audio_file,
218+
)
219+
```
220+
221+
Audio requests are internally bridged to `CompletionRequest`/`CompletionResponse` via the `AudioRequest` and `AudioResponse` types (see [ConnectorInterfaces.md -- Audio](ConnectorInterfaces.html#audio)). The same rotation, failover, and pool logic applies to audio providers.
222+
191223
---
192224

193225
## Rotation Policies
@@ -219,7 +251,9 @@ Interface: [ConnectorInterfaces.md — Secret Store](ConnectorInterfaces.html#se
219251
| **`secret-store.google.secret-manager.v1`** | Google Cloud managed secrets with IAM and audit logging | 6 active versions free; 10K access ops/month free | [cloud.google.com/secret-manager](https://cloud.google.com/secret-manager) |
220252
| **`secret-store.microsoft.key-vault.v1`** | Microsoft cloud secret, key, and certificate management | 10K operations/month free (Standard tier) | [azure.microsoft.com/en-us/products/key-vault](https://azure.microsoft.com/en-us/products/key-vault) |
221253
| **`secret-store.1password.connect.v1`** | Secrets Automation API for CI/CD and server-side use | No free API tier; requires Business or Enterprise plan | [developer.1password.com](https://developer.1password.com) |
222-
| **`secret-store.modelmesh.json-secrets.v1`** | Reads secrets from a local JSON file. Keys are top-level object keys; values are strings. | Built-in | - |
254+
| **`secret-store.modelmesh.json-secrets.v1`** | Reads secrets from a local JSON file. Keys are top-level object keys; values are strings. Supports dot-notation for nested keys. | Built-in | - |
255+
| **`secret-store.modelmesh.memory-secrets.v1`** | Holds secrets in an in-memory dictionary. Ideal for testing, scripting, and user-provided keys. Supports runtime add/remove via SecretManagement interface. | Built-in | - |
256+
| **`secret-store.modelmesh.encrypted-file.v1`** | AES-256-GCM encrypted JSON file. Secrets are decrypted at initialization using a passphrase (PBKDF2) or raw key. Supports save/load round-trips. | Built-in | - |
223257
| **`secret-store.modelmesh.keyring.v1`** | Resolves secrets from the OS keyring (macOS Keychain, Windows Credential Locker, Linux Secret Service). | Built-in | - |
224258

225259
### Connector-Specific Configuration
@@ -265,19 +299,44 @@ Interface: [ConnectorInterfaces.md — Secret Store](ConnectorInterfaces.html#se
265299

266300
| Parameter | Type | Description |
267301
| --- | --- | --- |
268-
| `path` | string | Path to the JSON secrets file. Default: `./secrets.json`. |
302+
| `file_path` | string | Path to the JSON secrets file. |
303+
| `json_path` | string | Dot-notation path to scope lookups to a nested object (e.g., `secrets.production`). |
304+
| `fail_on_missing` | boolean | Throw on missing keys. Default: `true`. |
305+
306+
**`secret-store.modelmesh.memory-secrets.v1`:**
307+
308+
| Parameter | Type | Description |
309+
| --- | --- | --- |
310+
| `secrets` | object | Dictionary of secret name/value pairs. |
311+
| `fail_on_missing` | boolean | Throw on missing keys. Default: `true`. |
312+
| `cache_enabled` | boolean | Enable TTL-based caching. Default: `true`. |
313+
314+
**`secret-store.modelmesh.encrypted-file.v1`:**
315+
316+
| Parameter | Type | Description |
317+
| --- | --- | --- |
318+
| `file_path` | string | Path to the encrypted secrets file. |
319+
| `passphrase` | string | Human-readable passphrase for PBKDF2 key derivation. |
320+
| `encryption_key` | string | Raw 32-byte key as a 64-character hex string. Overrides passphrase. |
321+
| `pbkdf2_iterations` | integer | PBKDF2 iteration count. Default: `600000`. |
322+
| `fail_on_missing` | boolean | Throw on missing keys. Default: `true`. |
269323

270324
**`secret-store.modelmesh.keyring.v1`:**
271325

272326
| Parameter | Type | Description |
273327
| --- | --- | --- |
274328
| `service_name` | string | Keyring service/application name. Default: `modelmesh`. |
329+
| `fail_on_missing` | boolean | Throw on missing keys. Default: `true`. |
275330

276331
### Deployment Patterns
277332

278333
| Environment | Recommended Store | Reason |
279334
| --- | --- | --- |
280335
| Local dev | `modelmesh.dotenv.v1` / `modelmesh.env.v1` | simple, no infra |
336+
| Unit tests | `modelmesh.memory-secrets.v1` | inject known secrets without env setup |
337+
| Scripts | `modelmesh.memory-secrets.v1` | pass keys from user input at runtime |
338+
| Shared config | `modelmesh.encrypted-file.v1` | keys encrypted at rest, safe to commit |
339+
| Desktop apps | `modelmesh.keyring.v1` | OS-native secure storage |
281340
| AWS / GCP | native secret manager | IAM integration |
282341
| Serverless | cloud secret manager | runtime injection |
283342
| Client-side | server proxy | keys never reach client |

docs/ConnectorInterfaces.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,3 +226,29 @@ Parameters shared by all discovery connectors. Individual connectors may add con
226226
| `discovery.health.timeout` | duration | Probe timeout (e.g., `10s`). |
227227
| `discovery.health.failure_threshold` | integer | Consecutive failures before deactivation. |
228228
| `discovery.health.providers` | list | Providers to probe. Default: all enabled providers. |
229+
230+
---
231+
232+
## Audio
233+
234+
Audio capabilities (text-to-speech, speech-to-text) are integrated into the provider interface through dedicated request and response types that bridge into the existing `CompletionRequest`/`CompletionResponse` pipeline. This allows audio providers (ElevenLabs TTS, AssemblyAI STT) to participate in the same rotation, failover, and pool routing as text-generation providers.
235+
236+
### AudioRequest and AudioResponse Types
237+
238+
| Type | Purpose | Key fields |
239+
| --- | --- | --- |
240+
| **AudioRequest** | Wraps a TTS or STT request for routing through the provider pipeline. | `input` (text for TTS, audio buffer for STT), `voice`, `format`, `model`, `language` |
241+
| **AudioResponse** | Wraps audio provider output. | `audio` (binary data or stream for TTS), `text` (transcript for STT), `duration`, `usage` |
242+
243+
Audio connectors bridge these types into `CompletionRequest`/`CompletionResponse` internally. The provider's `complete()` method receives a `CompletionRequest` whose `extra` field carries the audio-specific parameters; the response's `extra` field carries audio-specific output. This preserves the uniform provider interface while supporting audio-specific data.
244+
245+
### MeshClient Audio Namespace
246+
247+
The `MeshClient` exposes audio through an OpenAI SDK-compatible namespace:
248+
249+
| Method | Capability | Description |
250+
| --- | --- | --- |
251+
| `client.audio.speech.create()` | `generation.audio.text-to-speech` | Generate speech from text. Routes to TTS providers (ElevenLabs, OpenAI, Google Cloud). |
252+
| `client.audio.transcriptions.create()` | `understanding.audio.speech-to-text` | Transcribe audio to text. Routes to STT providers (AssemblyAI, OpenAI Whisper, Groq). |
253+
254+
Audio requests follow the same routing pipeline as text requests: capability resolution, pool selection, rotation policy, retry, and failover. Pools targeting `generation.audio` or `understanding.audio` collect all audio-capable models automatically.

docs/CoverageMatrix.md

Lines changed: 59 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ title: "Test Coverage Matrix"
55

66
# Test Coverage Matrix
77

8-
Correlates documented features with test coverage. Generated from the 356-test suite across 12 test files.
8+
Correlates documented features with test coverage. The project includes 640 Python tests across 14 test files and 168 TypeScript tests across 10 test files, for a total of 808 tests.
99

1010
---
1111

@@ -24,7 +24,23 @@ Correlates documented features with test coverage. Generated from the 356-test s
2424
| Observability Stack | 26 | `test_observability.py` | Covered |
2525
| Pre-shipped Connectors | 32 | `test_connectors.py` | Covered |
2626
| CDK Specialized + Mixins + Helpers | 97 | `test_specialized.py` | Covered |
27-
| **Total** | **356** | **12 files** | |
27+
| **Total** | **640** | **14 files** | |
28+
29+
### TypeScript Test Suite
30+
31+
| Component | Tests | Test File | Status |
32+
| --- | ---: | --- | --- |
33+
| Interfaces (data types, factories, enums) | 12 | `interfaces.test.ts` | Covered |
34+
| CapabilityTree | 9 | `capability-tree.test.ts` | Covered |
35+
| EventEmitter | 10 | `event-emitter.test.ts` | Covered |
36+
| StateManager | 16 | `state-manager.test.ts` | Covered |
37+
| CapabilityPool | 15 | `pool.test.ts` | Covered |
38+
| ModelMesh facade | 16 | `mesh.test.ts` | Covered |
39+
| Router | 5 | `router.test.ts` | Covered |
40+
| Pre-shipped Connectors | 47 | `connectors.test.ts` | Covered |
41+
| MeshConfig + Auto-detect | 22 | `config.test.ts` | Covered |
42+
| MeshClient (OpenAI compat) | 16 | `client.test.ts` | Covered |
43+
| **Total** | **168** | **10 files** | |
2844

2945
---
3046

@@ -159,10 +175,46 @@ Correlates documented features with test coverage. Generated from the 356-test s
159175
| CONNECTOR_REGISTRY (8 entries) | `ConnectorCatalogue.md` | `test_has_8_connectors`, `test_all_have_connector_id` | Registry completeness |
160176
| OpenAI Provider | `connectors/openai-llm.md` | `TestOpenAIProvider` (7 tests) | ID, URL, headers, models, endpoint |
161177
| Anthropic Provider | `connectors/anthropic-llm.md` | `TestAnthropicProvider` (10 tests) | ID, headers, payload, response parsing |
162-
| Env Secret Store | `connectors/modelmesh-env.md` | `TestEnvSecretStore` (5 tests) | ID, resolve, prefix, missing |
178+
| Env Secret Store | `connectors/modelmesh-env.md` | `TestEnvSecretStoreComprehensive` (7 tests) | ID, resolve, prefix, missing, interface |
179+
| Dotenv Secret Store | `ConnectorCatalogue.md` | `TestDotenvSecretStoreComprehensive` (8 tests) | Parsing, comments, quotes, multiline, env override |
180+
| JSON Secret Store | `ConnectorCatalogue.md` | `TestJsonSecretStoreComprehensive` (8 tests) | Flat, nested, dot-notation, json_path scoping |
181+
| Memory Secret Store | `guides/SecretStores.md` | `TestMemorySecretStore` (12 tests) | CRUD, interface, caching, set invalidation |
182+
| Encrypted File Store | `guides/SecretStores.md` | `TestEncryptedFileSecretStore` (12 tests) | Save/load, passphrase, hex key, round-trip, plaintext check |
183+
| Keyring Secret Store | `ConnectorCatalogue.md` | `TestKeyringSecretStoreComprehensive` (6 tests) | ID, service name, availability, fallback |
184+
| Azure Speech TTS | `ConnectorCatalogue.md` | `TestAzureSpeechProvider` (14 tests) | ID, region, URL, headers, SSML, models, XML escape |
163185
| Stick-Until-Failure | `connectors/modelmesh-stick-until-failure.md` | (via `TestBaseRotation`) | Tested through base rotation tests |
164186
| Local File Storage | `connectors/modelmesh-local-file.md` | (via `TestBaseStorage`) | Tested through base storage tests |
165187

188+
### 11. Browser Provider (`docs/guides/BrowserUsage.md`)
189+
190+
| Feature | Doc Reference | Test(s) | Notes |
191+
| --- | --- | --- | --- |
192+
| BrowserBaseProvider construction | `BrowserUsage.md` | `test_browser_provider_config` | Config defaults, proxyUrl |
193+
| Fetch-based complete() | `BrowserUsage.md` | `test_browser_provider_complete` | Fetch API transport |
194+
| Streaming via ReadableStream | `BrowserUsage.md` | `test_browser_provider_stream` | ReadableStream SSE parsing |
195+
| CORS proxy URL prefixing | `BrowserUsage.md` | `test_browser_proxy_url_prefix` | proxyUrl prepended to API URL |
196+
| createBrowser() convenience | `BrowserUsage.md` | `test_create_browser` | Browser-optimized create() |
197+
198+
### 12. Audio Interfaces (`docs/ConnectorInterfaces.md`)
199+
200+
| Feature | Doc Reference | Test(s) | Notes |
201+
| --- | --- | --- | --- |
202+
| AudioRequest type | `ConnectorInterfaces.md` | `test_audio_request_creation` | Type construction, defaults |
203+
| AudioResponse type | `ConnectorInterfaces.md` | `test_audio_response_creation` | Type construction, defaults |
204+
| client.audio.speech.create() | `ConnectorCatalogue.md` | `test_audio_speech_create` | TTS routing through pool |
205+
| client.audio.transcriptions.create() | `ConnectorCatalogue.md` | `test_audio_transcriptions_create` | STT routing through pool |
206+
207+
### 13. CDK Specialized Classes (`docs/cdk/BaseClasses.md`)
208+
209+
| Feature | Doc Reference | Test(s) | Notes |
210+
| --- | --- | --- | --- |
211+
| ThresholdRotationPolicy | `BaseClasses.md` | `TestThresholdRotation` (in `test_specialized.py`) | Threshold-based deactivation/recovery |
212+
| ConsoleObservability | `BaseClasses.md` | `TestConsoleObservability` (in `test_specialized.py`) | ANSI output, severity filtering |
213+
| KeyValueStorage | `BaseClasses.md` | `TestKeyValueStorage` (in `test_specialized.py`) | Memory and file backends |
214+
| FileSecretStore | `BaseClasses.md` | `TestFileSecretStore` (in `test_specialized.py`) | .env, JSON, TOML file loading |
215+
| HttpHealthDiscovery | `BaseClasses.md` | -- | Requires HTTP mock; coverage gap |
216+
| CDK test helpers | `Helpers.md` | `TestConnectorTestHarness`, `TestMockHttpClient` (in `test_specialized.py`) | mockCompletionRequest, mockModelSnapshot, MockHttpClient, ConnectorTestHarness |
217+
166218
---
167219

168220
## Coverage Gaps
@@ -175,6 +227,8 @@ Correlates documented features with test coverage. Generated from the 356-test s
175227
| Webhook observability connector | Low | Documented in catalogue but not implemented |
176228
| HttpHealthDiscovery | Low | Specialized class exists but no dedicated test (requires HTTP mock) |
177229
| MetricsMixin | Low | Tested indirectly via provider tests |
230+
| Browser provider end-to-end (real fetch) | Low | Unit tests mock fetch; no integration test with actual browser |
231+
| Audio streaming (TTS binary stream) | Low | Audio types tested; binary stream end-to-end not yet covered |
178232

179233
---
180234

@@ -201,3 +255,5 @@ Correlates documented features with test coverage. Generated from the 356-test s
201255
| `interfaces/SecretStore.md` | SecretStore ABC | `test_cdk.py::TestBaseSecretStore` (5 tests) | Direct |
202256
| `interfaces/Storage.md` | Storage ABC | `test_cdk.py::TestBaseStorage` (9 tests) | Direct |
203257
| `interfaces/Discovery.md` | Discovery ABC | *(no dedicated tests)* | Gap |
258+
| `guides/BrowserUsage.md` | BrowserBaseProvider, CORS proxy, createBrowser() | Browser provider tests | Direct |
259+
| `ConnectorInterfaces.md` (Audio) | AudioRequest, AudioResponse, audio namespace | Audio interface tests | Direct |

0 commit comments

Comments
 (0)