Skip to content

Commit fe61d5e

Browse files
plutolessplutoless
authored
Merge pull request #60 from seymourtang/feat/add-new-tts
feat: add GradiumTTS and MistralTTS vendors
2 parents 03f9b5d + 17ce521 commit fe61d5e

13 files changed

Lines changed: 233 additions & 3 deletions

File tree

‎docs/concepts/vendors.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,8 @@ Used with `agent.with_tts()`. Each TTS vendor produces audio at a specific sampl
6868
| `MiniMaxTTS` | MiniMax | `model` for supported Agora-managed global models; `key`, `group_id`, `model`, `voice_id`, `url` for BYOK | — |
6969
| `GenericTTS` | Generic OpenAI-compatible TTS over HTTP(S) | `url` | Configurable |
7070
| `DeepgramTTS` | Deepgram | `api_key`, `model` | Configurable |
71+
| `GradiumTTS` | Gradium | `api_key` | Configurable |
72+
| `MistralTTS` | Mistral | `api_key` | — |
7173
| `SarvamTTS` | Sarvam | `api_key` | — |
7274
| `XaiTTS` | xAI | `api_key`, `language` | Configurable |
7375

‎docs/guides/regional-routing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ If you omit `with_stt()`, AgentKit uses `FengmingSTT` by default for `Area.CN` c
4040

4141
| Client area | STT classes | LLM classes | TTS classes | Avatar classes |
4242
|---|---|---|---|---|
43-
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
43+
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
4444
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |
4545

4646
Global client example:

‎docs/reference/vendors.md‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Construct vendors directly from `agora_agent`, then bind a client with `Agent(cl
1919

2020
| Area | STT classes | LLM classes | TTS classes | Avatar classes |
2121
|---|---|---|---|---|
22-
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
22+
| `Area.US`, `Area.EU`, `Area.AP` | `DeepgramSTT`, `SpeechmaticsSTT`, `MicrosoftSTT`, `OpenAISTT`, `GoogleSTT`, `AmazonSTT`, `AssemblyAISTT`, `AresSTT`, `SarvamSTT`, `XaiSTT` | `OpenAI`, `AzureOpenAI`, `Anthropic`, `Gemini`, `Groq`, `VertexAILLM`, `AmazonBedrock`, `Dify`, `CustomLLM` | `ElevenLabsTTS`, `MicrosoftTTS`, `OpenAITTS`, `CartesiaTTS`, `GoogleTTS`, `AmazonTTS`, `DeepgramTTS`, `GradiumTTS`, `MistralTTS`, `HumeAITTS`, `RimeTTS`, `FishAudioTTS`, `MiniMaxTTS`, `MurfTTS`, `SarvamTTS`, `GenericTTS`, `XaiTTS` | `LiveAvatarAvatar`, `HeyGenAvatar`, `AkoolAvatar`, `AnamAvatar`, `GenericAvatar` |
2323
| `Area.CN` | `FengmingSTT`, `TencentSTT`, `MicrosoftCNSTT`, `XfyunSTT`, `XfyunBigModelSTT`, `XfyunDialectSTT` | `AliyunLLM`, `BytedanceLLM`, `DeepSeekLLM`, `TencentLLM` | `MiniMaxCNTTS`, `TencentTTS`, `BytedanceTTS`, `MicrosoftCNTTS`, `CosyVoiceTTS`, `BytedanceDuplexTTS`, `StepFunTTS`, `GenericTTS` | `SenseTimeAvatar`, `SpatiusAvatar` |
2424

2525
Global example:
@@ -303,6 +303,18 @@ The SDK also includes named helpers for the remaining Agora-supported LLM provid
303303
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Deepgram TTS parameters, flattened into `params` |
304304
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |
305305

306+
### `GradiumTTS`
307+
308+
| Parameter | Type | Required | Default | Description |
309+
|---|---|---|---|---|
310+
| `api_key` | `str` | Yes | — | Gradium API key |
311+
| `url` | `str` | No | `None` | WebSocket endpoint for streaming TTS output |
312+
| `model_name` | `str` | No | `None` | Gradium TTS model name (e.g., `default`) |
313+
| `voice_id` | `str` | No | `None` | Gradium voice identifier |
314+
| `sample_rate` | `int` | No | `None` | Audio sample rate in Hz |
315+
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Gradium TTS parameters, flattened into `params` |
316+
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |
317+
306318
### `HumeAITTS`
307319

308320
| Parameter | Type | Required | Default | Description |
@@ -389,6 +401,16 @@ AgentKit serializes `credential_mode` at the top level of the Rime TTS configura
389401

390402
`key` and `group_id` are required together for BYOK. `url` is optional. In both BYOK and managed modes, exactly one of `voice_id` or `timber_weights` must be provided. Without `key`, `model` must be one of the supported Agora-managed MiniMax models.
391403

404+
### `MistralTTS`
405+
406+
| Parameter | Type | Required | Default | Description |
407+
|---|---|---|---|---|
408+
| `api_key` | `str` | Yes | — | Mistral API key |
409+
| `model` | `str` | No | `None` | Mistral TTS model name (e.g., `voxtral-mini-tts-2603`) |
410+
| `voice` | `str` | No | `None` | Mistral voice identifier |
411+
| `additional_params` | `Dict[str, Any]` | No | `None` | Additional Mistral TTS parameters, flattened into `params` |
412+
| `skip_patterns` | `List[int]` | No | `None` | Skip patterns |
413+
392414
### `MurfTTS`
393415

394416
| Parameter | Type | Required | Default | Description |

‎src/agora_agent/agentkit/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,7 @@
175175
GenericAvatar,
176176
GoogleSTT,
177177
GoogleTTS,
178+
GradiumTTS,
178179
Groq,
179180
HeyGenAvatar,
180181
HumeAITTS,
@@ -185,6 +186,7 @@
185186
MicrosoftSTT,
186187
MicrosoftTTS,
187188
MiniMaxTTS,
189+
MistralTTS,
188190
MurfTTS,
189191
OpenAI,
190192
OpenAIRealtime,
@@ -393,12 +395,14 @@
393395
"CartesiaTTS",
394396
"DeepgramTTS",
395397
"GoogleTTS",
398+
"GradiumTTS",
396399
"AmazonTTS",
397400
"HumeAITTS",
398401
"RimeTTS",
399402
"FishAudioTTS",
400403
"MiniMaxTTS",
401404
"MiniMaxCNTTS",
405+
"MistralTTS",
402406
"TencentTTS",
403407
"BytedanceTTS",
404408
"CosyVoiceTTS",

‎src/agora_agent/agentkit/regional_agent.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,9 +54,11 @@
5454
ElevenLabsTTS,
5555
FishAudioTTS,
5656
GoogleTTS,
57+
GradiumTTS,
5758
HumeAITTS,
5859
MicrosoftTTS,
5960
MiniMaxTTS,
61+
MistralTTS,
6062
MurfTTS,
6163
OpenAITTS,
6264
GenericTTS,
@@ -99,6 +101,8 @@
99101
GenericTTS,
100102
XaiTTS,
101103
DeepgramTTS,
104+
GradiumTTS,
105+
MistralTTS,
102106
]
103107
GlobalLLM = typing.Union[
104108
OpenAI,

‎src/agora_agent/agentkit/vendors/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,9 +54,11 @@
5454
ElevenLabsTTS,
5555
FishAudioTTS,
5656
GoogleTTS,
57+
GradiumTTS,
5758
HumeAITTS,
5859
MicrosoftTTS,
5960
MiniMaxTTS,
61+
MistralTTS,
6062
MurfTTS,
6163
OpenAITTS,
6264
GenericTTS,
@@ -92,12 +94,14 @@
9294
"CartesiaTTS",
9395
"DeepgramTTS",
9496
"GoogleTTS",
97+
"GradiumTTS",
9598
"AmazonTTS",
9699
"HumeAITTS",
97100
"RimeTTS",
98101
"FishAudioTTS",
99102
"MiniMaxTTS",
100103
"MiniMaxCNTTS",
104+
"MistralTTS",
101105
"MurfTTS",
102106
"SarvamTTS",
103107
"GenericTTS",

‎src/agora_agent/agentkit/vendors/catalog.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,11 @@
3535
ElevenLabsTTS,
3636
FishAudioTTS,
3737
GoogleTTS,
38+
GradiumTTS,
3839
HumeAITTS,
3940
MicrosoftTTS,
4041
MiniMaxTTS,
42+
MistralTTS,
4143
MurfTTS,
4244
OpenAITTS,
4345
GenericTTS,
@@ -102,6 +104,8 @@ def __init__(
102104
"generic": GenericTTS,
103105
"xai": XaiTTS,
104106
"deepgram": DeepgramTTS,
107+
"gradium": GradiumTTS,
108+
"mistral": MistralTTS,
105109
},
106110
avatar={
107111
"akool": AkoolAvatar,

‎src/agora_agent/agentkit/vendors/namespaces.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,9 +32,11 @@
3232
ElevenLabsTTS,
3333
FishAudioTTS,
3434
GoogleTTS,
35+
GradiumTTS,
3536
HumeAITTS,
3637
MicrosoftTTS,
3738
MiniMaxTTS,
39+
MistralTTS,
3840
MurfTTS,
3941
OpenAITTS,
4042
GenericTTS,
@@ -101,6 +103,8 @@ class GlobalTTSVendors:
101103
generic = GenericTTS
102104
xai = XaiTTS
103105
deepgram = DeepgramTTS
106+
gradium = GradiumTTS
107+
mistral = MistralTTS
104108

105109

106110
class CNTTSVendors:

‎src/agora_agent/agentkit/vendors/region.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,8 @@
5656
"generic",
5757
"xai",
5858
"deepgram",
59+
"gradium",
60+
"mistral",
5961
)
6062

6163
CN_LLM_VENDORS: typing.Tuple[str, ...] = (

‎src/agora_agent/agentkit/vendors/tts.py‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,35 @@ def to_config(self) -> Dict[str, Any]:
257257
return result
258258

259259

260+
class GradiumTTS(BaseTTS):
261+
model_config = ConfigDict(extra="forbid")
262+
263+
api_key: str = Field(..., description="Gradium API key")
264+
url: Optional[str] = Field(default=None, description="WebSocket endpoint for streaming TTS output")
265+
model_name: Optional[str] = Field(default=None, description="Gradium TTS model name")
266+
voice_id: Optional[str] = Field(default=None, description="Gradium voice identifier")
267+
sample_rate: Optional[int] = Field(default=None, description="Audio sample rate in Hz")
268+
additional_params: Optional[Dict[str, Any]] = Field(default=None, description="Additional Gradium TTS parameters")
269+
skip_patterns: Optional[List[int]] = Field(default=None)
270+
271+
def to_config(self) -> Dict[str, Any]:
272+
params: Dict[str, Any] = dict(self.additional_params or {})
273+
params["api_key"] = self.api_key
274+
if self.url is not None:
275+
params["url"] = self.url
276+
if self.model_name is not None:
277+
params["model_name"] = self.model_name
278+
if self.voice_id is not None:
279+
params["voice_id"] = self.voice_id
280+
if self.sample_rate is not None:
281+
params["sample_rate"] = self.sample_rate
282+
283+
result: Dict[str, Any] = {"vendor": "gradium", "params": params}
284+
if self.skip_patterns is not None:
285+
result["skip_patterns"] = self.skip_patterns
286+
return result
287+
288+
260289
class HumeAITTS(BaseTTS):
261290
model_config = ConfigDict(extra="forbid")
262291

@@ -455,6 +484,29 @@ def to_config(self) -> Dict[str, Any]:
455484
return result
456485

457486

487+
class MistralTTS(BaseTTS):
488+
model_config = ConfigDict(extra="forbid")
489+
490+
api_key: str = Field(..., description="Mistral API key")
491+
model: Optional[str] = Field(default=None, description="Mistral TTS model name")
492+
voice: Optional[str] = Field(default=None, description="Mistral voice identifier")
493+
additional_params: Optional[Dict[str, Any]] = Field(default=None, description="Additional Mistral TTS parameters")
494+
skip_patterns: Optional[List[int]] = Field(default=None)
495+
496+
def to_config(self) -> Dict[str, Any]:
497+
params: Dict[str, Any] = dict(self.additional_params or {})
498+
params["api_key"] = self.api_key
499+
if self.model is not None:
500+
params["model"] = self.model
501+
if self.voice is not None:
502+
params["voice"] = self.voice
503+
504+
result: Dict[str, Any] = {"vendor": "mistral", "params": params}
505+
if self.skip_patterns is not None:
506+
result["skip_patterns"] = self.skip_patterns
507+
return result
508+
509+
458510
class SarvamTTS(BaseTTS):
459511
model_config = ConfigDict(extra="forbid")
460512

0 commit comments

Comments
 (0)