Skip to content

Commit 93d6eac

Browse files
Feat/updating documentation for Gnani (#6004)
Co-authored-by: Tina Nguyen <72938484+tinalenguyen@users.noreply.github.com>
1 parent f6a3dd2 commit 93d6eac

8 files changed

Lines changed: 449 additions & 127 deletions

File tree

livekit-plugins/livekit-plugins-gnani/README.md

Lines changed: 103 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
[![PyPI](https://img.shields.io/pypi/v/livekit-plugins-gnani)](https://pypi.org/project/livekit-plugins-gnani/)
44
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
55

6-
[LiveKit Agents](https://github.com/livekit/agents) plugin for **[Gnani Vachana](https://gnani.ai/)** — high-accuracy Speech-to-Text and low-latency Text-to-Speech for Indian languages.
6+
[LiveKit Agents](https://github.com/livekit/agents) plugin for **[Gnani](https://gnani.ai/)** — high-accuracy Speech-to-Text (Prisma) and low-latency Text-to-Speech (Timbre) for Indian languages.
77

8-
> **Vachana** is a production-ready speech AI platform by [Gnani.ai](https://gnani.ai) supporting 10+ Indian languages with real-time streaming, multilingual transcription, and code-switching capabilities.
8+
>[Gnani.ai](https://gnani.ai) featuring **Prisma** (STT) and **Timbre** (TTS) models, supporting 10+ Indian languages with real-time streaming, multilingual transcription, and code-switching capabilities.
99
1010
## Installation
1111

@@ -19,80 +19,147 @@ This will also install the [`websockets`](https://pypi.org/project/websockets/)
1919

2020
You need a Gnani API key. Email **[speechstack@gnani.ai](mailto:speechstack@gnani.ai)** to get started — all new accounts receive free credits, no credit card required.
2121

22-
Set your credentials as environment variables:
22+
### Authentication
23+
24+
All APIs require a single API key — no `organization_id` or `user_id` needed.
25+
26+
**Option 1 — Environment variable (recommended):**
2327

2428
```bash
2529
export GNANI_API_KEY="your-api-key"
30+
```
31+
32+
**Option 2 — Constructor argument:**
2633

27-
# For REST STT only (optional):
28-
export GNANI_ORGANIZATION_ID="your-org-id"
29-
export GNANI_USER_ID="your-user-id"
34+
```python
35+
stt = STT(api_key="your-api-key", language="hi-IN")
36+
tts = TTS(api_key="your-api-key")
3037
```
3138

39+
> **Migration note:** If upgrading from an earlier version, remove any `organization_id` and `user_id` parameters — they are no longer accepted.
40+
3241
## Quick Start
3342

34-
### Speech-to-Text
43+
### Speech-to-Text (REST + Streaming)
3544

3645
```python
3746
from livekit.plugins.gnani import STT
3847

3948
stt = STT(language="hi-IN")
4049

41-
# Use with a LiveKit voice agent pipeline
50+
# REST STT (file-based transcription)
51+
speech_event = await stt.recognize(audio_buffer)
52+
53+
# Streaming STT (real-time WebSocket)
54+
speech_stream = stt.stream()
4255
```
4356

4457
### Text-to-Speech
4558

4659
```python
4760
from livekit.plugins.gnani import TTS
4861

49-
# REST (default) single-request batch synthesis
50-
tts = TTS(voice="Karan")
62+
# REST (default) - single-request batch synthesis
63+
tts_rest = TTS(voice="Karan")
5164

52-
# SSE — streaming via Server-Sent Events (lower latency)
53-
tts = TTS(voice="Karan", synthesize_method="sse")
65+
# SSE - chunked synthesis via Server-Sent Events (lower latency)
66+
tts_sse = TTS(voice="Karan", synthesize_method="sse")
5467

55-
# WebSocket — real-time streaming via stream() (lowest latency)
56-
tts = TTS(voice="Karan", synthesize_method="websocket")
68+
# WebSocket - chunked synthesis over WS (lowest latency)
69+
tts_ws = TTS(voice="Karan", synthesize_method="websocket")
5770
```
5871

5972
All three modes work with the standard LiveKit voice agent pipeline.
6073
The `synthesize_method` controls which transport `synthesize()` uses
6174
(REST, SSE, or WebSocket). The `stream()` method always uses WebSocket
6275
regardless of this setting.
6376

77+
## Full Constructor Reference
78+
79+
### STT — All parameters
80+
81+
```python
82+
from livekit.plugins.gnani import STT
83+
84+
stt = STT(
85+
language="en-IN", # Default: "en-IN"
86+
sample_rate=16000, # Default: 16000 (also: 8000)
87+
format="verbatim", # Default: "verbatim" (also: "transcribe")
88+
preferred_language=None, # Default: None
89+
itn_native_numerals=False, # Default: False
90+
api_key=None, # Default: None (reads GNANI_API_KEY env var)
91+
base_url="https://api.vachana.ai", # Default
92+
)
93+
```
94+
95+
### TTS — All parameters
96+
97+
```python
98+
from livekit.plugins.gnani import TTS
99+
100+
tts = TTS(
101+
voice="Karan", # Default: "Karan" (also: Simran, Nara, Riya, Viraj, Raju)
102+
model="vachana-voice-v3", # Default: "vachana-voice-v3"
103+
sample_rate=16000, # Default: 16000 (also: 8000, 22050, 44100)
104+
encoding="linear_pcm", # Default: "linear_pcm" (also: "oggopus")
105+
container="wav", # Default: "wav" (also: "raw", "mp3", "mulaw", "ogg")
106+
num_channels=1, # Default: 1
107+
bitrate=None, # Default: None (also: "96k", "128k", "192k")
108+
synthesize_method="rest", # Default: "rest" (also: "sse", "websocket")
109+
api_key=None, # Default: None (reads GNANI_API_KEY env var)
110+
base_url="https://api.vachana.ai", # Default
111+
)
112+
```
113+
64114
## Features
65115

66-
### STT
116+
### STT (Prisma)
67117

68-
- **Batch recognition** — REST API (`POST /stt/v3`) for file-based transcription
69-
- **Real-time streaming** — WebSocket API for live audio transcription with VAD
70-
- **10 Indian languages**bn-IN, en-IN, gu-IN, hi-IN, kn-IN, ml-IN, mr-IN, pa-IN, ta-IN, te-IN
71-
- **Code-switching**Hinglish (en-hi-IN-latn) and Hindi-English mixed (en-hi-in-cm) for streaming
118+
- **REST recognition** — REST API (`POST /stt/v3`) for file-based transcription
119+
- **Real-time streaming** — WebSocket API (`wss://api.vachana.ai/stt/v3/stream`) for live audio transcription with VAD
120+
- **10+ Indian languages**see [supported language codes](https://docs.gnani.ai/api/STT/stt-websocket#supported-languages)
121+
- **Code-switching**supports multilingual and code-mixed audio
72122
- **Sample rates** — 8 kHz and 16 kHz
123+
- **ITN support** — Inverse Text Normalization via `format="transcribe"`
124+
125+
#### Streaming PCM Specification
126+
127+
All streaming audio must be sent as **raw PCM binary frames** — no container format (WAV, MP3) mid-stream.
73128

74-
### TTS
129+
| Property | 16 kHz | 8 kHz |
130+
|-------------------|-------------------------------------------|-------------------------------------------|
131+
| Encoding | PCM signed 16-bit little-endian | PCM signed 16-bit little-endian |
132+
| Sample Rate | 16,000 Hz | 8,000 Hz |
133+
| Channels | 1 (mono) | 1 (mono) |
134+
| Samples per chunk | 512 | 512 |
135+
| **Bytes per frame** | **1,024 bytes** (512 samples × 2 bytes) | **1,024 bytes** (512 samples × 2 bytes) |
136+
| Frame duration | 32 ms | 64 ms |
137+
138+
Frames must be sent at **real-time cadence**. See **[STT Realtime — PCM Specification](https://docs.gnani.ai/api/STT/stt-websocket#pcm-specification)** for full details.
139+
140+
### TTS (Timbre)
75141

76142
- **REST synthesis** — single-request batch audio generation (`synthesize_method="rest"`)
77143
- **SSE streaming** — lower-latency chunked synthesis via Server-Sent Events (`synthesize_method="sse"`)
78144
- **WebSocket synthesis** — lowest-latency synthesis via `synthesize_method="websocket"` or the `stream()` method
79145
- **6 voices** — Karan, Simran, Nara, Riya, Viraj, Raju
80146
- **Configurable output** — sample rate (8000–44100), encoding (linear_pcm, oggopus), container (raw, mp3, wav, mulaw, ogg)
147+
- **Runtime updates** — change voice or model via `update_options()`
81148

82149
## Supported Languages
83150

84-
| Language | Code |
85-
|-----------------|---------|
86-
| Bengali | `bn-IN` |
87-
| English (India) | `en-IN` |
88-
| Gujarati | `gu-IN` |
89-
| Hindi | `hi-IN` |
90-
| Kannada | `kn-IN` |
91-
| Malayalam | `ml-IN` |
92-
| Marathi | `mr-IN` |
93-
| Punjabi | `pa-IN` |
94-
| Tamil | `ta-IN` |
95-
| Telugu | `te-IN` |
151+
### STT Languages (Prisma)
152+
153+
Prisma uses BCP-47 locale codes (e.g. `hi-IN`). Supported:
154+
155+
- **[STT REST — Supported Languages](https://docs.gnani.ai/api/STT/speech-to-text#supported-languages)**
156+
- **[STT Realtime — Supported Languages](https://docs.gnani.ai/api/STT/stt-websocket#supported-languages)**
157+
158+
---
159+
160+
### TTS Languages (Timbre)
161+
162+
For the full list of supported languages, see **[TTS — Supported Languages](https://docs.gnani.ai/api/TTS/tts-inference#supported-languages)**.
96163

97164
## Available Voices
98165

@@ -107,12 +174,14 @@ regardless of this setting.
107174

108175
## Architecture
109176

110-
This plugin directly implements the Gnani Vachana REST and WebSocket APIs using `aiohttp` (for batch STT/TTS) and `websockets` (for streaming STT/TTS), adapting them into LiveKit's `stt.STT` and `tts.TTS` base classes. No external SDK is required — all connection logic, authentication, and audio format handling is self-contained.
177+
This plugin directly implements the Gnani REST and WebSocket APIs using `aiohttp` (for REST STT/TTS) and `websockets` (for streaming STT/TTS), adapting them into LiveKit's `stt.STT` and `tts.TTS` base classes. It uses the **Prisma** model for speech-to-text and the **Timbre** model for text-to-speech. No external SDK is required — all connection logic, authentication, and audio format handling is self-contained. Authentication uses a single `api_key` passed via the `X-API-Key-ID` header.
111178

112179
## Documentation
113180

114-
- [Vachana API Docs](https://docs.inya.ai/vachana/introduction/introduction)
181+
- [Gnani API Docs](https://docs.gnani.ai/)
115182
- [LiveKit Agents Docs](https://docs.livekit.io/agents/)
183+
- [Gnani STT Plugin Guide](https://docs.livekit.io/agents/integrations/stt/gnani/)
184+
- [Gnani TTS Plugin Guide](https://docs.livekit.io/agents/integrations/tts/gnani/)
116185

117186
## License
118187

livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.py

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,32 @@
1+
# Copyright 2025 LiveKit, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
115
"""Gnani Vachana plugin for LiveKit Agents
216
317
Support for speech-to-text and text-to-speech with [Gnani's Vachana platform](https://gnani.ai/).
418
519
Vachana provides high-accuracy STT and low-latency TTS for Indian languages,
620
including multilingual and code-switching scenarios.
721
8-
For API access, email speechstack@gnani.ai
22+
See https://docs.livekit.io/agents/integrations/stt/gnani/ for more information.
923
"""
1024

11-
from .stt import STT
12-
from .tts import TTS
25+
from .stt import STT, SpeechStream
26+
from .tts import TTS, SynthesizeStream
1327
from .version import __version__
1428

15-
__all__ = ["STT", "TTS", "__version__"]
29+
__all__ = ["STT", "SpeechStream", "TTS", "SynthesizeStream", "__version__"]
1630

1731

1832
from livekit.agents import Plugin
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,17 @@
1+
# Copyright 2025 LiveKit, Inc.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
115
import logging
216

317
logger = logging.getLogger(__name__)

0 commit comments

Comments
 (0)