|
1 | | -# Gemini Demo |
| 1 | +# Gemini Live x Fishjam Cloud Demo |
2 | 2 |
|
3 | | -A minimal example of a video call with a Gemini Live AI agent using Fishjam Cloud. |
| 3 | +A video call where you can invite a Gemini Live AI agent into the room. The agent hears everyone, sees your camera, responds with voice, and can search the web for real-time answers. Participants can customize the agent's behavior with a system prompt. |
4 | 4 |
|
5 | | -## What it does |
| 5 | +## Running locally |
6 | 6 |
|
7 | | -- Create a room and join a video call |
8 | | -- Spawn a Gemini Live voice agent with a custom system prompt |
9 | | -- The agent joins the call, listens to participants, and responds with voice |
10 | | -- Supports Google Search for real-time information |
| 7 | +1. Copy `.env.example` to `.env` and fill in your Fishjam and Gemini credentials: |
11 | 8 |
|
12 | | -## Setup |
13 | | - |
14 | | -1. Copy `.env.example` to `.env` and fill in your credentials: |
15 | | - |
16 | | -``` |
| 9 | +```bash |
17 | 10 | cp .env.example .env |
18 | 11 | ``` |
19 | 12 |
|
20 | 13 | 2. Install dependencies: |
21 | 14 |
|
22 | | -``` |
23 | | -cd backend && npm install |
24 | | -cd ../web && npm install |
| 15 | +```bash |
| 16 | +yarn install |
25 | 17 | ``` |
26 | 18 |
|
27 | | -3. Start the backend: |
| 19 | +3. Start the backend and frontend in separate terminals: |
28 | 20 |
|
29 | | -``` |
30 | | -cd backend && npm run start |
31 | | -``` |
32 | | - |
33 | | -4. Start the frontend (in another terminal): |
34 | | - |
35 | | -``` |
36 | | -cd web && npm run start |
| 21 | +```bash |
| 22 | +cd backend && yarn start |
| 23 | +cd web && yarn start |
37 | 24 | ``` |
38 | 25 |
|
39 | | -5. Open http://localhost:5173 |
40 | | - |
41 | | -## Architecture |
42 | | - |
43 | | -``` |
44 | | -backend/src/main.ts - Fastify + tRPC server, Fishjam SDK, Gemini Live API |
45 | | -web/src/App.tsx - React frontend with Fishjam React Client |
46 | | -web/src/trpc.ts - tRPC client setup |
47 | | -``` |
48 | | - |
49 | | -### Audio flow |
50 | | - |
51 | | -``` |
52 | | -Peer audio (16kHz) → Fishjam Agent → Gemini Live API |
53 | | - ↓ |
54 | | - Fishjam Agent Track ← Gemini response (24kHz) |
55 | | - ↓ |
56 | | - All peers hear the agent |
57 | | -``` |
| 26 | +4. Open http://localhost:5173 |
0 commit comments