Speech Interaction Backend

Voice assistant backend powered by LiveKit Agents with OpenAI Realtime voice and built‑in noise cancellation. Conversation events are logged to Supabase for later analysis.

Key Features

• Realtime voice assistant using an OpenAI voice model
• Built‑in noise cancellation (BVC)
• Structured conversation logging to Supabase
• Dockerized and ready for LiveKit Cloud deployment

Architecture Overview

• LiveKit hosts the realtime audio room and runs this Agent.
• Agent (this repo) connects to LiveKit, handles audio and conversation logic, calls OpenAI, and writes logs.
• OpenAI provides the realtime voice model and reasoning.
• Supabase stores structured conversation logs in a Postgres table.

Project Layout

• src/agent.py – LiveKit Agent entrypoint and Supabase logging
• Dockerfile – Container image definition for LiveKit Cloud
• livekit.toml – LiveKit CLI/Cloud configuration (subdomain, agent ID)
• requirements.txt – Python dependencies

Requirements

• Python 3.11+ (for local development)
• LiveKit CLI (for deploys): https://docs.livekit.io/cloud/agents/deploy/
• Supabase project with a table to receive logs
• OpenAI API key (recommended) for the OpenAI plugin

Environment Variables

Set these either in a local .env file or in your deployment environment:

Variable	Description
SUPABASE_URL	Supabase project URL
SUPABASE_KEY	Service role or anon key with INSERT permission into the log table
OPENAI_API_KEY	API key used by livekit-plugins-openai
LIVEKIT_URL	LiveKit server URL for local runs (not required on LiveKit Cloud)
LIVEKIT_API_KEY	LiveKit API key for local runs (LiveKit Cloud injects this automatically in deployments)
LIVEKIT_API_SECRET	LiveKit API secret for local runs (LiveKit Cloud injects this automatically in deployments)

Important: Supabase is initialized lazily at runtime. During the Docker build we run python src/agent.py download-files to prefetch assets. Missing SUPABASE_* variables will not break the image build; logging is simply skipped until credentials are provided at runtime.

Conversation Logging

Conversation events are inserted into the Supabase table test-speech-interaction with the following columns:

• room_number (text) – Room identifier from LiveKit
• modi (text) – Experimental condition / mode label
• participant_id (text) – Participant identifier
• role (text) – Speaker role (e.g. user, assistant)
• message (text) – Message content (transcript)
• interrupted (boolean) – Whether the message was interrupted
• ts (timestamptz) – Event timestamp

Example SQL to create the table in Supabase:

create table if not exists public."test-speech-interaction" (
  room_number   text,
  modi          text,
  participant_id text,
  role          text,
  message       text,
  interrupted   boolean,
  ts            timestamptz default now()
);

Local Development

1. (Optional but recommended) Create and activate a virtual environment:

   python -m venv .venv
   .venv\Scripts\activate  # Windows PowerShell

2. Create a .env file in the repo root:

   SUPABASE_URL=...
   SUPABASE_KEY=...
   OPENAI_API_KEY=...
   # For local runs (if not using Cloud):
   LIVEKIT_URL=...
   LIVEKIT_API_KEY=...
   LIVEKIT_API_SECRET=...

3. Install dependencies and start the Agent:

   pip install -r requirements.txt
   python src/agent.py start

4. (Optional) Pre‑download plugin assets (same step used in the Docker build):

   python src/agent.py download-files

Once the Agent is running and connected to a LiveKit room, join the room with a LiveKit client (e.g. a web or mobile app) to talk to the assistant. Logged conversation events should appear in your Supabase table.

Deploy to LiveKit Cloud

1. Install and log in to the LiveKit CLI (see the LiveKit docs linked above).

2. Ensure livekit.toml is configured with your subdomain and Agent settings.

3. Deploy the Agent:

   lk agent deploy

This builds the Docker image (using the provided Dockerfile), uploads it, and deploys your Agent. After code changes, run lk agent deploy again to push a new version.