Skip to content

Commit 16b00e4

Browse files
codeSTACKrcodeSTACKr
authored
Add Anam healthcare intake assistant demo and update README (#70)
- Introduced a new Anam healthcare intake assistant demo, featuring a voice AI agent named Liv that guides patients through a medical intake form using an Anam lip-synced avatar. - Added detailed documentation for the demo, including installation instructions, prerequisites, and usage guidelines. - Updated the main README to include the new Anam demo and revised the description of the LemonSlice negotiation coach for clarity.
1 parent c9b3c87 commit 16b00e4

123 files changed

Lines changed: 22723 additions & 1 deletion

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,8 @@ Bring your agent to life with animated avatars.
160160
| [Dynamic Avatar](complex-agents/avatars/hedra/dynamically_created_avatar/) | Create avatars on-the-fly | Intermediate |
161161
| [Education Avatar](complex-agents/avatars/hedra/education_avatar/) | Teaching avatar with flash cards via RPC | Advanced |
162162
| [Tavus Avatar](complex-agents/avatars/tavus/) | Tavus-powered avatar assistant | Intermediate |
163-
| [Negotiation Avatar](complex-agents/avatars/lemonslice/) | LemonSlice-powered Salary Negotiation Coach | Advanced |
163+
| [LemonSlice Negotiation Coach](complex-agents/avatars/lemonslice/) | Salary negotiation practice with boss personalities and real-time coaching | Advanced |
164+
| [Anam Healthcare Intake](complex-agents/avatars/anam/) | Voice intake form with Anam lip-synced avatar and real-time RPC updates | Intermediate |
164165

165166
### Translation & Multilingual
166167

‎complex-agents/avatars/anam/README.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# Anam healthcare intake assistant
2+
3+
A LiveKit Agents demo that guides patients through a medical intake form using voice and an Anam lip-synced avatar. The agent (Liv) collects information one field at a time and updates the form in real time via RPC to the frontend. The demo includes both a **Python** and a **TypeScript** backend; run whichever you prefer. The frontend works with either.
4+
5+
## Features
6+
7+
- **Voice intake flow**: Liv asks for each form field in order (name, DOB, address, phone, emergency contact, medications, allergies, reason for visit)
8+
- **One question at a time**: Confirms each answer before moving to the next field
9+
- **Real-time form updates**: Agent updates the on-screen form as the user speaks via RPC
10+
- **Anam avatar**: Lip-synced video avatar powered by the Anam plugin
11+
- **Voice pipeline**: Deepgram STT, OpenAI LLM, and ElevenLabs TTS (via LiveKit Inference)
12+
- **Form submission**: Final confirmation loop, then submit from the frontend
13+
14+
## Prerequisites
15+
16+
- **For the Python backend**: Python 3.10+, uv
17+
- **For the TypeScript backend**: Node.js, pnpm
18+
- **For the frontend**: Node.js, pnpm
19+
- LiveKit account
20+
- Anam API key
21+
22+
## Installation
23+
24+
1. Clone this repository and go to the anam demo
25+
```bash
26+
git clone <repository-url>
27+
cd complex-agents/avatars/anam
28+
```
29+
30+
2. Install the backend you want to use (or both).
31+
- **Python backend**
32+
```bash
33+
cd agent-py
34+
uv sync
35+
cd ..
36+
```
37+
- **TypeScript backend**
38+
```bash
39+
cd agent-ts
40+
pnpm install
41+
cd ..
42+
```
43+
44+
3. Install frontend dependencies
45+
```bash
46+
cd frontend
47+
pnpm install
48+
cd ..
49+
```
50+
51+
4. Set up environment for your chosen agent
52+
- **Python**: In `agent-py`, copy `.env.example` to `.env.local` and add your LiveKit and Anam credentials.
53+
- **TypeScript**: In `agent-ts`, copy `.env.example` to `.env.local` and add your LiveKit and Anam credentials.
54+
55+
5. Set up environment for the frontend
56+
In `frontend`, copy `.env.example` to `.env.local`. Add the same LiveKit credentials and set `AGENT_NAME=Anam-Demo` so the frontend dispatches to the agent.
57+
58+
## Configuration
59+
60+
Enter your environment variables in the `.env.local` file:
61+
62+
```bash
63+
LIVEKIT_URL=wss://your-project.livekit.cloud
64+
LIVEKIT_API_KEY=your_livekit_api_key
65+
LIVEKIT_API_SECRET=your_livekit_api_secret
66+
ANAM_API_KEY=your_anam_api_key
67+
```
68+
69+
Get your Anam API key from the [Anam dashboard](https://lab.anam.ai/api-keys). You can get your LiveKit environment variables from the [LiveKit dashboard](https://cloud.livekit.io/) or load them from the [LiveKit CLI](https://docs.livekit.io/home/cli/cli-setup):
70+
71+
```bash
72+
lk cloud auth
73+
lk app env -w -d .env.local
74+
```
75+
76+
Run that from either `agent-py` or `agent-ts` depending on which backend you use.
77+
78+
### Frontend (`frontend/.env.local`)
79+
80+
```bash
81+
LIVEKIT_API_KEY=your_livekit_api_key
82+
LIVEKIT_API_SECRET=your_livekit_api_secret
83+
LIVEKIT_URL=wss://your-project.livekit.cloud
84+
AGENT_NAME=Anam-Demo
85+
```
86+
87+
Use the same LiveKit values as the agent. `AGENT_NAME=Anam-Demo` matches both backends so the frontend connects to whichever agent you run.
88+
89+
## Usage
90+
91+
Run **one** backend (Python or TypeScript) and the frontend. Do not run both backends at the same time.
92+
93+
### Python backend
94+
95+
1. One-time: download Silero VAD and the turn detector model
96+
```bash
97+
cd agent-py
98+
uv run python src/agent.py download-files
99+
```
100+
101+
2. Start the agent
102+
```bash
103+
uv run python src/agent.py dev
104+
```
105+
106+
The agent registers with your LiveKit project as `Anam-Demo`.
107+
108+
### TypeScript backend
109+
110+
1. One-time: download Silero VAD and the turn detector model
111+
```bash
112+
cd agent-ts
113+
pnpm run download-files
114+
```
115+
116+
2. Start the agent
117+
```bash
118+
pnpm run dev
119+
```
120+
121+
The agent registers with your LiveKit project as `Anam-Demo`.
122+
123+
### Frontend
124+
125+
1. Open a new terminal and go to the frontend:
126+
```bash
127+
cd frontend
128+
```
129+
130+
2. Start the dev server:
131+
```bash
132+
pnpm dev
133+
```
134+
135+
3. In your browser, open:
136+
```
137+
http://localhost:3000
138+
```
139+
140+
4. Start a session and complete the intake form with Liv.
141+
142+
## Project structure
143+
144+
```
145+
anam/
146+
├── agent-py/ # Python backend (optional)
147+
│ ├── src/agent.py # Intake agent and Anam avatar session
148+
│ ├── pyproject.toml
149+
│ └── .env.local
150+
├── agent-ts/ # TypeScript backend (optional)
151+
│ ├── src/ # Agent entry and intake logic
152+
│ ├── package.json
153+
│ └── .env.local
154+
└── frontend/
155+
├── app/ # Next.js app router
156+
├── components/app/ # Session, avatar, intake form UI
157+
└── .env.local # LiveKit credentials + AGENT_NAME
158+
```
159+
160+
Both agents use the same agent name (`Anam-Demo`) and the same function tools (`update_field`, `get_form_state`, `submit_form`) to call RPCs on the frontend so the form stays in sync with the conversation.
161+
162+
## Built with
163+
164+
- [LiveKit Agents](https://docs.livekit.io/agents/) - Agent framework
165+
- [Anam plugin](https://docs.livekit.io/agents/models/avatar/plugins/anam/) - Lip-synced avatar
166+
- [LiveKit Inference](https://docs.livekit.io/agents/models/) - STT, LLM, TTS models

‎complex-agents/avatars/anam/agent-py/.dockerignore‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Python bytecode and artifacts
2+
__pycache__/
3+
*.py[cod]
4+
*.pyo
5+
*.pyd
6+
*.egg-info/
7+
dist/
8+
build/
9+
10+
# Virtual environments
11+
.venv/
12+
venv/
13+
14+
# Caches and test output
15+
.cache/
16+
.pytest_cache/
17+
.ruff_cache/
18+
coverage/
19+
20+
# Logs and temp files
21+
*.log
22+
*.gz
23+
*.tgz
24+
.tmp
25+
.cache
26+
27+
# Environment variables
28+
.env
29+
.env.*
30+
31+
# VCS, editor, OS
32+
.git
33+
.gitignore
34+
.gitattributes
35+
.github/
36+
.idea/
37+
.vscode/
38+
.DS_Store
39+
40+
# Project docs and misc
41+
README.md
42+
LICENSE
43+
44+
# Project tests
45+
test/
46+
tests/
47+
eval/
48+
evals/

‎complex-agents/avatars/anam/agent-py/.env.example‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
LIVEKIT_URL=
2+
LIVEKIT_API_KEY=
3+
LIVEKIT_API_SECRET=
4+
ANAM_API_KEY=

‎complex-agents/avatars/anam/agent-py/.github/assets/livekit-mark.png‎

832 Bytes
Loading

‎complex-agents/avatars/anam/agent-py/.github/workflows/ruff.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: Ruff
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
ruff-check:
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v4
15+
16+
- name: Install uv
17+
uses: astral-sh/setup-uv@v1
18+
with:
19+
version: "latest"
20+
21+
- name: Set up Python
22+
uses: actions/setup-python@v4
23+
with:
24+
python-version: "3.12"
25+
26+
- name: Install dependencies
27+
run: UV_GIT_LFS=1 uv sync --dev
28+
29+
- name: Run ruff linter
30+
run: uv run ruff check --output-format=github .
31+
32+
- name: Run ruff formatter
33+
run: uv run ruff format --check --diff .

‎complex-agents/avatars/anam/agent-py/.github/workflows/template-check.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# As this is a starter template project, we don't want to check in the uv.lock and livekit.toml files in its template form
2+
# However, once you have cloned this repo for your own use, LiveKit recommends you check them in and delete this github workflow entirely
3+
4+
name: Template Check
5+
6+
on:
7+
push:
8+
branches: [ main ]
9+
pull_request:
10+
branches: [ main ]
11+
12+
jobs:
13+
check-template-files:
14+
runs-on: ubuntu-latest
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- name: Check template files not tracked in git
20+
run: |
21+
if git ls-files | grep -q "^uv\.lock$"; then
22+
echo "Error: uv.lock should not be checked into git"
23+
echo "Disable this test and commit the file once you have cloned this repo for your own use"
24+
exit 1
25+
fi
26+
if git ls-files | grep -q "^livekit\.toml$"; then
27+
echo "Error: livekit.toml should not be checked into git"
28+
echo "Disable this test and commit the file once you have cloned this repo for your own use"
29+
exit 1
30+
fi
31+
echo "✓ uv.lock and livekit.toml are correctly not tracked in git"

‎complex-agents/avatars/anam/agent-py/.github/workflows/tests.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v4
15+
16+
- name: Install uv
17+
uses: astral-sh/setup-uv@v1
18+
with:
19+
version: "latest"
20+
21+
- name: Set up Python
22+
uses: actions/setup-python@v4
23+
with:
24+
python-version: "3.12"
25+
26+
- name: Install dependencies
27+
run: UV_GIT_LFS=1 uv sync --dev
28+
29+
- name: Run tests
30+
env:
31+
LIVEKIT_URL: ${{ secrets.LIVEKIT_URL }}
32+
LIVEKIT_API_KEY: ${{ secrets.LIVEKIT_API_KEY }}
33+
LIVEKIT_API_SECRET: ${{ secrets.LIVEKIT_API_SECRET }}
34+
run: uv run pytest -v

‎complex-agents/avatars/anam/agent-py/.gitignore‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
.env
2+
.env.*
3+
!.env.example
4+
.DS_Store
5+
__pycache__
6+
.idea
7+
KMS
8+
.venv
9+
.vscode
10+
*.egg-info
11+
.pytest_cache
12+
.ruff_cache

0 commit comments

Comments
 (0)