ParleyLab

ParleyLab is an AI-powered negotiation training simulator. Users practice structured negotiation scenarios — salary offers, freelance contracts, apartment leases, and equity splits — against an AI opponent whose strategy is driven by a trained PPO reinforcement learning agent and whose dialogue is generated by an LLM. After each move, an independent critic agent evaluates the user's technique in real time against established negotiation theory.

The core architectural decision is the decoupling of strategy from language: the PPO policy decides what the opponent does (hold firm, concede, bluff, walk away), and the LLM independently translates that decision into natural language. This keeps strategic behavior principled and consistent while keeping dialogue natural and persona-specific.

Features

• Four built-in negotiation scenarios (salary, freelance contract, rent, equity split), each with distinct opponent personas and hidden objectives
• RL-driven opponent: a PPO agent controls strategic decisions independently of language generation
• Per-turn critic feedback grounded in negotiation theory, returned as structured JSON: strengths, weaknesses, an actionable suggestion, and a concept tag (anchoring, concession pacing, BATNA signaling, etc.)
• Hidden state enforcement: opponent target, BATNA, and persona are held server-side and never sent to the client under any normal response
• End-of-session reveal: opponent hidden context is exposed only after the session ends, alongside a performance score that blends outcome quality with move quality
• Voice input via browser-native Web Speech API — transcription happens client-side with no server dependency
• Multi-currency support: scenario values are randomized per session and converted using approximate exchange rates at session start
• Pluggable LLM backends: Gemini (default), Groq, and Ollama supported via a unified router with template-based fallbacks when the LLM is unavailable
• Opponent and critic LLM calls run in parallel (asyncio.gather), reducing per-turn latency by 3–5 seconds compared to sequential execution

Technology Stack

Layer	Tool
RL algorithm	Stable-Baselines3 PPO
RL environment	Custom Gymnasium environment
Neural network backend	PyTorch (CPU; no GPU required)
Backend	FastAPI + Uvicorn
Session storage	In-memory Python dict (single-user scope)
LLM — default	Gemini 2.0 Flash (free tier)
LLM — alternative	Groq (Llama 3.1 8B, free tier)
LLM — local	Ollama (Llama 3.1 8B / Qwen 2.5 7B)
Frontend	Next.js 14 (App Router), React 18
UI styling	Tailwind CSS
State management	Zustand
Voice input	Web Speech API (browser-native)
Icons	lucide-react

System Architecture

See docs/SystemArchitecture.txt for the full component breakdown, API contract, state model, failure modes, and deployment topology.

Implementation Details

Per-turn pipeline

Each negotiation move runs through six stages, all server-side within a single HTTP call:

1. User input — text typed or voice transcribed in-browser via Web Speech API
2. Move parsing — LLM call (low temperature, JSON schema) converts free-form text to {primary_move, offered_value, signals, tone}
3. State update — 7-dimensional normalized observation vector is rebuilt from offer history, concession rates, and turn count
4. Strategic decision (RL) — the observation is passed to the loaded PPO policy; one of five discrete actions is returned in under 1 ms on CPU
5. Parallel LLM calls — opponent agent and critic agent run concurrently; the opponent translates the RL action to natural language, the critic independently evaluates the user's move without access to the opponent's hidden context
6. Response — only user-safe fields are returned; opponent hidden state is never included in normal responses

Hidden state enforcement

Three layers prevent the opponent's private information from reaching the client:

• Storage: hidden fields (opponent_target, opponent_batna, opponent_persona, opponent_urgency) are stored only in the FastAPI in-memory session dict
• Serialization: SessionState.to_user_dict() explicitly enumerates allowed fields; the reveal serializer (to_reveal_dict()) is only called after the session is marked complete
• Prompt boundaries: the critic agent prompt is constructed without access to hidden context, preventing indirect leakage through coaching text

Scenario system

Scenarios are JSON files in scenarios/. Each specifies user role, opponent role, both parties' targets and BATNAs, opening offer, opponent persona, urgency level, and value unit. At session start, monetary amounts are randomized by ±15% and converted to the user's chosen currency. Adding a new scenario requires only a new JSON file — no code changes.

LLM provider routing

All three LLM roles (move parser, opponent agent, critic agent) share a single LLMRouter (parley_ai/llm/router.py). Provider and model are selected via environment variable. Each agent has a template-based string fallback so sessions continue if the LLM is unreachable.

API reference

Method	Path	Description
GET	/healthz	Liveness probe; reports RL model load status and active LLM provider
GET	/api/scenario/list	List available scenarios
POST	/api/scenario/start	Create a new session
POST	/api/chat/message	Submit user move; returns opponent reply and critic feedback
POST	/api/chat/evaluate	Request critic feedback asynchronously
GET	/session/{id}/reveal	Return opponent hidden state and final score (session must be complete)

Legacy v1 routes (POST /session/start, POST /session/{id}/move) are retained for backward compatibility.

Setup and Installation

Prerequisites

• Python 3.10+
• Node.js 18+
• A Gemini API key (free tier) — https://aistudio.google.com/apikey

Clone and configure

git clone https://github.com/Karan1114Anand/ParleyLab.git
cd ParleyLab
cp .env.example .env
# Edit .env and set GEMINI_API_KEY

Backend

python -m venv .venv
source .venv/bin/activate       # Windows: .venv\Scripts\activate
pip install -r requirements.txt

cd backend
uvicorn main:app --port 8000 --reload

Wait for Orchestrator ready in the log before starting the frontend.

Frontend

Open a new terminal:

cd frontend
npm install
npm run dev

Open http://localhost:3000, select a scenario, and start negotiating.

Switching LLM providers

Edit .env to change the active provider:

# Groq — free tier, fast
PARLEYLAB_LLM_PROVIDER=groq
GROQ_API_KEY=gsk_...
PARLEYLAB_MODEL=llama-3.1-8b-instant

# Ollama — local, no API key required
PARLEYLAB_LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
PARLEYLAB_MODEL=llama3.1:8b

Running Ollama with both the opponent model (8B) and the critic model (7B) simultaneously requires approximately 16 GB RAM.

Convenience launcher (Linux)

./dev.sh

Kills any existing processes on ports 8000 and 3000, then starts the backend in the current terminal and opens the frontend and an LLM status monitor in separate terminal windows (requires gnome-terminal, xterm, konsole, or xfce4-terminal).

Reinforcement Learning

Environment

NegotiationEnv (defined in training/train.py) is a custom Gymnasium environment. There is no external dataset — scenarios are randomly generated each episode within a fixed parametric range, with a guaranteed Zone of Possible Agreement (ZOPA).

State space — 7-dimensional, all values in [0, 1]:

Index	Feature	Description
0	own_offer_norm	Opponent's current offer / 100
1	opponent_offer_norm	User's current offer / 100
2	turn_norm	Current turn / max turns
3	own_concession_rate	Opponent's total movement from opening
4	opponent_concession_rate	User's total movement from opening
5	gap_norm	Gap between offers / scale
6	turns_since_last_concession_norm	Turns since opponent last conceded

Action space — Discrete(5):

ID	Name	Behavior
0	Hold Firm	No change to offer
1	Concede Small	Move 5–12% of gap toward user (scales with difficulty)
2	Concede Large	Move 15–30% of gap toward user (scales with difficulty)
3	Bluff	Move 5% of gap away from user
4	Walk Away	Terminate session

Reward function:

Outcome	Reward
Deal closed	+10.0 + (surplus over BATNA / 100) × 100
Walk away	−15.0
Timeout	−3.0
Per-step	−0.1

Model

• Algorithm: PPO (Proximal Policy Optimization)
• Policy architecture: MLP (Stable-Baselines3 default; two hidden layers of 64 units)
• Total training timesteps: 1,000,000
• Training setup: 5 parallel environments (SubprocVecEnv) at difficulty levels 0.1, 0.3, 0.5, 0.7, 0.9; evaluation every 10,000 steps over 50 episodes
• Entropy coefficient: annealed linearly from 0.05 to 0.005 over training to shift from exploration to exploitation
• Key hyperparameters: learning_rate=3e-4, n_steps=2048, batch_size=320, n_epochs=10, gamma=0.99, gae_lambda=0.95, clip_range=0.2
• Saved artifact: models/best_model.zip (~148 KB; best checkpoint saved by EvalCallback during training)
• Inference cost: single MLP forward pass, sub-millisecond on CPU

Evaluation

The following metrics were recorded during development-time evaluation (100 episodes, deterministic policy, difficulty=0.5, rule-based counterpart opponent). Evaluation logs are not tracked in the repository; the numbers below are from the system architecture documentation.

Metric	Value
Deal close rate	86%
Timeout rate	14%
Walk-away rate	0%
Average episode reward	+10.09
Average deal value (buyer side; lower is better)	62.01
Best deal value achieved	42.75

To re-run evaluation against the saved model:

cd training
python train.py --mode eval --model_path ../models/best_model --episodes 100