Skip to content

README.md

Latest commit

 

History

History
304 lines (224 loc) · 7.51 KB

README.md

File metadata and controls

304 lines (224 loc) · 7.51 KB
Raw

TicketRouterEnv: Customer Support Optimization

A production-quality OpenEnv RL environment for optimizing customer support ticket routing and resolution.

License: MIT Python 3.8+ OpenEnv

🎯 Problem Statement

Customer support teams waste millions annually on inefficient operations:

  • 30% of tickets routed to wrong department
  • Manual prioritization leads to inconsistencies
  • No learning from past resolutions
  • CSAT scores suffer

This RL environment enables agents to learn optimal routing, prioritization, and response strategies.

✨ Key Features

  • ✅ Real-World Task: Not a game, but actual customer support workflow
  • ✅ Dense Rewards: Every decision gets immediate feedback
  • ✅ 3 Progressive Tasks: Easy → Medium → Hard
  • ✅ Deterministic Grading: Reproducible evaluation
  • ✅ OpenEnv Compliant: Full specification included
  • ✅ FREE LLM Integration: Uses Groq (no cost!)
  • ✅ Production Ready: Docker + Hugging Face Spaces support

📊 Environment Overview

Observation Space

{
  "ticket_id": str,                    # Unique ID
  "category": str,                     # billing|technical|account|general
  "sentiment": str,                    # negative|neutral|positive
  "message_length": int,               # 50-500 words
  "customer_tenure_days": int,         # 1-3650 days
  "previous_tickets": int,             # 0-100
  "has_attachments": bool,             # File attachments?
  "urgency_flag": bool,                # Customer marked urgent?
  "estimated_complexity": float        # 0.0-1.0 complexity
}

Action Space

{
  "target_tier": str,                  # tier_1|tier_2|specialist|escalation
  "priority": str,                     # low|medium|high|critical
  "suggested_template": str,           # Optional response template
  "estimated_resolution_time_minutes": int  # 5-240 minutes
}

Reward Function

Total Reward = 0.25*routing + 0.15*priority + 0.25*efficiency + 0.30*satisfaction + 0.05*escalation

Range: [-1.0, 1.0]

🎮 Tasks

Task Difficulty Episodes Episodes Success Criteria
Basic Tier Routing Easy 50 20 Routing accuracy ≥ 70%
Balanced Optimization Medium 100 50 Avg reward ≥ 0.65
Full System Opt Hard 150 100 Avg reward ≥ 0.75

🚀 Quick Start

Installation

# Clone repository
git clone https://github.com/YOUR_USERNAME/ticket-router-env.git
cd ticket-router-env

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Setup

  1. Get Groq API Key (FREE):

  2. Create .env file:

    GROQ_API_KEY=gsk_your-key-here

  3. Run:

    python main.py

📈 Expected Results

============================================================
TicketRouterEnv Baseline Agent (GROQ - FREE & FAST)
============================================================
✓ Groq API initialized (FREE)

Task: Basic Tier Routing (easy)
  Episode 10/50: 0.718
  Episode 20/50: 0.752
  ...
[RUN 1] Final Score: 0.798

============================================================
RESULTS
============================================================
EASY     | Score: 0.798 | ✓ PASS
MEDIUM   | Score: 0.672 | ✓ PASS
HARD     | Score: 0.763 | ✓ PASS

API Calls: 234
Total Tokens: 18456
Estimated Cost: $0.00 🎉 FREE!

📁 Project Structure

ticket-router-env/
├── .env.example           # Example environment variables
├── .gitignore
├── README.md              # This file
├── requirements.txt       # Python dependencies
├── Dockerfile             # Docker containerization
├── openenv.yaml           # OpenEnv specification
├── main.py                # Entry point
├── app/
│   ├── __init__.py
│   ├── env.py             # Main environment
│   ├── models.py          # Pydantic models
│   ├── tasks.py           # Task definitions
│   ├── grader.py          # Grading logic
│   └── run_agent.py       # Baseline agent (Groq)
└── docs/
    ├── ARCHITECTURE.md
    └── API_REFERENCE.md

🔧 Usage

Basic Usage

from app.env import TicketRouterEnv, RoutingDecision

# Create environment
env = TicketRouterEnv()
state = env.reset()

# Make decision
action = RoutingDecision(
    target_tier="tier_2",
    priority="high",
    estimated_resolution_time_minutes=45
)

# Step
next_state, reward, done = env.step(action)
print(f"Total Reward: {reward.total_reward:.3f}")

Training Custom Agent

from app.tasks import TicketRouterTasks
from app.grader import TicketRouterGrader

env = TicketRouterEnv()
task = TicketRouterTasks.EASY
grader = TicketRouterGrader(task)

for episode in range(task.num_episodes):
    state = env.reset()
    for step in range(task.max_steps):
        action = my_agent.select_action(state)
        state, reward, done = env.step(action)
        my_agent.learn(reward)
        if done:
            break

🐳 Docker Deployment

Local Testing

docker build -t ticket-router-env .
docker run --env GROQ_API_KEY=gsk_... ticket-router-env

Hugging Face Spaces

  1. Create space at: https://huggingface.co/new-space
  2. Select Docker SDK
  3. Clone space and push files
  4. Add GROQ_API_KEY as secret
  5. Access at: https://huggingface.co/spaces/USERNAME/ticket-router-env

📊 Baseline Results

Model: Mixtral-8x7b-32768 (Groq) Cost: $0.00 (FREE!)

Task Score Status
Easy 0.798 ✓ PASS
Medium 0.672 ✓ PASS
Hard 0.763 ✓ PASS

🧠 How It Works

  1. Environment generates random tickets
  2. Agent observes ticket and system state
  3. Agent makes routing/priority decision
  4. Environment simulates resolution
  5. Reward is computed based on quality
  6. Grader evaluates performance

💡 Real-World Applications

  • 📞 Call Center Optimization: Route calls to best agents
  • 📧 Email Triage: Prioritize incoming support tickets
  • 💬 Live Chat: Queue management and agent assignment
  • 🤖 Chatbot Integration: When to escalate to human
  • 📊 Workforce Planning: Predict staffing needs

🔮 Future Improvements

  • Multi-agent simulation (concurrent tickets)
  • Dialogue management for response generation
  • Long-horizon planning (ticket tracking)
  • Customer satisfaction modeling
  • Real company data integration
  • Online learning capabilities
  • Performance benchmarking suite

📜 License

MIT License - See LICENSE for details

🙏 Acknowledgments

  • Built with OpenEnv specification
  • LLM integration via Groq
  • Pydantic for data validation

📞 Support

⭐ If you find this useful, please star the repo!