TicketRouterEnv: Customer Support Optimization

A production-quality OpenEnv RL environment for optimizing customer support ticket routing and resolution.

---

🎯 Problem Statement

Customer support teams waste millions annually on inefficient operations:

• 30% of tickets routed to wrong department
• Manual prioritization leads to inconsistencies
• No learning from past resolutions
• CSAT scores suffer

This RL environment enables agents to learn optimal routing, prioritization, and response strategies.

---

✨ Key Features

• ✅ Real-World Task: Not a game, but actual customer support workflow
• ✅ Dense Rewards: Every decision gets immediate feedback
• ✅ 3 Progressive Tasks: Easy → Medium → Hard
• ✅ Deterministic Grading: Reproducible evaluation
• ✅ OpenEnv Compliant: Full specification included
• ✅ FREE LLM Integration: Uses Groq (no cost!)
• ✅ Production Ready: Docker + Hugging Face Spaces support

---

📊 Environment Overview

Observation Space

{
  "ticket_id": str,                    # Unique ID
  "category": str,                     # billing|technical|account|general
  "sentiment": str,                    # negative|neutral|positive
  "message_length": int,               # 50-500 words
  "customer_tenure_days": int,         # 1-3650 days
  "previous_tickets": int,             # 0-100
  "has_attachments": bool,             # File attachments?
  "urgency_flag": bool,                # Customer marked urgent?
  "estimated_complexity": float        # 0.0-1.0 complexity
}

Action Space

{
  "target_tier": str,                  # tier_1|tier_2|specialist|escalation
  "priority": str,                     # low|medium|high|critical
  "suggested_template": str,           # Optional response template
  "estimated_resolution_time_minutes": int  # 5-240 minutes
}

Reward Function

Total Reward = 0.25*routing + 0.15*priority + 0.25*efficiency + 0.30*satisfaction + 0.05*escalation

Range: [-1.0, 1.0]

---

🎮 Tasks

Task	Difficulty	Episodes	Episodes	Success Criteria
Basic Tier Routing	Easy	50	20	Routing accuracy ≥ 70%
Balanced Optimization	Medium	100	50	Avg reward ≥ 0.65
Full System Opt	Hard	150	100	Avg reward ≥ 0.75

---

🚀 Quick Start

Installation

# Clone repository
git clone https://github.com/YOUR_USERNAME/ticket-router-env.git
cd ticket-router-env

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Setup

1. Get Groq API Key (FREE):

   • Go to: https://console.groq.com/keys
   • Create API key
   • Copy key (starts with gsk_)

2. Create .env file:

   GROQ_API_KEY=gsk_your-key-here
   

3. Run:

   python main.py

---

📈 Expected Results

============================================================
TicketRouterEnv Baseline Agent (GROQ - FREE & FAST)
============================================================
✓ Groq API initialized (FREE)

Task: Basic Tier Routing (easy)
  Episode 10/50: 0.718
  Episode 20/50: 0.752
  ...
[RUN 1] Final Score: 0.798

============================================================
RESULTS
============================================================
EASY     | Score: 0.798 | ✓ PASS
MEDIUM   | Score: 0.672 | ✓ PASS
HARD     | Score: 0.763 | ✓ PASS

API Calls: 234
Total Tokens: 18456
Estimated Cost: $0.00 🎉 FREE!

---

📁 Project Structure

ticket-router-env/
├── .env.example           # Example environment variables
├── .gitignore
├── README.md              # This file
├── requirements.txt       # Python dependencies
├── Dockerfile             # Docker containerization
├── openenv.yaml           # OpenEnv specification
├── main.py                # Entry point
├── app/
│   ├── __init__.py
│   ├── env.py             # Main environment
│   ├── models.py          # Pydantic models
│   ├── tasks.py           # Task definitions
│   ├── grader.py          # Grading logic
│   └── run_agent.py       # Baseline agent (Groq)
└── docs/
    ├── ARCHITECTURE.md
    └── API_REFERENCE.md

---

🔧 Usage

Basic Usage

from app.env import TicketRouterEnv, RoutingDecision

# Create environment
env = TicketRouterEnv()
state = env.reset()

# Make decision
action = RoutingDecision(
    target_tier="tier_2",
    priority="high",
    estimated_resolution_time_minutes=45
)

# Step
next_state, reward, done = env.step(action)
print(f"Total Reward: {reward.total_reward:.3f}")

Training Custom Agent

from app.tasks import TicketRouterTasks
from app.grader import TicketRouterGrader

env = TicketRouterEnv()
task = TicketRouterTasks.EASY
grader = TicketRouterGrader(task)

for episode in range(task.num_episodes):
    state = env.reset()
    for step in range(task.max_steps):
        action = my_agent.select_action(state)
        state, reward, done = env.step(action)
        my_agent.learn(reward)
        if done:
            break

---

🐳 Docker Deployment

Local Testing

docker build -t ticket-router-env .
docker run --env GROQ_API_KEY=gsk_... ticket-router-env

Hugging Face Spaces

1. Create space at: https://huggingface.co/new-space
2. Select Docker SDK
3. Clone space and push files
4. Add GROQ_API_KEY as secret
5. Access at: https://huggingface.co/spaces/USERNAME/ticket-router-env

---

📊 Baseline Results

Model: Mixtral-8x7b-32768 (Groq) Cost: $0.00 (FREE!)

Task	Score	Status
Easy	0.798	✓ PASS
Medium	0.672	✓ PASS
Hard	0.763	✓ PASS

---

🧠 How It Works

1. Environment generates random tickets
2. Agent observes ticket and system state
3. Agent makes routing/priority decision
4. Environment simulates resolution
5. Reward is computed based on quality
6. Grader evaluates performance

---

💡 Real-World Applications

• 📞 Call Center Optimization: Route calls to best agents
• 📧 Email Triage: Prioritize incoming support tickets
• 💬 Live Chat: Queue management and agent assignment
• 🤖 Chatbot Integration: When to escalate to human
• 📊 Workforce Planning: Predict staffing needs

---

🔮 Future Improvements

• Multi-agent simulation (concurrent tickets)
• Dialogue management for response generation
• Long-horizon planning (ticket tracking)
• Customer satisfaction modeling
• Real company data integration
• Online learning capabilities
• Performance benchmarking suite

---

📜 License

MIT License - See LICENSE for details

---

🙏 Acknowledgments

• Built with OpenEnv specification
• LLM integration via Groq
• Pydantic for data validation

---

📞 Support

• Issues: Open GitHub issue
• Discussions: GitHub Discussions
• Email: anandvaidya46@gmail.com

---

⭐ If you find this useful, please star the repo!