Work on csv (#12)

* feat: Implement CSV Ticket Viewer

- Refactor App component to replace existing features with CSV Ticket Table.
- Add CSVTicketTable component for displaying tickets from CSV data source.
- Introduce API functions for fetching CSV ticket fields, tickets, and statistics.
- Create CSV data source in backend to handle loading and processing of CSV files.
- Enhance AgentChat component to display error details from API responses.
- Update styles and layout for improved user experience in ticket viewing.

Signed-off-by: Andre Bossard <anbossar@microsoft.com>

* refactor: Update import formatting and enhance status badge display in CSVTicketTable component

Signed-off-by: Andre Bossard <anbossar@microsoft.com>

* feat: Add Nivo chart visualizations for CSV tickets and enhance documentation

Signed-off-by: Andre Bossard <anbossar@microsoft.com>

---------

Signed-off-by: Andre Bossard <anbossar@microsoft.com>

Author: abossard

.github/copilot-instructions.md

@@ -1,41 +1,53 @@
 # Copilot Instructions
 
-## Architecture
+## Purpose
 
-- `backend/app.py` hosts REST (`/api/*`), MCP JSON-RPC (`/mcp`), and LangGraph agent (`/api/agents/run`) on port 5001; new capabilities should use the `@operation` decorator in `backend/operations.py` so all interfaces stay in sync.
-- Business logic lives in service modules: `tasks.py` (TaskService), `tickets.py` (ticket models + SLA calculations); keep services as the single source of truth.
-- The React side is feature-first: `frontend/src/App.jsx` switches tabs via React Router; each folder under `frontend/src/features` owns its state, calculations, and FluentUI layout.
-- All network calls go through `frontend/src/services/api.js`; localStorage helpers live in separate modules like `reminderStorage.js`.
+This repository is a **learning environment** for processing and visualizing CSV ticket data using Copilot.
 
-## Backend Patterns
+## Scope Boundaries
 
-- Define operations with `@operation` (see `op_create_task`, `op_get_task`): annotate parameters with Pydantic models or enums so MCP schemas and REST validation are generated automatically, then build thin Quart wrappers that only handle HTTP concerns.
-- Keep methods like `TaskService.create_task` and `TaskService.update_task` “deep”: they already validate, mutate `_tasks_db`, and return `Task` models, so avoid splitting them into tiny helpers or duplicating validation elsewhere.
-- Use the provided Pydantic models (`TaskCreate`, `TaskUpdate`, `TaskStats`, `TaskFilter`) for all inputs/outputs; MCP tooling and REST serializers expect `.model_dump()` objects with these exact field names (`created_at`, `completed`, etc.).
-- The SSE endpoint `time_stream` streams `{"time","date","timestamp"}` JSON strings; if you change its shape, also update `connectToTimeStream` consumers and the Dashboard expectations.
+**Do:**
 
-## Frontend Patterns
+- Work with `csv/data.csv` and related parsing logic (`backend/csv_data.py`)
+- Write/adjust small, focused Python scripts or notebooks for data exploration (pandas, matplotlib, seaborn, plotly)
+- Compute summaries, statistics, filters, aggregations on ticket data
+- Propose visualizations and queries to understand ticket patterns
+- Help interpret CSV schema and column mappings
 
-- Feature components isolate **calculations** (pure helpers like `getTaskStats`), **actions** (API calls, event handlers), and **data** (React state) in that order; preserve that structure when extending `TaskList` or `Dashboard`.
-- Stick to FluentUI v9 primitives with `makeStyles`/`tokens`; prefer adjusting the `useStyles` definitions over inline styles to maintain theming consistency.
-- Always go through `frontend/src/services/api.js` (`fetchJSON`, `createTask`, `updateTask`, etc.); components expect rejected promises to carry friendly `Error.message`, so don’t bypass these helpers.
-- Task UI and tests rely on deterministic `data-testid` attributes (`task-menu-${id}`, `filter-completed`, etc.); keep them stable or update the Playwright suite alongside UI changes.
-- Real-time widgets follow the `connectToTimeStream` contract (subscribe in `useEffect`, capture cleanup function, surface errors via component state); reuse that approach for any new SSE sources.
+**Do NOT:**
 
-## Workflows
+- Change backend architecture (`backend/app.py`, `operations.py`, decorators)
+- Modify Pydantic models (`tickets.py`, `tasks.py`) or core services
+- Alter REST/MCP endpoints, React frontend structure, or tests
+- Touch `UNIFIED_ARCHITECTURE.md`, `README.md`, or Playwright tests
 
-- One-time setup: run `./setup.sh` from the repo root to create the repo-level `.venv`, install frontend deps, and provision Playwright browsers.
-- Dev loop: `source .venv/bin/activate && cd backend && python app.py` (serves on 5001) plus `cd frontend && npm run dev`; `./start-dev.sh` or the VS Code “Full Stack: Backend + Frontend” launch config will start both for you.
-- MCP testing uses JSON-RPC POSTs to `http://localhost:5001/mcp` (`tools/list`, `tools/call`); no extra process is required because the Quart server already exposes it.
+## CSV Data Structure
 
-## Testing
+Primary source: `csv/data.csv` (BMC Remedy/ITSM export).
+Key columns:
 
-- E2E coverage lives in `tests/e2e/app.spec.js`; from the repo root run `npm run test:e2e` (`:ui` for interactive, `:report` to inspect results). Start both servers first to avoid cold-start delays.
-- The suite assumes sample tasks exist (created by `TaskService.initialize_sample_data()`), tabs are labeled via `data-testid`, and SSE data matches the backend format; keep those invariants or update the tests deliberately.
-- For deterministic assertions, prefer editing helpers like `waitForAppToLoad` or the shared selector strings instead of scattering hard-coded waits.
+- Identifiers: `Entry ID`, `Incident ID*+`, `Corporate ID`
+- Summary: `Summary*`, `Notes`, `Resolution`
+- Status/Priority: `Status*`, `Priority*`, `Urgency*`, `Impact*`
+- Assignment: `Assignee+`, `Assigned Group*+`, `Owner Group+`
+- Requester: `Full Name`, `First Name+`, `Last Name+`, `Internet E-mail`
+- Location: `City`, `Country`, `Site ID`, `Company`
+- Timestamps: `Reported Date+`, `Last Modified Date`, `Closed Date`
+- Categories: `Operational Categorization Tier 1+/2/3`, `Product Categorization Tier 1/2/3`
+
+Reference implementation: `backend/csv_data.py` (column mapping, status/priority mapping, date parsing).
+
+## Working with the Data
+
+- Load: `pd.read_csv("csv/data.csv", encoding="latin-1")`
+- Dates: parse `DD.MM.YYYY HH:MM:SS` (e.g., `22.10.2025 11:53:33`)
+- Status mapping: New, Assigned, In Progress, Pending, Resolved, Closed, Cancelled
+- Priority mapping: 1-Critical, 2-High, 3-Medium, 4-Low
+- Prefer notebooks for exploration; keep scripts small and focused
 
 ## Conventions
 
-- Follow “Grokking Simplicity”: keep actions (I/O) in services or event handlers, calculations pure (formatting, stats), and data as plain objects/React state; this is enforced heavily in docs and existing code.
-- Follow “A Philosophy of Software Design”: favor deep modules (e.g., extend `TaskService` instead of sprinkling logic across routes/components) and hide complexity behind simple interfaces.
-- When you touch the architecture (operation decorators, unified REST/MCP flow, feature structure), update `README.md` and `UNIFIED_ARCHITECTURE.md` so future agents inherit the correct mental model.
+- Follow “Grokking Simplicity”: separate data (CSV), calculations (pure), actions (I/O)
+- Keep changes minimal, targeted, and reversible
+- No broad refactors; prioritize clarity for learners
+- Document findings inline (markdown cells/comments)

.gitignore

@@ -50,3 +50,4 @@ Thumbs.db
 *.log
 logs/
 *.db
+csv/*.csv

README.md

@@ -1,5 +1,7 @@
 # Quart + Vite + React Demo Application
 
+**Task:** Visualize the CSV tickets with richer panels (status, priority, timeline, geography, SLA). **How would you like to view the tickets?**
+
 > A teaching-oriented full-stack sample that pairs a Python Quart backend with a React + FluentUI frontend, real-time Server-Sent Events (SSE), and Playwright tests.
 
 ## Why this repo?

backend/app.py

@@ -33,6 +33,9 @@
 from agents import AgentRequest, AgentResponse, agent_service
 from api_decorators import operation
 
+# CSV ticket service
+from csv_data import Ticket, get_csv_ticket_service
+
 # FastMCP client for direct ticket MCP calls (no AI)
 from fastmcp import Client as MCPClient
 from mcp_handler import handle_mcp_request
@@ -386,6 +389,180 @@ async def get_qa_tickets():
 # NON-TASK ENDPOINTS
 # ============================================================================
 
+# ============================================================================
+# CSV TICKET ENDPOINTS
+# ============================================================================
+
+# Initialize CSV ticket service
+_csv_ticket_service = get_csv_ticket_service()
+
+# Load CSV data on startup (using relative path from backend folder)
+_csv_data_path = Path(__file__).parent.parent / "csv" / "data.csv"
+if _csv_data_path.exists():
+    _csv_loaded = _csv_ticket_service.load_csv(_csv_data_path)
+    print(f"📊 Loaded {_csv_loaded} tickets from CSV")
+
+
+# Define which fields are available for display
+CSV_TICKET_FIELDS = [
+    {"name": "id", "label": "ID", "type": "uuid"},
+    {"name": "summary", "label": "Summary", "type": "string"},
+    {"name": "status", "label": "Status", "type": "enum"},
+    {"name": "priority", "label": "Priority", "type": "enum"},
+    {"name": "assignee", "label": "Assignee", "type": "string"},
+    {"name": "assigned_group", "label": "Assigned Group", "type": "string"},
+    {"name": "requester_name", "label": "Requester", "type": "string"},
+    {"name": "requester_email", "label": "Email", "type": "string"},
+    {"name": "city", "label": "City", "type": "string"},
+    {"name": "country", "label": "Country", "type": "string"},
+    {"name": "service", "label": "Service", "type": "string"},
+    {"name": "incident_type", "label": "Incident Type", "type": "string"},
+    {"name": "product_name", "label": "Product", "type": "string"},
+    {"name": "manufacturer", "label": "Manufacturer", "type": "string"},
+    {"name": "created_at", "label": "Created", "type": "datetime"},
+    {"name": "updated_at", "label": "Updated", "type": "datetime"},
+    {"name": "urgency", "label": "Urgency", "type": "string"},
+    {"name": "impact", "label": "Impact", "type": "string"},
+    {"name": "resolution", "label": "Resolution", "type": "string"},
+    {"name": "notes", "label": "Notes", "type": "string"},
+]
+
+
+@app.route("/api/csv-tickets/fields", methods=["GET"])
+async def get_csv_ticket_fields():
+    """Get metadata about available CSV ticket fields."""
+    return jsonify({
+        "fields": CSV_TICKET_FIELDS,
+        "total_tickets": _csv_ticket_service.total_count,
+    })
+
+
+@app.route("/api/csv-tickets", methods=["GET"])
+async def get_csv_tickets():
+    """
+    Get CSV tickets with optional filtering, sorting, and field selection.
+    
+    Query params:
+    - fields: comma-separated list of field names to include
+    - status: filter by status (new, assigned, in_progress, pending, resolved, closed)
+    - has_assignee: filter by assignee presence (true/false)
+    - assigned_group: filter by group name
+    - sort: field name to sort by
+    - sort_dir: asc or desc (default: asc)
+    - limit: max number of results
+    - offset: number of results to skip
+    """
+    from tickets import TicketStatus
+
+    # Parse query params
+    fields_param = request.args.get("fields", "")
+    status_param = request.args.get("status")
+    has_assignee_param = request.args.get("has_assignee")
+    assigned_group_param = request.args.get("assigned_group")
+    sort_param = request.args.get("sort", "created_at")
+    sort_dir = request.args.get("sort_dir", "desc")
+    limit = request.args.get("limit", type=int)
+    offset = request.args.get("offset", 0, type=int)
+    
+    # Determine which fields to include
+    if fields_param:
+        selected_fields = [f.strip() for f in fields_param.split(",")]
+    else:
+        # Default fields for table display
+        selected_fields = ["summary", "status", "priority", "assignee", "assigned_group", "requester_name", "city", "created_at"]
+    
+    # Parse filters
+    status_filter = None
+    if status_param:
+        try:
+            status_filter = TicketStatus(status_param)
+        except ValueError:
+            pass
+    
+    has_assignee_filter = None
+    if has_assignee_param is not None:
+        has_assignee_filter = has_assignee_param.lower() == "true"
+    
+    # Get tickets with filters
+    tickets = _csv_ticket_service.list_tickets(
+        status=status_filter,
+        assigned_group=assigned_group_param,
+        has_assignee=has_assignee_filter,
+    )
+    
+    # Sort tickets
+    def get_sort_key(ticket: Ticket):
+        val = getattr(ticket, sort_param, None)
+        if val is None:
+            return ""
+        if hasattr(val, "value"):  # Enum
+            return val.value
+        return val
+    
+    try:
+        tickets = sorted(tickets, key=get_sort_key, reverse=(sort_dir == "desc"))
+    except TypeError:
+        pass  # Skip sorting if types are incompatible
+    
+    total_count = len(tickets)
+    
+    # Apply pagination
+    if limit:
+        tickets = tickets[offset:offset + limit]
+    elif offset:
+        tickets = tickets[offset:]
+    
+    # Build response with selected fields only
+    result = []
+    for ticket in tickets:
+        row = {}
+        for field in selected_fields:
+            val = getattr(ticket, field, None)
+            if val is None:
+                row[field] = None
+            elif hasattr(val, "value"):  # Enum
+                row[field] = val.value
+            elif hasattr(val, "isoformat"):  # datetime
+                row[field] = val.isoformat()
+            elif hasattr(val, "hex"):  # UUID
+                row[field] = str(val)
+            else:
+                row[field] = val
+        result.append(row)
+    
+    return jsonify({
+        "tickets": result,
+        "total": total_count,
+        "offset": offset,
+        "limit": limit,
+        "fields": selected_fields,
+    })
+
+
+@app.route("/api/csv-tickets/stats", methods=["GET"])
+async def get_csv_ticket_stats():
+    """Get statistics about CSV tickets."""
+    from collections import Counter
+    
+    tickets = _csv_ticket_service.list_tickets()
+    
+    statuses = Counter(t.status.value for t in tickets)
+    priorities = Counter(t.priority.value for t in tickets)
+    groups = Counter(t.assigned_group for t in tickets if t.assigned_group)
+    cities = Counter(t.city for t in tickets if t.city)
+    
+    unassigned_count = sum(1 for t in tickets if t.assignee is None and t.assigned_group is not None)
+    
+    return jsonify({
+        "total": len(tickets),
+        "unassigned": unassigned_count,
+        "by_status": dict(statuses),
+        "by_priority": dict(priorities),
+        "by_group": dict(groups.most_common(10)),
+        "by_city": dict(cities.most_common(10)),
+    })
+
+
 @app.route("/api/health", methods=["GET"])
 async def health_check():
     """Health check endpoint."""