feat(rag): chatkit

Author: Rayder-23

.claude/skills/backend-debugging-dependency-resolution/SKILL.md

@@ -0,0 +1,131 @@
+---
+name: backend-debugging-dependency-resolution
+description: Systematic approach to diagnose and resolve backend service issues including dependency conflicts, database schema mismatches, import errors, and middleware configuration problems in FastAPI applications.
+---
+
+# Backend Debugging and Dependency Resolution Skill
+
+## Purpose
+
+This skill provides a systematic methodology for diagnosing and resolving backend service issues in FastAPI applications, with particular focus on dependency conflicts, database schema mismatches, import errors, and middleware configuration problems.
+
+## When to Use This Skill
+
+This skill should be used when:
+- FastAPI applications fail to start or respond with internal server errors
+- Dependency version conflicts arise between FastAPI, Starlette, and related packages
+- Database foreign key constraint violations occur
+- Import-related errors prevent module loading
+- Middleware configuration causes request processing failures
+- Authentication or session management systems malfunction
+- RAG agent or vector store integration fails
+
+## Implementation Process
+
+### Phase 1: Dependency Conflict Diagnosis
+
+1. **Check version compatibility matrix**
+   - Identify FastAPI version (e.g., 0.104.1)
+   - Identify Starlette version and verify compatibility range
+   - Identify anyio and other dependency versions
+   - Consult official compatibility documentation
+
+2. **Resolve dependency conflicts**
+   - Install compatible versions (e.g., Starlette <0.28.0 for FastAPI 0.104.1)
+   - Pin dependency versions in requirements.txt
+   - Test with minimal dependency set first
+
+3. **Verify resolution**
+   - Test basic FastAPI functionality with simple endpoint
+   - Confirm middleware stack processes requests without errors
+   - Validate dependency compatibility with test client
+
+### Phase 2: Database Schema Consistency
+
+1. **Identify foreign key mismatches**
+   - Check error messages for "ForeignKeyViolation" or "constraint violation"
+   - Compare model definitions with actual database schema
+   - Identify which table references which in foreign key relationships
+
+2. **Fix relationship inconsistencies**
+   - Update ForeignKey references to point to correct tables
+   - Ensure relationship definitions match foreign key constraints
+   - Update related model relationships accordingly
+
+3. **Apply schema changes**
+   - Drop and recreate tables if safe to do so (development)
+   - Use Alembic migrations for production environments
+   - Verify new relationships work correctly
+
+### Phase 3: Import and Module Structure Resolution
+
+1. **Diagnose import errors**
+   - Identify missing module errors (e.g., "No module named 'embeddings'")
+   - Trace import chains to find incorrect paths
+   - Check for circular imports or namespace pollution
+
+2. **Fix import paths**
+   - Update import statements to reflect actual module structure
+   - Use explicit imports instead of wildcard imports
+   - Ensure all `__init__.py` files properly export required modules
+
+3. **Validate module structure**
+   - Test individual module imports in isolation
+   - Verify `__init__.py` files contain proper exports
+   - Confirm package structure matches import expectations
+
+### Phase 4: Middleware and Configuration Fixes
+
+1. **Diagnose middleware errors**
+   - Look for "too many values to unpack" errors in middleware context
+   - Check middleware stack construction and configuration
+   - Verify middleware compatibility with FastAPI version
+
+2. **Resolve configuration issues**
+   - Remove conflicting middleware configurations
+   - Ensure middleware is added in correct order
+   - Validate middleware parameters and options
+
+### Phase 5: System Integration Verification
+
+1. **Test core components**
+   - Verify RAG agent functionality with sample queries
+   - Test authentication endpoints (registration, login, profile)
+   - Confirm database connectivity and session management
+   - Validate vector store integration
+
+2. **Validate end-to-end flows**
+   - Test complete user journey from authentication to content retrieval
+   - Verify session persistence across requests
+   - Confirm error handling and fallback mechanisms
+
+## Validation Checklist
+
+Before considering the debugging complete, verify:
+
+- [ ] All dependency version conflicts resolved
+- [ ] Database schema consistency achieved
+- [ ] Import errors eliminated
+- [ ] Middleware configuration fixed
+- [ ] Core functionality tested (RAG, auth, sessions)
+- [ ] Error handling working properly
+- [ ] End-to-end flows functional
+- [ ] No regressions introduced
+
+## Critical Indicators
+
+- **Dependency conflicts**: Look for "ValueError: too many values to unpack" in middleware context
+- **Schema mismatches**: Check for "ForeignKeyViolation" or "constraint violation" errors
+- **Import issues**: Watch for "No module named 'X'" errors
+- **Middleware problems**: Monitor for errors in `build_middleware_stack` calls
+
+## Expected Outcomes
+
+Upon successful application of this skill:
+- Backend services start without errors
+- API endpoints respond with appropriate status codes
+- Database operations complete successfully
+- Authentication and session management work properly
+- RAG agent processes queries and returns responses
+- Frontend integration functions correctly
+- Error handling provides meaningful feedback

backend/__init__.py

@@ -0,0 +1,3 @@
+"""
+Backend package initialization for Physical AI Book Platform
+"""

backend/agents/__init__.py

@@ -0,0 +1,3 @@
+from .agent import rag_agent, QueryMode
+
+__all__ = ["rag_agent", "QueryMode"]

backend/agent.py backend/agents/agent.pybackend/agent.py renamed to backend/agents/agent.py

@@ -7,7 +7,7 @@
 from enum import Enum
 
 # Import the vector store
-from vector_store import vector_store
+from data.vector_store import vector_store
 
 # Load environment variables
 load_dotenv()
@@ -67,7 +67,7 @@ def __call__(self, query: str, mode: str, selected_text: Optional[str] = None) -
         Returns:
             Dictionary containing retrieved content and metadata
         """
-        from embeddings import embeddings_service
+        from data.embeddings import embeddings_service
 
         try:
             # Generate embedding for the query
@@ -274,14 +274,15 @@ def _format_retrieved_content(self, retrieved_items: List[Dict]) -> str:
 
         return full_content
 
-    def _call_agent(self, user_query: str, mode: QueryMode, selected_text: Optional[str] = None) -> AgentResponse:
+    def _call_agent(self, user_query: str, mode: QueryMode, selected_text: Optional[str] = None, conversation_history: Optional[List[Dict[str, str]]] = None) -> AgentResponse:
         """
         Call the agent to process a user query
 
         Args:
             user_query: The user's question
             mode: Query mode (full-book or selected-text)
             selected_text: Text selected by user (for selected-text mode)
+            conversation_history: List of previous messages in the conversation for context
 
         Returns:
             AgentResponse with content, citations, and token usage
@@ -315,8 +316,27 @@ def _call_agent(self, user_query: str, mode: QueryMode, selected_text: Optional[
                     token_usage={}
                 )
 
-            # Create the full prompt
-            full_prompt = f"""
+            # Create the full prompt with conversation history if available
+            if conversation_history:
+                # Format conversation history
+                history_text = "Previous conversation:\n"
+                for msg in conversation_history[-5:]:  # Use last 5 messages to avoid token overflow
+                    sender = msg.get("sender_type", "user")
+                    content = msg.get("content", "")
+                    history_text += f"{sender.capitalize()}: {content}\n"
+                history_text += "\n"
+
+                full_prompt = f"""
+{history_text}
+Question: {user_query}
+
+Context from Physical AI book:
+{formatted_context}
+
+Based on the context above and the previous conversation, please answer the user's question. Remember to cite specific sections, chapters, or pages when possible, and only provide information that is grounded in the book content.
+"""
+            else:
+                full_prompt = f"""
 Question: {user_query}
 
 Context from Physical AI book:
@@ -382,21 +402,22 @@ def _call_agent(self, user_query: str, mode: QueryMode, selected_text: Optional[
                 token_usage={}
             )
 
-    def process_query(self, user_query: str, mode: QueryMode = QueryMode.FULL_BOOK, selected_text: Optional[str] = None) -> AgentResponse:
+    def process_query(self, user_query: str, mode: QueryMode = QueryMode.FULL_BOOK, selected_text: Optional[str] = None, conversation_history: Optional[List[Dict[str, str]]] = None) -> AgentResponse:
         """
         Process a user query through the RAG agent
 
         Args:
             user_query: The user's question
             mode: Query mode (full-book or selected-text)
             selected_text: Text selected by user (for selected-text mode)
+            conversation_history: List of previous messages in the conversation for context
 
         Returns:
             AgentResponse with content, citations, and token usage
         """
         logger.info(f"Processing query in {mode.value} mode: {user_query[:50]}...")
 
-        return self._call_agent(user_query, mode, selected_text)
+        return self._call_agent(user_query, mode, selected_text, conversation_history)
 
 # Create a global instance
 rag_agent = RAGAgent(vector_store)

backend/auth/__init__.py

@@ -0,0 +1,4 @@
+from .models import User, UserProfile
+from .services import auth_service
+
+__all__ = ["User", "UserProfile", "auth_service"]

backend/src/auth/models.py backend/auth/models.pybackend/src/auth/models.py renamed to backend/auth/models.py

@@ -9,7 +9,7 @@
 from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
 from sqlalchemy.orm import Session
 from typing import Optional
-from database import get_db
+from database.database import get_db
 from .models import User
 from .services import auth_service
 from pydantic import BaseModel

backend/src/auth/routes.py backend/auth/routes.pybackend/src/auth/routes.py renamed to backend/auth/routes.py

@@ -13,7 +13,7 @@
 import bcrypt
 import jwt
 from .models import User, UserProfile
-from database import get_db
+from database.database import get_db
 
 
 class AuthService:

backend/src/auth/services.py backend/auth/services.pybackend/src/auth/services.py renamed to backend/auth/services.py

@@ -0,0 +1,3 @@
+from .adapter import ChatkitAdapter
+
+__all__ = ["ChatkitAdapter"]