docs: update CLAUDE.md for Issue #18 agent implementation

Update project documentation to reflect completed agent features:
- Add agent module to directory structure
- Document AgentText2SQL usage and API
- Add ReAct loop flow description
- Update API endpoints list with agent routes
- Add new agent configuration options
- Add agent test commands
- Mark Issue #18 as COMPLETED

Part of Issue #18: smolagents Agent Framework

Author: Sakeeb91

.claude/CLAUDE.md

@@ -6,9 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Arctic Text2SQL Agent: A production-grade AI agent that converts natural language to SQL using Snowflake's Arctic-Text2SQL-R1-7B model with a ReAct (Reasoning + Acting) framework for multi-step reasoning and self-correction.
 
-**Current State**: Core orchestration layer (`Text2SQLEngine`) is implemented with query intent classification, SQL validation, confidence-based retry logic, and schema alignment checking. The engine is integrated with API routes.
-
-**Next Critical Piece**: Issue #18 (smolagents agent framework) for full ReAct loop implementation with self-correction capabilities.
+**Current State**: Full agent-based architecture implemented with smolagents integration. Both the core `Text2SQLEngine` and the new `AgentText2SQL` engine are available, with the agent version providing multi-step reasoning, self-correction, and query history for retry functionality.
 
 ## Commands
 
@@ -45,10 +43,15 @@ app/
 ├── main.py              # FastAPI entry point, lifespan management
 ├── config.py            # 8 nested Pydantic settings classes
 ├── routes.py            # API endpoints (integrated with engine)
-├── text2sql_engine.py   # Core orchestrator (NEW - Issue #4)
+├── text2sql_engine.py   # Core orchestrator (Issue #4)
 ├── middleware.py        # CORS, logging, security headers
 ├── exceptions.py        # Custom exception hierarchy → HTTP status codes
-└── security/            # JWT auth, rate limiting, input validation
+├── security/            # JWT auth, rate limiting, input validation
+└── agent/               # Agent-based architecture (Issue #18)
+    ├── __init__.py      # Module exports
+    ├── models.py        # AgentResult, AgentStep, QueryHistoryEntry
+    ├── tools.py         # SQL executor, validator, schema inspector tools
+    └── engine.py        # AgentText2SQL with ReAct loop
 
 db/
 ├── connection.py        # DatabaseManager singleton, async pooling
@@ -61,42 +64,68 @@ models/
 └── prompts.py           # Schema-aware prompt templates
 ```
 
-### Text2SQL Engine (app/text2sql_engine.py)
+### Agent-Based Architecture (app/agent/)
 
-The central orchestrator that coordinates all SQL generation:
+The new agent module (Issue #18) provides self-correction capabilities:
 
 ```python
-from app.text2sql_engine import get_text2sql_engine, Text2SQLEngine
+from app.agent import get_agent_engine, AgentText2SQL
 
-engine = await get_text2sql_engine()
+engine = await get_agent_engine()
 result = await engine.generate_sql(
     natural_query="Show all customers from California",
     database_id="my_db",
-    execute=False,
+    execute=True,
     show_reasoning=True,
 )
-print(result.sql)           # Generated SQL
-print(result.confidence)    # Model confidence (0.0-1.0)
-print(result.valid_syntax)  # Validation result
-print(result.intent)        # QueryIntent enum (SELECT, AGGREGATE, JOIN, SUBQUERY)
+print(result.sql)              # Generated SQL
+print(result.confidence)       # Overall confidence (0.0-1.0)
+print(result.reasoning_trace)  # Full ReAct reasoning steps
+print(result.validation_result)  # Validation outcome
 ```
 
 **Key Components**:
-- `QueryIntent` enum: Classifies queries (SELECT, AGGREGATE, JOIN, SUBQUERY, UNKNOWN)
-- `SQLValidator`: Syntax checks, injection detection, schema alignment
-- `SchemaContext`: Database schema formatted for prompts
-- `SQLResult`: Complete result with metadata, warnings, reasoning trace
+- `AgentText2SQL`: Main engine with ReAct loop
+- `AgentStep`: Individual reasoning step (thought, action, observation)
+- `AgentResult`: Complete result with trace and validation
+- `QueryHistoryEntry`: Stored queries for retry functionality
+- Tools: `sql_executor`, `result_validator`, `schema_inspector`
+
+**ReAct Loop Flow**:
+1. **Thought**: Analyze query and determine approach
+2. **Action**: Generate SQL using Text2SQL model
+3. **Observation**: Execute and inspect results
+4. **Validation**: Check if results answer the question
+5. **Self-Correction**: If validation fails, iterate with hints
+
+### Text2SQL Engine (app/text2sql_engine.py)
+
+The original orchestrator (still available for simpler use cases):
+
+```python
+from app.text2sql_engine import get_text2sql_engine
+
+engine = await get_text2sql_engine()
+result = await engine.generate_sql(
+    natural_query="Show all customers from California",
+    database_id="my_db",
+)
+```
 
 ### Key Patterns
 
-- **Singletons**: `get_database()`, `get_settings()`, `get_model_loader()`, `get_text2sql_engine()` use `@lru_cache` or global instance
+- **Singletons**: `get_database()`, `get_settings()`, `get_model_loader()`, `get_text2sql_engine()`, `get_agent_engine()`
 - **Async-First**: All I/O is async; use `AsyncSession` from SQLAlchemy
 - **Exception Mapping**: `Text2SQLException` subclasses map to HTTP status codes automatically
 - **Rate Limiting**: Slowapi requires first parameter named exactly `request: Request`
 
 ### What's Implemented
 
 **Fully Working**:
+- Agent-based Text2SQL with ReAct framework (Issue #18)
+- Multi-step reasoning with self-correction
+- Query history and retry functionality
+- Result validation (checks if SQL answers the question)
 - Text2SQL Engine with orchestration pipeline
 - Query intent classification (SELECT, AGGREGATE, JOIN, SUBQUERY)
 - SQL validation (syntax, security, schema alignment)
@@ -106,42 +135,49 @@ print(result.intent)        # QueryIntent enum (SELECT, AGGREGATE, JOIN, SUBQUER
 - Schema introspection
 - Model loading with quantization
 - JWT authentication, rate limiting
-- Comprehensive test suite (277 tests)
+- Comprehensive test suite (300+ tests)
+
+**API Endpoints**:
+- `POST /api/v1/query` - Generate SQL with optional execution
+- `POST /api/v1/validate` - Validate SQL syntax and schema
+- `GET /api/v1/schema/{database_id}` - Get database schema
+- `GET /api/v1/agent/reasoning/{query_id}` - Get reasoning trace
+- `POST /api/v1/agent/retry` - Retry with correction hints
+- `POST /api/v1/auth/token` - JWT authentication
+- `GET /api/v1/health` - Health check
 
 **Partially Implemented**:
 - `POST /api/v1/schema/register` - Basic placeholder
-- `GET /api/v1/agent/reasoning/{query_id}` - Needs storage backend
-- `POST /api/v1/agent/retry` - Returns 501 (needs query history storage)
-
-**Planned for Issue #18**:
-- Full smolagents ReAct loop
-- Multi-step reasoning with self-correction
-- Tool-based SQL execution and validation
 
 ## Configuration
 
 Settings loaded via Pydantic from environment variables:
 
 ```python
 settings = get_settings()
-settings.huggingface.token      # HUGGINGFACE_TOKEN
-settings.database.url           # DATABASE_URL
-settings.api.debug              # API_DEBUG
-settings.agent.max_steps        # AGENT_MAX_STEPS (default: 5)
-settings.agent.min_confidence   # AGENT_MIN_CONFIDENCE (default: 0.7)
-settings.security.secret_key    # SECRET_KEY
+settings.huggingface.token        # HUGGINGFACE_TOKEN
+settings.database.url             # DATABASE_URL
+settings.api.debug                # API_DEBUG
+settings.agent.max_steps          # AGENT_MAX_STEPS (default: 5)
+settings.agent.min_confidence     # AGENT_MIN_CONFIDENCE (default: 0.7)
+settings.agent.enable_validation  # AGENT_ENABLE_VALIDATION (default: True)
+settings.agent.enable_self_correction  # AGENT_ENABLE_SELF_CORRECTION (default: True)
+settings.agent.verbosity          # AGENT_VERBOSITY (default: 1)
+settings.security.secret_key      # SECRET_KEY
 ```
 
 **Required env vars**: `HUGGINGFACE_TOKEN`, `DATABASE_URL`
 
 ## Testing
 
 ```bash
-pytest                                    # All tests
-pytest tests/unit/test_text2sql_engine.py -v  # Engine tests
-pytest tests/unit/test_inference.py -v   # Single file
-pytest -k "test_config" -v               # By name pattern
-pytest --pdb                             # Debug on failure
+pytest                                     # All tests
+pytest tests/unit/test_agent_engine.py -v  # Agent tests
+pytest tests/unit/test_agent_tools.py -v   # Agent tools tests
+pytest tests/unit/test_agent_models.py -v  # Agent models tests
+pytest tests/unit/test_text2sql_engine.py -v  # Core engine tests
+pytest -k "test_config" -v                 # By name pattern
+pytest --pdb                               # Debug on failure
 ```
 
 **Fixtures** (in `tests/conftest.py`): `test_settings`, `async_engine`, `db_manager`, `sample_schema`, `test_client`
@@ -191,10 +227,21 @@ if password == "demo":
 if password == "demo":  # nosec B105
 ```
 
+### Agent Self-Correction
+
+The agent automatically attempts correction when validation fails:
+
+```python
+# Validation detects: "Question asks for aggregation but SQL has none"
+# Agent will regenerate with hints:
+# - "Previous SQL was: SELECT amount FROM orders"
+# - "Consider using SUM(), COUNT(), AVG(), etc."
+```
+
 ## Project Tracking
 
 See GitHub Issues:
 - **#17**: Meta tracker with all issues
 - **#4**: Core Text2SQL Engine ✅ COMPLETED
-- **#18**: smolagents Agent Framework (CRITICAL - next priority)
+- **#18**: smolagents Agent Framework ✅ COMPLETED
 - **#12**: CI/CD Pipeline (needs deployment workflow)