|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Project Overview |
| 6 | + |
| 7 | +**pyplots** is an AI-powered platform for Python data visualization that automatically discovers, generates, tests, and maintains plotting examples. The platform is specification-driven: every plot starts as a library-agnostic Markdown spec, then AI generates implementations for matplotlib, seaborn, plotly, and other libraries. |
| 8 | + |
| 9 | +**Core Principle**: Community proposes plot ideas via GitHub Issues → AI generates code → Multi-LLM quality checks → Deployed. |
| 10 | + |
| 11 | +## Essential Commands |
| 12 | + |
| 13 | +### Development Setup |
| 14 | + |
| 15 | +```bash |
| 16 | +# Install dependencies (uses uv - fast Python package manager) |
| 17 | +uv sync --all-extras |
| 18 | + |
| 19 | +# Start backend API |
| 20 | +uv run uvicorn api.main:app --reload --port 8000 |
| 21 | + |
| 22 | +# Run database migrations |
| 23 | +uv run alembic upgrade head |
| 24 | +``` |
| 25 | + |
| 26 | +### Testing |
| 27 | + |
| 28 | +```bash |
| 29 | +# Run all tests |
| 30 | +uv run pytest |
| 31 | + |
| 32 | +# Run with coverage |
| 33 | +uv run pytest --cov=. --cov-report=html |
| 34 | + |
| 35 | +# Run specific test file |
| 36 | +uv run pytest tests/unit/api/test_routers.py |
| 37 | + |
| 38 | +# Run a single test |
| 39 | +uv run pytest tests/unit/api/test_routers.py::test_get_specs |
| 40 | +``` |
| 41 | + |
| 42 | +### Code Quality |
| 43 | + |
| 44 | +```bash |
| 45 | +# Check code formatting and linting |
| 46 | +uv run ruff check . |
| 47 | + |
| 48 | +# Auto-fix issues |
| 49 | +uv run ruff check . --fix |
| 50 | + |
| 51 | +# Format code |
| 52 | +uv run ruff format . |
| 53 | +``` |
| 54 | + |
| 55 | +### Frontend Development |
| 56 | + |
| 57 | +```bash |
| 58 | +cd app |
| 59 | +yarn install |
| 60 | +yarn dev # Development server |
| 61 | +yarn build # Production build |
| 62 | +``` |
| 63 | + |
| 64 | +## Architecture |
| 65 | + |
| 66 | +### Specification-First Design |
| 67 | + |
| 68 | +Every plot follows this flow: |
| 69 | +``` |
| 70 | +specs/{spec-id}.md → plots/{library}/{plot-type}/{spec-id}/default.py |
| 71 | +``` |
| 72 | + |
| 73 | +Example: |
| 74 | +``` |
| 75 | +specs/scatter-basic-001.md → plots/matplotlib/scatter/scatter-basic-001/default.py |
| 76 | + → plots/seaborn/scatterplot/scatter-basic-001/default.py |
| 77 | + → plots/plotly/scatter/scatter-basic-001/default.py |
| 78 | +``` |
| 79 | + |
| 80 | +The same spec ID links implementations across all libraries. |
| 81 | + |
| 82 | +### Directory Structure |
| 83 | + |
| 84 | +- **`specs/`**: Library-agnostic plot specifications (Markdown) |
| 85 | +- **`plots/{library}/{plot_type}/{spec_id}/{variant}.py`**: Library-specific implementations |
| 86 | +- **`core/`**: Shared business logic (database, repositories, config) |
| 87 | +- **`api/`**: FastAPI backend (routers, schemas, dependencies) |
| 88 | +- **`app/`**: Next.js frontend (React + TypeScript + Vite + MUI) |
| 89 | +- **`rules/`**: Versioned rules for AI code generation and quality evaluation |
| 90 | +- **`tests/unit/`**: Unit tests mirroring source structure |
| 91 | +- **`docs/`**: Architecture and workflow documentation |
| 92 | + |
| 93 | +### Key Architecture Patterns |
| 94 | + |
| 95 | +1. **Repository Pattern**: Data access layer in `core/repositories/` |
| 96 | +2. **Async Everything**: FastAPI + SQLAlchemy async + asyncpg |
| 97 | +3. **Clean Repo**: Only production code in git. Quality reports → GitHub Issues. Preview images → GCS. |
| 98 | +4. **Issue-Based Workflow**: GitHub Issues as state machine for plot lifecycle |
| 99 | + |
| 100 | +## Tech Stack |
| 101 | + |
| 102 | +- **Backend**: FastAPI, SQLAlchemy (async), PostgreSQL, Python 3.10+ |
| 103 | +- **Frontend**: Next.js 14, TypeScript, Tailwind CSS, MUI 7 |
| 104 | +- **Plotting**: matplotlib, seaborn, plotly |
| 105 | +- **Package Manager**: uv (fast Python installer) |
| 106 | +- **Infrastructure**: Google Cloud Run, Cloud SQL, Cloud Storage |
| 107 | +- **Automation**: GitHub Actions (code workflows) + n8n Cloud (external services) |
| 108 | +- **AI**: Claude (code generation), Vertex AI (multi-LLM quality checks) |
| 109 | + |
| 110 | +## Code Standards |
| 111 | + |
| 112 | +### Python Style |
| 113 | + |
| 114 | +- **Linter/Formatter**: Ruff (enforces PEP 8) |
| 115 | +- **Line Length**: 120 characters |
| 116 | +- **Type Hints**: Required for all functions |
| 117 | +- **Docstrings**: Google style for all public functions |
| 118 | +- **Import Order**: Standard library → Third-party → Local |
| 119 | + |
| 120 | +Example: |
| 121 | +```python |
| 122 | +def create_plot(data: pd.DataFrame, x: str, y: str, **kwargs) -> Figure: |
| 123 | + """ |
| 124 | + Create a scatter plot from DataFrame |
| 125 | +
|
| 126 | + Args: |
| 127 | + data: Input DataFrame with data |
| 128 | + x: Column name for x-axis |
| 129 | + y: Column name for y-axis |
| 130 | + **kwargs: Additional plotting parameters |
| 131 | +
|
| 132 | + Returns: |
| 133 | + Matplotlib Figure object |
| 134 | +
|
| 135 | + Raises: |
| 136 | + ValueError: If x or y column not found in data |
| 137 | + """ |
| 138 | + pass |
| 139 | +``` |
| 140 | + |
| 141 | +### Testing Standards |
| 142 | + |
| 143 | +- **Coverage Target**: 90%+ |
| 144 | +- **Test Structure**: Mirror source structure |
| 145 | +- **Naming**: `test_{what_it_does}` |
| 146 | +- **Fixtures**: Use pytest fixtures in `tests/conftest.py` |
| 147 | + |
| 148 | +### File Naming Conventions |
| 149 | + |
| 150 | +**Spec IDs**: `{type}-{variant}-{number}` (e.g., `scatter-basic-001`) |
| 151 | + |
| 152 | +**Implementation Variants**: |
| 153 | +- `default.py`: Standard implementation (required) |
| 154 | +- `{style}_style.py`: Style variants (e.g., `ggplot_style.py`) |
| 155 | +- `py{version}.py`: Version-specific (only when necessary) |
| 156 | + |
| 157 | +## Database |
| 158 | + |
| 159 | +**Connection**: PostgreSQL via SQLAlchemy async + asyncpg |
| 160 | + |
| 161 | +**What's Stored**: |
| 162 | +- Spec metadata (title, description, tags) |
| 163 | +- Implementation metadata (library, variant, quality score) |
| 164 | +- GCS URLs for preview images |
| 165 | +- Social media promotion queue |
| 166 | + |
| 167 | +**What's NOT Stored**: |
| 168 | +- Plot code (in repository) |
| 169 | +- Preview images (in GCS) |
| 170 | +- Quality reports (in GitHub Issues) |
| 171 | + |
| 172 | +**Migrations**: Managed with Alembic |
| 173 | +```bash |
| 174 | +# Create migration |
| 175 | +uv run alembic revision --autogenerate -m "description" |
| 176 | + |
| 177 | +# Apply migrations |
| 178 | +uv run alembic upgrade head |
| 179 | +``` |
| 180 | + |
| 181 | +## Versioned Rules System |
| 182 | + |
| 183 | +The `rules/` directory contains versioned rules for AI code generation and quality evaluation. |
| 184 | + |
| 185 | +### Working with Rules |
| 186 | + |
| 187 | +```bash |
| 188 | +# View current rules |
| 189 | +cat rules/versions.yaml |
| 190 | +cat rules/generation/v1.0.0-draft/code-generation-rules.md |
| 191 | + |
| 192 | +# Create new version |
| 193 | +cp -r rules/generation/v1.0.0-draft rules/generation/v1.1.0-draft |
| 194 | +# Edit files, then update rules/versions.yaml |
| 195 | +``` |
| 196 | + |
| 197 | +**Rule States**: draft → active → deprecated → archived |
| 198 | + |
| 199 | +**Why Versioned?** Enables A/B testing of rule improvements, provides audit trail, allows rollback. |
| 200 | + |
| 201 | +## Implementation Guidelines |
| 202 | + |
| 203 | +### Plot Implementation Template |
| 204 | + |
| 205 | +Every implementation file should: |
| 206 | +1. Start with docstring describing spec ID, library, variant |
| 207 | +2. Define `create_plot()` function with type hints |
| 208 | +3. Validate inputs first (empty data, missing columns) |
| 209 | +4. Use sensible defaults (`figsize=(10, 6)`, `alpha=0.8`) |
| 210 | +5. Include grid for readability |
| 211 | +6. Return the Figure object |
| 212 | +7. Optionally include `if __name__ == '__main__':` for standalone testing |
| 213 | + |
| 214 | +### Anti-Patterns to Avoid |
| 215 | + |
| 216 | +- No `preview.png` files in repository (use GCS) |
| 217 | +- No `quality_report.json` files (use GitHub Issues) |
| 218 | +- No hardcoded API keys (use environment variables) |
| 219 | +- Avoid version-specific files unless necessary (prefer single `default.py` for Python 3.10-3.13) |
| 220 | + |
| 221 | +## GitHub Actions Workflows |
| 222 | + |
| 223 | +Located in `.github/workflows/`: |
| 224 | + |
| 225 | +- **spec-to-code.yml**: Generates code from approved specs |
| 226 | +- **test-and-preview.yml**: Runs tests + creates preview images |
| 227 | +- **quality-check.yml**: Multi-LLM quality evaluation |
| 228 | +- **deploy.yml**: Deploys to Cloud Run |
| 229 | + |
| 230 | +## Environment Variables |
| 231 | + |
| 232 | +Required in `.env`: |
| 233 | +```bash |
| 234 | +DATABASE_URL=postgresql+asyncpg://user:pass@localhost:5432/pyplots |
| 235 | +ANTHROPIC_API_KEY=sk-ant-... |
| 236 | +GCS_BUCKET=pyplots-images-dev |
| 237 | +ENVIRONMENT=development |
| 238 | +FRONTEND_URL=http://localhost:3000 |
| 239 | +``` |
| 240 | + |
| 241 | +See `.env.example` for full list. |
| 242 | + |
| 243 | +## Common Development Tasks |
| 244 | + |
| 245 | +### Adding a New Plot Type |
| 246 | + |
| 247 | +1. Create GitHub Issue with spec description |
| 248 | +2. Add label `plot-idea` and wait for approval |
| 249 | +3. AI generates spec file in `specs/` |
| 250 | +4. AI generates implementations for all libraries |
| 251 | +5. Multi-LLM quality check runs automatically |
| 252 | +6. Human reviews PR and merges |
| 253 | + |
| 254 | +### Updating an Existing Implementation |
| 255 | + |
| 256 | +1. Create GitHub Issue referencing original spec |
| 257 | +2. Update implementation file |
| 258 | +3. Run tests: `pytest tests/unit/plots/{library}/test_{spec_id}.py` |
| 259 | +4. Generate preview by running implementation standalone |
| 260 | +5. Create PR with new preview |
| 261 | +6. Quality check runs automatically |
| 262 | + |
| 263 | +### Adding a Style Variant |
| 264 | + |
| 265 | +1. Create new file: `plots/{library}/{plot_type}/{spec_id}/{style}_style.py` |
| 266 | +2. Add tests |
| 267 | +3. Update database metadata |
| 268 | +4. Generate preview |
| 269 | + |
| 270 | +### Testing Plot Generation Locally |
| 271 | + |
| 272 | +```bash |
| 273 | +# Run implementation file directly |
| 274 | +python plots/matplotlib/scatter/scatter_basic_001/default.py |
| 275 | +``` |
| 276 | + |
| 277 | +## Cloud Deployment |
| 278 | + |
| 279 | +### Backend (FastAPI) |
| 280 | + |
| 281 | +```bash |
| 282 | +# Deploy to Cloud Run |
| 283 | +gcloud builds submit --config=api/cloudbuild.yaml --project=YOUR_PROJECT_ID |
| 284 | +``` |
| 285 | + |
| 286 | +### Frontend (Next.js) |
| 287 | + |
| 288 | +```bash |
| 289 | +# Deploy to Cloud Run |
| 290 | +gcloud builds submit --config=app/cloudbuild.yaml \ |
| 291 | + --substitutions=_VITE_API_URL=https://api.pyplots.ai |
| 292 | +``` |
| 293 | + |
| 294 | +## Debugging Tips |
| 295 | + |
| 296 | +### Database Connection Issues |
| 297 | +```bash |
| 298 | +psql -U pyplots -d pyplots -h localhost |
| 299 | +uv run alembic current |
| 300 | +``` |
| 301 | + |
| 302 | +### Import Errors |
| 303 | +```bash |
| 304 | +uv pip list |
| 305 | +uv sync --reinstall |
| 306 | +``` |
| 307 | + |
| 308 | +### Test Failures |
| 309 | +```bash |
| 310 | +pytest -v # Verbose |
| 311 | +pytest -s # Show prints |
| 312 | +pytest --pdb # Debug on failure |
| 313 | +``` |
| 314 | + |
| 315 | +## Key Documentation Files |
| 316 | + |
| 317 | +- **docs/development.md**: Comprehensive development guide |
| 318 | +- **docs/architecture/repository-structure.md**: Detailed directory structure |
| 319 | +- **docs/architecture/system-overview.md**: Architecture diagrams and data flows |
| 320 | +- **docs/workflow.md**: 7 automation workflows (Discovery → Deployment → Social) |
| 321 | +- **rules/README.md**: Rule versioning system documentation |
| 322 | + |
| 323 | +## Project Philosophy |
| 324 | + |
| 325 | +- **No manual coding**: AI generates and maintains all plot implementations |
| 326 | +- **Spec improvements over code fixes**: If a plot has issues, improve the spec, not the code |
| 327 | +- **Your data first**: Examples work with real user data, not fake data |
| 328 | +- **Community-driven**: Anyone can propose plots via GitHub Issues |
| 329 | +- **Multi-LLM quality**: Claude + Gemini + GPT ensure quality (score ≥85 required) |
| 330 | +- **Full transparency**: All feedback documented in GitHub Issues, not hidden in repo files |
0 commit comments