Skip to content

Commit 065875a

Browse files
chore: implement audit quick wins and architectural doc updates (#5506)
This PR addresses several quick wins identified in the recent audit: - Fix legacy Python 2 except syntax in scripts and plots - Update Claude model name to valid Anthropic identifier - Sync architectural docs and agentic commands with modular router structure - Complete Web Vitals reporting (added FCP and TTFB) - Fix typo in agentic command filename (dokument.md -> document.md) - Use settings.claude_max_tokens in scripts to avoid hardcoded limits - Optimize model routing in plan.py (Phase 1 uses small model)
1 parent dbba743 commit 065875a

17 files changed

Lines changed: 169 additions & 235 deletions

File tree

agentic/audits/2026-04-27-all.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Audit Report: anyplot
2+
3+
**Date:** 2026-04-27 | **Scope:** all | **Mode:** full
4+
**Health Score:** 30 | **Baseline:** ruff: 0 issues, format: formatted
5+
**Auditors:** 15 ran (backend, frontend, infra, quality, llm-pipeline, db, security, observability, agentic, gcloud, github, plausible, pagespeed, seo, catalog) | **Findings:** 22 | **Auto-fixable:** 3/22
6+
**External sources:**
7+
- GCP project: pyplots (gcloud-auditor - BLOCKED: project mismatch)
8+
- Plausible site: anyplot.ai (plausible-auditor - BLOCKED: credentials missing)
9+
- Search Console mode: structural-only | freshness: 2026-04-27 (seo-auditor)
10+
- GitHub: MarkusNeusinger / anyplot (github-auditor)
11+
- Catalog DB rows: 327 specs (catalog-auditor)
12+
13+
## Summary
14+
The anyplot repository exhibits high technical excellence in its frontend and core AI generation workflows but suffers from critical infrastructure and baseline safety issues. The use of experimental Python 3.14, a major command injection vulnerability in CI/CD, and broken Python syntax in automation scripts significantly compromise production readiness.
15+
16+
**Note:** Several critical and high-severity findings listed below (Python syntax errors, invalid model name, documentation drift, and missing Web Vitals) were addressed in PR #5506.
17+
18+
## Quick Wins (Importance ≥4 & Effort=S)
19+
| # | Finding | Auto-fix | Files | Hint |
20+
|---|---------|----------|-------|------|
21+
| 1 | SyntaxError: Python 2 style 'except' blocks | ruff | `automation/scripts/sync_to_postgres.py`, `plots/stereonet-equal-area/implementations/python/highcharts.py` | Change `except E1, E2:` to `except (E1, E2):` |
22+
| 2 | Critical Versioning Risk: Python 3.14 | manual | `pyproject.toml`, Dockerfiles, `.github/workflows/*.yml` | Downgrade to stable Python 3.12 or 3.13 |
23+
| 3 | Invalid Claude Model Name | manual | `core/config.py` | Change `claude-sonnet-4-6` to `claude-3-5-sonnet-20240620` |
24+
| 4 | Stale Agentic Commands / Docs | manual | `agentic/commands/*.md`, `docs/reference/*.md` | Update structure references to modular routers |
25+
26+
## Critical (Importance 5)
27+
| # | Finding | Effort | Auto-fix | Files | Hint |
28+
|---|---------|--------|----------|-------|------|
29+
| 1 | Command Injection in Workflows | M | manual | `.github/workflows/spec-create.yml` | Use unquoted heredoc `<<'EOF'` to prevent shell expansion of untrusted content |
30+
| 2 | SyntaxError in Python scripts | S | ruff | `automation/scripts/sync_to_postgres.py`, `plots/.../highcharts.py` | Fix Python 2 style except blocks |
31+
| 3 | Experimental Python 3.14 in Production | S | manual | `pyproject.toml`, `Dockerfile`, `.github/workflows/` | Downgrade to stable Python (3.12/3.13) |
32+
| 4 | Missing Branch Protection on `main` | S | manual | `gh:branches/main` | Enable required reviews and status checks via GH settings |
33+
34+
## High (Importance 4)
35+
| # | Finding | Effort | Auto-fix | Files | Hint |
36+
|---|---------|--------|----------|-------|------|
37+
| 1 | Model-migration Drift (Missing Indexes) | M | manual | `alembic/versions/`, `core/database/models.py` | Run `alembic revision --autogenerate` to sync indexes |
38+
| 2 | Invalid Model Name `claude-sonnet-4-6` | S | manual | `core/config.py` | Use valid Anthropic model identifier |
39+
| 3 | Missing Prompt Caching | M | manual | `.github/workflows/`, `prompts/` | Add `cache_control: {"type": "ephemeral"}` to static guides |
40+
| 4 | Missing Web Vitals (FCP/TTFB) | S | manual | `app/src/analytics/reportWebVitals.ts` | Instrument missing Core Web Vitals |
41+
| 5 | Missing LLM Observability | M | manual | `scripts/evaluate-plot.py`, `scripts/upgrade_specs_ai.py` | Log token counts and latency for all LLM calls |
42+
| 6 | Lack of Request/Correlation IDs | M | manual | `api/main.py`, `core/config.py` | Add Request-ID middleware for async log correlation |
43+
| 7 | Type-checking Bypass (mypy ignore_errors) | M | manual | `pyproject.toml` | Remove `ignore_errors = true` for core modules |
44+
| 8 | Architectural Drift in Documentation | S | manual | `docs/reference/`, `README.md` | Update docs to reflect modular router structure |
45+
46+
## Medium (Importance 3)
47+
| # | Finding | Effort | Auto-fix | Files | Hint |
48+
|---|---------|--------|----------|-------|------|
49+
| 1 | Scalability Bottleneck in Filtering | L | manual | `api/routers/plots.py` | Move filtering logic from in-memory to SQL |
50+
| 2 | God Test File `test_routers.py` | M | manual | `tests/unit/api/test_routers.py` | Split large test file into modular router tests |
51+
| 3 | Implementation Gaps in Catalog | XL | manual | `plots/` | Generate missing implementations for newer specs |
52+
| 4 | Label Fragmentation | S | manual | `gh:labels` | Consolidate quality score labels |
53+
| 5 | Agentic Command Typo (`dokument.md`) | S | codemod | `agentic/commands/dokument.md` | Rename to `document.md` and update references |
54+
55+
## Positive Patterns (Importance 1)
56+
- **Exceptional Frontend Quality**: React 19, zero `any` usage, robust accessibility, and smart error boundaries.
57+
- **Secure Prompt Design**: Hallucination mitigation via grounding examples and strict role definitions.
58+
- **Strong Test Coverage**: 1:1 test mapping for automation scripts ensuring reliability of the generation pipeline.
59+
- **Conditional Context Loading**: `agentic/commands/context.md` efficiently manages context window.
60+
61+
## Statistics
62+
- Total: 22 | Critical: 4, High: 8, Medium: 6, Low: 0, Positive: 4
63+
- Effort: S 10, M 8, L 2, XL 2
64+
- Auto-fix: ruff 1, codemod 1, manual 20
65+
- By Auditor: backend 5, frontend 0, infra 2, quality 2, llm 3, db 1, security 1, obs 4, agentic 2, gcloud 0, github 1, plausible 0, pagespeed 0, seo 0, catalog 1
66+
- Cross-validation: 13 reviewed, 0 dropped, 0 downgraded
67+
- Coverage: 8 auditors complete, 4 partial, 3 blocked (gcloud, plausible, pagespeed)

0 commit comments

Comments
 (0)