Merge branch 'development' into deck-pitch

Author: marcvergees

README.md

@@ -102,14 +102,74 @@ FireForm is designed from the ground up to ensure that all generated and collect
 
 ## 🔒 Privacy & Applicable Laws
 
-FireForm is built on a local-first architecture, ensuring that all processing (including AI extraction) occurs locally on the user's hardware. We do not collect, process, or share any Personally Identifiable Information (PII) with third parties. By keeping data completely offline, FireForm natively assists deploying organizations in complying with strict data protection laws such as **HIPAA**, **CCPA**, and **GDPR**. 
+FireForm is built on a **local-first architecture**: all processing (including AI extraction) occurs locally on the operator's hardware and nothing is transmitted to external servers by default. However, operators should be aware of the following:
 
-For complete details on our data handling, consent management procedures, and how the solution was designed to comply with HIPAA, CCPA, and GDPR, please review our public [Privacy Policy](https://fireform-core.github.io/FireForm/privacy.html).
+- **PII is processed locally.** Incident descriptions, names, dates, and other personally identifiable information are extracted by the local LLM, written into filled PDF files, and persisted in the local database (`FormSubmission` records). These files and records remain on disk until explicitly deleted.
+- **No external transmission.** Data does not leave the machine by default. Audio transcription (Whisper) and LLM inference (Ollama) are both local services.
+- **Operator responsibility.** Because PII is stored locally, the security of that data depends entirely on the operator's host system, filesystem permissions, backup practices, and access controls.
+
+For complete details on data handling and compliance guidance, please review our public [Privacy Policy](https://fireform-core.github.io/FireForm/privacy.html).
+
+## 🗑️ Data Deletion & Retention
+
+FireForm provides API endpoints and an automated purge mechanism to manage stored PII.
+
+### Deleting individual records
+
+```bash
+# Delete a single form submission (removes DB record + output PDF)
+curl -X DELETE http://localhost:8000/api/v1/forms/{submission_id} \
+  -H "X-API-Key: your-key"
+
+# Delete a template and all associated submissions
+curl -X DELETE http://localhost:8000/api/v1/templates/{template_id} \
+  -H "X-API-Key: your-key"
+```
+
+### Bulk purge
+
+```bash
+# Purge all submissions older than 30 days
+curl -X POST "http://localhost:8000/api/v1/forms/purge?days=30" \
+  -H "X-API-Key: your-key"
+```
+
+### Automated retention
+
+Set `RETENTION_PERIOD_DAYS` in `docker/.env.dev` (default: `30`). Celery Beat will automatically run a purge job every night at 03:00 UTC:
+
+```env
+RETENTION_PERIOD_DAYS=30
+```
+
+To enable the scheduler: `celery -A app.core.celery beat -l info`
+
+## 🔑 Access Control
+
+Deletion and purge endpoints can be protected by setting an API key:
+
+```env
+FIREFORM_API_KEY=your-strong-secret-key
+```
+
+When set, all `DELETE` and `POST /purge` requests must include one of:
+- **Header:** `X-API-Key: your-strong-secret-key`
+- **Bearer token:** `Authorization: Bearer your-strong-secret-key`
+
+Read-only endpoints (list, preview, fill) do **not** require authentication.
+
+## 🛡️ Operator Hardening Recommendations
+
+- **Filesystem permissions:** Restrict access to `data/inputs/` and `data/outputs/` to the application user only (`chmod 700`).
+- **Backups:** Encrypt any backups containing output PDFs.
+- **HTTPS:** Deploy behind a reverse proxy (e.g., nginx) with TLS enabled.
+- **API key rotation:** Rotate `FIREFORM_API_KEY` regularly and never commit it to version control.
+- **Retention policy:** Configure `RETENTION_PERIOD_DAYS` in line with applicable regulations (HIPAA, GDPR, CCPA).
 
 ## 🛡️ Do No Harm by Design
 
 FireForm is built to anticipate and prevent harm:
-- **Data Privacy & Security (9A):** We handle sensitive incident data entirely offline. By never transmitting data to the cloud, we natively prevent online data breaches of PII.
+- **Data Privacy & Security (9A):** We handle sensitive incident data entirely offline. By never transmitting data to the cloud, we natively prevent online data breaches of PII. Operators are responsible for securing local storage; see the Operator Hardening Recommendations above.
 - **Inappropriate Content (9B):** The application is an internal productivity tool without public social interactions or user-generated content hosting, completely mitigating the risks of public harassment or illegal content distribution.
 - **Protection from Harassment (9C):** For our open-source contributor community, we strictly enforce our [Code of Conduct](CODE_OF_CONDUCT.md) to ensure a safe, harassment-free environment for all contributors.
 

app/api/deps.py

@@ -1,4 +1,20 @@
 from app.db.database import get_session
+from fastapi import Header, Request
+from app.core.config import FIREFORM_API_KEY
+from app.core.errors.base import AppError
 
 def get_db():
-    yield from get_session()
+    yield from get_session()
+
+def verify_api_key(request: Request, x_api_key: str | None = Header(None)):
+    if not FIREFORM_API_KEY:
+        return
+    
+    provided_key = x_api_key
+    if not provided_key:
+        auth_header = request.headers.get("Authorization")
+        if auth_header and auth_header.startswith("Bearer "):
+            provided_key = auth_header[len("Bearer "):]
+            
+    if not provided_key or provided_key != FIREFORM_API_KEY:
+        raise AppError("Unauthorized", status_code=401, error_code="UNAUTHORIZED")

app/api/router.py

@@ -1,23 +1,12 @@
-"""Aggregates every route module into a single API router.
-
-Add new feature routers here; main.py only mounts this one router.
-
-IMPORTANT: Never add prefix="/api/v1" to api_router itself — that would
-silently move the existing /templates and /forms routes and break the frontend.
-New v1 endpoints live in app/api/v1/ and are included via v1_router below.
-"""
-
 from fastapi import APIRouter
 
-from app.api.routes import forms, jobs, templates
-from app.api.v1.router import v1_router
-from app.api.routes import forms, templates, weather, zipcode
+from app.api.routes import forms, templates, weather, zipcode, jobs, system
+from app.core.config import API_PREFIX
 
 api_router = APIRouter()
-api_router.include_router(templates.router)
-api_router.include_router(forms.router)
-api_router.include_router(v1_router)
-api_router.include_router(jobs.router)
-
-api_router.include_router(weather.router)
-api_router.include_router(zipcode.router)
+api_router.include_router(templates.router, prefix=API_PREFIX)
+api_router.include_router(forms.router, prefix=API_PREFIX)
+api_router.include_router(system.router, prefix=API_PREFIX)
+api_router.include_router(jobs.router, prefix=API_PREFIX)
+api_router.include_router(weather.router, prefix=API_PREFIX)
+api_router.include_router(zipcode.router, prefix=API_PREFIX)

app/api/routes/forms.py

@@ -1,20 +1,40 @@
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
 import requests
-from fastapi import APIRouter, Depends, File, UploadFile
-from sqlmodel import Session
+from fastapi import APIRouter, Depends, File, UploadFile, Query
+from sqlmodel import Session, select
 
-from app.api.deps import get_db
+from app.api.deps import get_db, verify_api_key
 from app.api.schemas.forms import (
     FormFill,
     FormFillResponse,
     ModelsResponse,
     TranscriptionResponse,
 )
-from app.core.config import OLLAMA_HOST, OLLAMA_MODEL, WHISPER_HOST
+from app.core.config import OLLAMA_HOST, OLLAMA_MODEL, WHISPER_HOST, BASE_DIR, RETENTION_PERIOD_DAYS
 from app.core.errors.base import AppError
-from app.db.repositories import create_form, get_template
-from app.models import FormSubmission, Template
+from app.db.repositories import create_form, get_template, get_form_submission, delete_form_submission
+from app.models import FormSubmission
 from app.services.controller import Controller
 
+PROJECT_ROOT = BASE_DIR
+
+def _resolve_project_file(file_path: str) -> Path:
+    raw_path = (file_path or "").strip()
+    if not raw_path:
+        raise AppError("Path is required", status_code=400)
+
+    candidate = Path(raw_path)
+    if not candidate.is_absolute():
+        candidate = (PROJECT_ROOT / candidate).resolve()
+    else:
+        candidate = candidate.resolve()
+
+    if candidate != PROJECT_ROOT and PROJECT_ROOT not in candidate.parents:
+        raise AppError("Path must be inside the project", status_code=400)
+
+    return candidate
+
 router = APIRouter(prefix="/forms", tags=["forms"])
 
 
@@ -106,6 +126,48 @@ def transcribe(audio: UploadFile = File(...)):
     return TranscriptionResponse(text=text)
 
 
+@router.delete("/{submission_id}", dependencies=[Depends(verify_api_key)])
+def delete_submission_endpoint(submission_id: int, db: Session = Depends(get_db)):
+    sub = get_form_submission(db, submission_id)
+    if not sub:
+        raise AppError("Submission not found", status_code=404, error_code="SUBMISSION_NOT_FOUND")
+
+    if sub.output_pdf_path:
+        try:
+            resolved_out = _resolve_project_file(sub.output_pdf_path)
+            if resolved_out.exists() and resolved_out.is_file():
+                resolved_out.unlink()
+        except Exception:
+            pass
+
+    delete_form_submission(db, sub)
+    return {"status": "success", "message": "Submission and associated output file deleted"}
+
+
+@router.post("/purge", dependencies=[Depends(verify_api_key)])
+def purge_submissions_endpoint(days: int = Query(default=None), db: Session = Depends(get_db)):
+    retention_days = days if days is not None else RETENTION_PERIOD_DAYS
+    cutoff_date = datetime.now(timezone.utc) - timedelta(days=retention_days)
+
+    statement = select(FormSubmission).where(FormSubmission.created_at < cutoff_date)
+    submissions = list(db.exec(statement))
+
+    purged_count = 0
+    for sub in submissions:
+        if sub.output_pdf_path:
+            try:
+                resolved_out = _resolve_project_file(sub.output_pdf_path)
+                if resolved_out.exists() and resolved_out.is_file():
+                    resolved_out.unlink()
+            except Exception:
+                pass
+        delete_form_submission(db, sub)
+        purged_count += 1
+
+    return {"status": "success", "purged_count": purged_count, "retention_days_used": retention_days}
+
+
+
 @router.get("/submissions")
 def get_submissions(db: Session = Depends(get_db)):
     from sqlmodel import select

app/api/v1/routes/system.py app/api/routes/system.pyapp/api/v1/routes/system.py renamed to app/api/routes/system.py

@@ -6,7 +6,7 @@
 from fastapi.responses import FileResponse
 from sqlmodel import Session
 
-from app.api.deps import get_db
+from app.api.deps import get_db, verify_api_key
 from app.api.schemas.templates import (
     TemplateCreate,
     TemplateResponse,
@@ -15,9 +15,10 @@
     MakeFillableResponse,
 )
 from app.core.config import BASE_DIR, DEFAULT_TEMPLATE_DIR
-from app.db.repositories import create_template, list_templates
-from app.models import Template
+from app.db.repositories import create_template, list_templates, get_template, delete_template
+from app.models import Template, FormSubmission, Job
 from app.services.controller import Controller
+from sqlmodel import select
 
 router = APIRouter(prefix="/templates", tags=["templates"])
 PROJECT_ROOT = BASE_DIR
@@ -197,3 +198,43 @@ def make_fillable(req: MakeFillableRequest):
         pdf_path=relative_path,
         field_count=_count_pdf_widgets(relative_path),
     )
+
+
+@router.delete("/{template_id}", dependencies=[Depends(verify_api_key)])
+def delete_template_endpoint(template_id: int, db: Session = Depends(get_db)):
+    template = get_template(db, template_id)
+    if not template:
+        raise HTTPException(status_code=404, detail="Template not found")
+
+    # 1. Clean up associated submissions and their generated PDFs
+    sub_stmt = select(FormSubmission).where(FormSubmission.template_id == template_id)
+    submissions = list(db.exec(sub_stmt))
+    for sub in submissions:
+        if sub.output_pdf_path:
+            try:
+                resolved_out = _resolve_project_file(sub.output_pdf_path)
+                if resolved_out.exists() and resolved_out.is_file():
+                    resolved_out.unlink()
+            except Exception:
+                pass
+        db.delete(sub)
+
+    # 2. Clean up associated jobs
+    job_stmt = select(Job).where(Job.template_id == template_id)
+    jobs = list(db.exec(job_stmt))
+    for job in jobs:
+        db.delete(job)
+
+    # 3. Delete template PDF file
+    if template.pdf_path:
+        try:
+            resolved_pdf = _resolve_project_file(template.pdf_path)
+            if resolved_pdf.exists() and resolved_pdf.is_file():
+                resolved_pdf.unlink()
+        except Exception:
+            pass
+
+    # 4. Delete the template itself
+    delete_template(db, template)
+    return {"status": "success", "message": "Template and all associated data deleted"}
+

app/api/routes/templates.py

@@ -16,4 +16,14 @@
     result_expires=86400,
 )
 
-celery_app.conf.include = ["app.tasks.fill"]
+celery_app.conf.include = ["app.tasks.fill", "app.tasks.purge"]
+
+# Optional Celery Beat schedule — runs purge_old_submissions once a day.
+# Enable by running: celery -A app.core.celery beat
+from celery.schedules import crontab  # noqa: E402
+celery_app.conf.beat_schedule = {
+    "daily-submission-purge": {
+        "task": "purge_old_submissions",
+        "schedule": crontab(hour=3, minute=0),  # 03:00 UTC daily
+    },
+}