Simplify blob paths, fix Locust for KEDA scaling, clean up demo checklist

- Store blobs as {loan_id}/{doc_type}.pdf instead of {uuid}/{loan_id}/{doc_type}.pdf
- Reweight Locust to 80% async uploads (triggers KEDA), remove sync generate task
- Remove stale Postgres Flexible Server step from demo checklist (runs in-cluster)
- Update journal with full session work

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: johnmathews

docs/demo-guide.md

@@ -10,8 +10,7 @@ commands, timing, talking points, and things to name-drop with context on **why*
 Before the interview:
 
 - [ ] AKS cluster is running (`az aks start -n DocumentStreamManagedCluster -g documentstream`)
-- [ ] PostgreSQL is running (`az postgres flexible-server start -n documentstream-pg -g documentstream`)
-- [ ] All pods are healthy (`kubectl get pods -n documentstream`)
+- [ ] All pods are healthy (`kubectl get pods -n documentstream`) — PostgreSQL runs in-cluster, no separate start needed
 - [ ] Grafana is accessible and dashboard is loaded
 - [ ] Chaos Mesh dashboard is accessible
 - [ ] Locust is ready (either in-cluster or local)

journal/260402-chaos-mesh-testing-and-demo-rehearsal.md

@@ -55,10 +55,34 @@ Applied via `helm upgrade --install`. Chaos daemons restarted and all experiment
 4. **Node capacity:** Only 2 of 3 configured nodes are running. Some KEDA-scaled
    pods couldn't schedule. Not a blocker for the demo narrative but worth noting.
 
+## Additional fixes
+
+- **Blob storage paths:** Removed unnecessary UUID prefix from blob names. Documents
+  are now stored as `CRE-123456/contract.pdf` instead of `{uuid}/CRE-123456/contract.pdf`.
+  The loan ID grouping in the filename is sufficient.
+- **Locust load test:** Reweighted tasks so 80% of requests use the async `/api/documents`
+  endpoint (which goes through Redis and triggers KEDA). Removed the sync `/api/generate`
+  task which bypassed Redis entirely. Reduced wait time from 1-3s to 0.5-1.5s.
+- **Chaos Mesh dashboard RBAC:** Created a `chaos-admin` service account with cluster-admin
+  binding to fix the dashboard permission error. Generated a long-lived token for demo use.
+- **Demo checklist:** Removed the PostgreSQL Flexible Server start step — Postgres runs
+  in-cluster as a pod, starts automatically with AKS.
+- **Documentation audit:** Fixed test count in README (51 → 92), corrected Azure resource
+  names across all docs, updated implementation plan progress.
+- **CI fix:** `src/worker/store.py` had a ruff format issue (function args on one line).
+
 ## Files changed
 
 - `k8s/chaos/pod-kill.yaml` — changed mode from `fixed`/`value: "2"` to `one`
 - `infra/helm-install.sh` — added containerd runtime settings for Chaos Mesh
+- `infra/setup.sh` — corrected storage account name
 - `docs/chaos-experiments.md` — added containerd prerequisite note and verified results
-- `docs/demo-guide.md` — updated rolling update section to use gateway instead of workers
+- `docs/demo-guide.md` — updated rolling update section, removed stale Postgres checklist item
+- `docs/implementation-plan.md` — marked chaos, rolling update, demo rehearsal as DONE
+- `locust/locustfile.py` — reweighted for async pipeline, removed sync generate task
+- `src/worker/store.py` — simplified blob path (filename only, no UUID prefix)
+- `tests/test_store.py` — updated blob path assertions
+- `README.md` — corrected test count (51 → 92)
 - `CLAUDE.md` — removed chaos mesh and demo rehearsal from "Not yet done"
+- `.engineering-team/architecture-plan.md` — updated demo script and Azure resource names
+- `.github/workflows/deploy.yml` — corrected AKS cluster name and resource group

locust/locustfile.py

@@ -33,7 +33,7 @@ def _generate_pdf() -> bytes:
 class DocumentStreamUser(HttpUser):
     """Simulates a user interacting with the DocumentStream gateway."""
 
-    wait_time = between(1, 3)
+    wait_time = between(0.5, 1.5)
 
     # Generate the PDF payload once for the entire class so every upload
     # reuses the same bytes instead of rebuilding a PDF per request.
@@ -43,33 +43,28 @@ class DocumentStreamUser(HttpUser):
     # Tasks                                                               #
     # ------------------------------------------------------------------ #
 
-    @task(3)
+    @task(8)
     def upload_pdf(self) -> None:
-        """Upload a small PDF via multipart form — the main pipeline driver."""
+        """Upload a small PDF via multipart form — the main pipeline driver.
+
+        This is the primary task because it pushes documents through the Redis
+        async pipeline, which is what KEDA monitors for autoscaling.
+        """
         self.client.post(
             "/api/documents",
             files={"file": ("loadtest.pdf", io.BytesIO(self._pdf_bytes), "application/pdf")},
             name="/api/documents [upload]",
         )
 
     @task(1)
-    def generate_scenario(self) -> None:
-        """Generate a full loan scenario (5 documents) via the API."""
-        self.client.post(
-            "/api/generate",
-            json={"count": 1},
-            name="/api/generate",
-        )
-
-    @task(5)
     def list_documents(self) -> None:
         """List processed documents — lightweight read, simulates monitoring."""
         self.client.get(
             "/api/documents",
             name="/api/documents [list]",
         )
 
-    @task(2)
+    @task(1)
     def health_check(self) -> None:
         """Hit the health endpoint — simulates K8s probes and monitoring."""
         self.client.get(

src/worker/store.py

@@ -147,7 +147,7 @@ def upload_blob(
     """Upload a PDF to Azure Blob Storage.
 
     Args:
-        doc_id: Document ID (used as blob prefix).
+        doc_id: Document ID (for logging).
         filename: Original filename.
         pdf_bytes: Raw PDF content.
         doc_type: Document type label for metrics.
@@ -166,7 +166,7 @@ def upload_blob(
         container = blob_service.get_container_client(BLOB_CONTAINER)
         with contextlib.suppress(Exception):
             container.create_container()
-        blob_name = f"{doc_id}/{filename}"
+        blob_name = filename
         container.upload_blob(blob_name, pdf_bytes, overwrite=True)
 
         blob_uploads_total.labels(doc_type=doc_type).inc()

tests/test_store.py

@@ -177,7 +177,7 @@ def test_successful_upload(self, mock_blob_class: MagicMock) -> None:
 
         result = upload_blob("doc-1", "test.pdf", b"pdf-content", doc_type="invoice")
 
-        assert result == "doc-1/test.pdf"
+        assert result == "test.pdf"
         mock_container.upload_blob.assert_called_once_with(
-            "doc-1/test.pdf", b"pdf-content", overwrite=True
+            "test.pdf", b"pdf-content", overwrite=True
         )