|
| 1 | +# DocumentStream — Implementation Plan |
| 2 | + |
| 3 | +**Timeline:** 3 days (2026-03-28 to 2026-03-30) |
| 4 | +**Interview:** After Day 3 |
| 5 | +**Last updated:** 2026-03-29 |
| 6 | + |
| 7 | +--- |
| 8 | + |
| 9 | +## Progress Dashboard |
| 10 | + |
| 11 | +| Stage | What | Priority | Est. | Status | |
| 12 | +|---|---|---|---|---| |
| 13 | +| -- | **Day 1: Foundation** | -- | -- | **DONE** | |
| 14 | +| 0 | Tool setup (helm, kustomize) | MUST | 15min | TODO | |
| 15 | +| 1 | Azure infrastructure | MUST | 1.5-2h | TODO | |
| 16 | +| 2 | Redis Streams pipeline refactor | MUST | 2.5-3h | TODO | |
| 17 | +| 3 | K8s manifests | MUST | 2-2.5h | TODO | |
| 18 | +| 4 | Build, push, deploy to AKS | MUST | 1-1.5h | TODO | |
| 19 | +| 5 | KEDA autoscaling | MUST | 1-1.5h | TODO | |
| 20 | +| 6 | Grafana dashboard | HIGH | 1.5-2h | TODO | |
| 21 | +| 7 | Chaos Mesh experiments | MEDIUM | 1h | TODO | |
| 22 | +| 8 | Locust load testing | MEDIUM | 1h | TODO | |
| 23 | +| 9 | CI/CD deploy workflow | MEDIUM | 1h | TODO | |
| 24 | +| 10 | Rolling update demo prep | LOW | 30min | TODO | |
| 25 | +| 11 | Polish and demo rehearsal | MUST | 1.5-2h | TODO | |
| 26 | + |
| 27 | +**If time runs short:** Cut from the bottom. Stages 0-5 + 11 are non-negotiable. Stage 6 (Grafana) is |
| 28 | +the most important "nice to have" because it's the visual centerpiece of the demo. Stages 7-10 can |
| 29 | +be done live during the interview with `kubectl apply` if needed. |
| 30 | + |
| 31 | +--- |
| 32 | + |
| 33 | +## Day 1: Foundation (March 28) -- DONE |
| 34 | + |
| 35 | +Everything below is built and working. |
| 36 | + |
| 37 | +| Component | Files | Status | |
| 38 | +|---|---|---| |
| 39 | +| FastAPI gateway (synchronous) | `src/gateway/app.py`, `src/gateway/templates/index.html` | DONE | |
| 40 | +| PDF generator (5 templates) | `src/generator/scenario.py`, `templates.py`, `generate.py` | DONE | |
| 41 | +| Text extractor | `src/worker/extract.py` | DONE | |
| 42 | +| Rule-based classifier | `src/worker/classify.py` | DONE | |
| 43 | +| Semantic classifier | `src/worker/semantic.py` | DONE | |
| 44 | +| Tests (51 passing, 94% coverage) | `tests/test_*.py`, `tests/conftest.py` | DONE | |
| 45 | +| Gateway Dockerfile | `src/gateway/Dockerfile` | DONE | |
| 46 | +| Docker Compose (gateway + redis + postgres) | `docker-compose.yml` | DONE | |
| 47 | +| CI workflow (lint + test) | `.github/workflows/ci.yml` | DONE | |
| 48 | +| Docker build+push to ghcr.io | `.github/workflows/docker.yml` | DONE | |
| 49 | +| Documentation | `docs/architecture.md`, `classification.md`, `demo-guide.md`, `dictionary.md` | DONE | |
| 50 | +| Demo samples (1 loan, 5 PDFs) | `demo_samples/CRE-729976/` | DONE | |
| 51 | +| README | `README.md` | DONE | |
| 52 | + |
| 53 | +**Current architecture:** PDF upload -> FastAPI -> `extract_text()` -> `classify_text()` + `classify_semantic()` -> return results. All synchronous, all in-memory. Redis and PostgreSQL containers exist in docker-compose but the gateway doesn't connect to them yet. |
| 54 | + |
| 55 | +--- |
| 56 | + |
| 57 | +## Stage 0: Tool Setup (15 min) |
| 58 | + |
| 59 | +| # | Task | Exit Criteria | Done | |
| 60 | +|---|---|---|---| |
| 61 | +| 0.1 | `brew install helm kustomize` | `helm version` and `kustomize version` succeed | [ ] | |
| 62 | + |
| 63 | +--- |
| 64 | + |
| 65 | +## Stage 1: Azure Infrastructure (1.5-2h) |
| 66 | + |
| 67 | +Start this first -- AKS provisioning takes 5-10 minutes. Write Stage 2 code while it provisions. |
| 68 | + |
| 69 | +| # | Task | Files | Exit Criteria | Done | |
| 70 | +|---|---|---|---|---| |
| 71 | +| 1.1 | Write Azure setup script | `infra/setup.sh` | Creates resource group, ACR (Basic), AKS (3x Standard_B2ms, Free tier), PostgreSQL Flexible (Burstable B1ms, pg16), Storage Account (Standard_LRS) + blob container | [ ] | |
| 72 | +| 1.2 | Write teardown script | `infra/teardown.sh` | `az group delete` for full teardown. Separate `stop()` and `start()` functions for cost management | [ ] | |
| 73 | +| 1.3 | Write Helm install script | `infra/helm-install.sh` | Installs 5 charts: ingress-nginx, kube-prometheus-stack, bitnami/redis, kedacore/keda, chaos-mesh. Each in its own namespace | [ ] | |
| 74 | +| 1.4 | Run setup.sh | -- | `kubectl get nodes` shows 3 Ready nodes | [ ] | |
| 75 | +| 1.5 | Run helm-install.sh | -- | All Helm releases show `deployed` status | [ ] | |
| 76 | + |
| 77 | +**Dependencies:** None. This is the first thing to start. |
| 78 | + |
| 79 | +--- |
| 80 | + |
| 81 | +## Stage 2: Redis Streams Pipeline Refactor (2.5-3h) |
| 82 | + |
| 83 | +This is the highest-risk stage. The gateway currently calls worker functions directly |
| 84 | +(`src/gateway/app.py` lines 107-113). This must become: gateway publishes to Redis, |
| 85 | +workers consume and process independently. |
| 86 | + |
| 87 | +### Stream Design |
| 88 | + |
| 89 | +``` |
| 90 | +raw-docs {doc_id, filename, pdf_b64} -> extract-group |
| 91 | +extracted {doc_id, filename, text, page_count, -> classify-group |
| 92 | + word_count, pdf_b64} |
| 93 | +classified {doc_id, filename, text, pdf_b64, -> store-group |
| 94 | + classification, confidence, |
| 95 | + semantic_privacy, environmental_impact, |
| 96 | + industries, embedding} |
| 97 | +``` |
| 98 | + |
| 99 | +Redis hash `doc:{doc_id}` tracks status for gateway polling (queued -> extracting -> classifying -> completed). |
| 100 | + |
| 101 | +### Tasks |
| 102 | + |
| 103 | +| # | Task | Files | Exit Criteria | Done | |
| 104 | +|---|---|---|---|---| |
| 105 | +| 2.1 | Redis Streams utility module | `src/worker/queue.py` | `publish()`, `consume()`, `ack()`, `set_doc_status()`, `get_doc_status()`. Consumer group auto-creation with MKSTREAM. Configurable via env vars (`REDIS_URL`, stream names). PDF bytes base64-encoded. | [ ] | |
| 106 | +| 2.2 | Refactor gateway to dual-mode | `src/gateway/app.py` | If `REDIS_URL` is set: upload publishes to `raw-docs`, returns `status: queued`, list/get read from Redis hash. If not set: existing synchronous behavior unchanged. | [ ] | |
| 107 | +| 2.3 | Extract worker runner | `src/worker/extract_runner.py` | Infinite loop: XREADGROUP from `raw-docs`, call `extract_text()`, publish to `extracted`, XACK. SIGTERM graceful shutdown. | [ ] | |
| 108 | +| 2.4 | Classify worker runner | `src/worker/classify_runner.py` | XREADGROUP from `extracted`, call `classify_text()` + `classify_semantic()`, publish to `classified`, XACK. | [ ] | |
| 109 | +| 2.5 | Store worker | `src/worker/store.py` | PostgreSQL insertion: metadata + classification + vector(384) via pgvector. Optional Azure Blob upload for original PDF. | [ ] | |
| 110 | +| 2.6 | Store worker runner | `src/worker/store_runner.py` | XREADGROUP from `classified`, call store logic, update status to `completed`, XACK. | [ ] | |
| 111 | +| 2.7 | Database schema | `src/worker/schema.sql` | `CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE documents(...)` with pgvector column. | [ ] | |
| 112 | +| 2.8 | Worker Dockerfile | `src/worker/Dockerfile` | Single image (python:3.13-slim + uv), CMD overridden per K8s deployment. | [ ] | |
| 113 | +| 2.9 | Update docker-compose for local pipeline test | `docker-compose.yml` | Add extract-worker, classify-worker, store-worker services. `docker compose up` -> upload PDF -> document lands in PostgreSQL. | [ ] | |
| 114 | +| 2.10 | Tests for queue module | `tests/test_queue.py` | Test publish/consume/ack with mocked Redis. | [ ] | |
| 115 | +| 2.11 | Verify existing tests still pass | -- | All 51 tests pass (sync fallback preserved). | [ ] | |
| 116 | + |
| 117 | +**Dependencies:** None for code (2.1-2.8, 2.10-2.11). Task 2.9 needs docker-compose running. Stage 1 is NOT required -- local testing uses docker-compose. |
| 118 | + |
| 119 | +**Key design decision:** Dual-mode gateway preserves all existing tests. No test refactoring needed. |
| 120 | + |
| 121 | +--- |
| 122 | + |
| 123 | +## Stage 3: K8s Manifests (2-2.5h) |
| 124 | + |
| 125 | +| # | Task | Files | Exit Criteria | Done | |
| 126 | +|---|---|---|---|---| |
| 127 | +| 3.1 | Namespace | `k8s/base/namespace.yaml` | Namespace `documentstream` | [ ] | |
| 128 | +| 3.2 | ConfigMap | `k8s/base/configmap.yaml` | `REDIS_URL`, `DATABASE_URL`, stream names | [ ] | |
| 129 | +| 3.3 | Gateway Deployment + Service | `k8s/base/gateway-deployment.yaml`, `k8s/base/gateway-service.yaml` | 2 replicas, 128Mi-256Mi mem, 100m-250m CPU, liveness+readiness on `/health`, ClusterIP port 8000 | [ ] | |
| 130 | +| 3.4 | Extract worker Deployment | `k8s/base/extract-deployment.yaml` | 1 replica, command: `python -m worker.extract_runner` | [ ] | |
| 131 | +| 3.5 | Classify worker Deployment | `k8s/base/classify-deployment.yaml` | 1 replica, 512Mi mem (sentence-transformers model ~80MB, process ~300-400MB total) | [ ] | |
| 132 | +| 3.6 | Store worker Deployment | `k8s/base/store-deployment.yaml` | 1 replica, env includes database secret | [ ] | |
| 133 | +| 3.7 | Ingress | `k8s/base/ingress.yaml` | NGINX ingress routing to gateway service | [ ] | |
| 134 | +| 3.8 | Kustomization | `k8s/base/kustomization.yaml` | Lists all resources, common labels | [ ] | |
| 135 | + |
| 136 | +**Manifest patterns to demonstrate:** |
| 137 | +- Resource requests AND limits on every container |
| 138 | +- `terminationGracePeriodSeconds: 30` on workers (finish in-flight messages before SIGTERM) |
| 139 | +- `imagePullPolicy: Always` |
| 140 | +- `app` label on all pods (used by KEDA selector and Chaos Mesh targeting) |
| 141 | + |
| 142 | +**Dependencies:** Stage 2 code must be written (manifests reference worker runner commands and env vars). |
| 143 | + |
| 144 | +--- |
| 145 | + |
| 146 | +## Stage 4: Build, Push, Deploy (1-1.5h) |
| 147 | + |
| 148 | +| # | Task | Exit Criteria | Done | |
| 149 | +|---|---|---|---| |
| 150 | +| 4.1 | Build images to ACR | `az acr build -r documentstreamacr -t gateway:latest` and `-t worker:latest` | Images visible in ACR | [ ] | |
| 151 | +| 4.2 | Create K8s secrets | `kubectl create secret` for PostgreSQL password and Blob connection string | Secret exists in cluster | [ ] | |
| 152 | +| 4.3 | Initialize PostgreSQL schema | Run `schema.sql` against Azure PostgreSQL | `documents` table exists with vector extension | [ ] | |
| 153 | +| 4.4 | Apply manifests | `kubectl apply -k k8s/base/` | All pods Running, gateway `/health` returns 200 via port-forward | [ ] | |
| 154 | +| 4.5 | End-to-end test on AKS | Upload PDF via `/api/documents` or hit `/api/generate` | Document flows through all stages, lands in PostgreSQL | [ ] | |
| 155 | + |
| 156 | +**Dependencies:** Stages 1 (Azure running), 2 (code ready), 3 (manifests ready). |
| 157 | + |
| 158 | +--- |
| 159 | + |
| 160 | +## Stage 5: KEDA Autoscaling (1-1.5h) |
| 161 | + |
| 162 | +| # | Task | Files | Exit Criteria | Done | |
| 163 | +|---|---|---|---|---| |
| 164 | +| 5.1 | Extract ScaledObject | `k8s/scaling/extract-scaledobject.yaml` | Redis Streams scaler, `pollingInterval: 15`, `cooldownPeriod: 60`, min 1 / max 8, `lagCount: "5"` | [ ] | |
| 165 | +| 5.2 | Classify ScaledObject | `k8s/scaling/classify-scaledobject.yaml` | Same pattern as 5.1 | [ ] | |
| 166 | +| 5.3 | Store ScaledObject | `k8s/scaling/store-scaledobject.yaml` | Same pattern as 5.1 | [ ] | |
| 167 | +| 5.4 | Verify scaling | -- | Generate 50+ docs. `kubectl get pods -w` shows workers scaling 1 -> 3+. After queue drains (60s), back to 1. | [ ] | |
| 168 | + |
| 169 | +**Dependencies:** Stage 4 (pipeline running on AKS). |
| 170 | + |
| 171 | +**Day 2 gate: Pipeline running on AKS with KEDA scaling visible.** |
| 172 | + |
| 173 | +--- |
| 174 | + |
| 175 | +## Stage 6: Grafana Dashboard (1.5-2h) |
| 176 | + |
| 177 | +| # | Task | Files | Exit Criteria | Done | |
| 178 | +|---|---|---|---|---| |
| 179 | +| 6.1 | Build dashboard from existing metrics | `grafana/documentstream-dashboard.json` | 6-8 panels using metrics already collected (no custom app instrumentation needed) | [ ] | |
| 180 | +| 6.2 | Import dashboard into Grafana | -- | Dashboard visible in Grafana UI, all panels populated | [ ] | |
| 181 | + |
| 182 | +**Dashboard panels (all from existing Prometheus metrics):** |
| 183 | + |
| 184 | +| Panel | Metric Source | |
| 185 | +|---|---| |
| 186 | +| Pod count per deployment | `kube_deployment_status_replicas` (kube-state-metrics) | |
| 187 | +| CPU usage per pod | `container_cpu_usage_seconds_total` (cAdvisor) | |
| 188 | +| Memory usage per pod | `container_memory_working_set_bytes` (cAdvisor) | |
| 189 | +| Redis stream lengths | Redis exporter (bitnami/redis chart) | |
| 190 | +| Pod restarts (self-healing) | `kube_pod_container_status_restarts_total` | |
| 191 | +| KEDA scaling decisions | `keda_metrics_adapter_scaler_metrics_value` | |
| 192 | + |
| 193 | +**Stretch:** Add `prometheus-client` to gateway for custom metrics (request rate, latency histogram). Optional -- the dashboard above is sufficient for a compelling demo. |
| 194 | + |
| 195 | +**Dependencies:** Stage 4 (cluster running with metrics flowing). |
| 196 | + |
| 197 | +--- |
| 198 | + |
| 199 | +## Stage 7: Chaos Mesh Experiments (1h) |
| 200 | + |
| 201 | +| # | Task | Files | Demo Value | Done | |
| 202 | +|---|---|---|---|---| |
| 203 | +| 7.1 | Pod kill experiment | `k8s/chaos/pod-kill.yaml` | Kill 2 classify-worker pods. K8s restarts in seconds. Redis re-delivers unacked messages. Zero data loss. | [ ] | |
| 204 | +| 7.2 | Network delay experiment | `k8s/chaos/network-delay.yaml` | 500ms latency on store-worker. Pipeline slows but doesn't break. Grafana shows latency spike. | [ ] | |
| 205 | +| 7.3 | CPU stress experiment | `k8s/chaos/cpu-stress.yaml` | 80% CPU burn on classify-worker. KEDA scales up additional pods to compensate. | [ ] | |
| 206 | + |
| 207 | +**Dependencies:** Stage 5 (KEDA must be active to show compensating scale-up for 7.3). |
| 208 | + |
| 209 | +--- |
| 210 | + |
| 211 | +## Stage 8: Locust Load Testing (1h) |
| 212 | + |
| 213 | +| # | Task | Files | Exit Criteria | Done | |
| 214 | +|---|---|---|---|---| |
| 215 | +| 8.1 | Write load test scenarios | `locust/locustfile.py` | Two tasks: (1) upload pre-generated PDF, (2) hit `/api/generate?count=1`. Ramp 1 to 50 users. | [ ] | |
| 216 | +| 8.2 | Run against AKS | -- | Locust UI shows request rate climbing. KEDA scaling visible in `kubectl get pods -w`. | [ ] | |
| 217 | + |
| 218 | +**Fallback:** A bash `for` loop with `curl` is sufficient for the demo if Locust proves problematic. |
| 219 | + |
| 220 | +**Dependencies:** Stage 4 (pipeline running on AKS). |
| 221 | + |
| 222 | +--- |
| 223 | + |
| 224 | +## Stage 9: CI/CD Deploy Workflow (1h) |
| 225 | + |
| 226 | +| # | Task | Files | Exit Criteria | Done | |
| 227 | +|---|---|---|---|---| |
| 228 | +| 9.1 | Write deploy workflow | `.github/workflows/deploy.yml` | On push to main: `az acr build` for both images, `kubectl apply -k k8s/base/`. Requires `AZURE_CREDENTIALS` GitHub secret. | [ ] | |
| 229 | + |
| 230 | +**Dependencies:** Stage 4 (AKS running, images in ACR). |
| 231 | + |
| 232 | +--- |
| 233 | + |
| 234 | +## Stage 10: Rolling Update Demo (30 min) |
| 235 | + |
| 236 | +| # | Task | Exit Criteria | Done | |
| 237 | +|---|---|---|---| |
| 238 | +| 10.1 | Prepare bad image tag | Use nonexistent tag like `worker:v999` to trigger `ImagePullBackOff`. Old pods keep serving (rolling update strategy). `kubectl rollout undo` restores health. | [ ] | |
| 239 | + |
| 240 | +No new files. This is a live demo technique using existing manifests. |
| 241 | + |
| 242 | +**Dependencies:** Stage 4 (pipeline running on AKS). |
| 243 | + |
| 244 | +--- |
| 245 | + |
| 246 | +## Stage 11: Polish and Demo Rehearsal (1.5-2h) |
| 247 | + |
| 248 | +| # | Task | Exit Criteria | Done | |
| 249 | +|---|---|---|---| |
| 250 | +| 11.1 | Update docs to reflect actual deployment | `docs/architecture.md`, `docs/demo-guide.md`, `README.md` reflect real AKS state | [ ] | |
| 251 | +| 11.2 | Dry-run full 8-minute demo | Run every demo step against live cluster with timing | [ ] | |
| 252 | +| 11.3 | Prepare printable materials | Architecture diagram, cost breakdown, K8s concepts list | [ ] | |
| 253 | +| 11.4 | Stop cluster for cost management | `az aks stop` + `az postgres flexible-server stop`. Document restart commands. | [ ] | |
| 254 | + |
| 255 | +**Dependencies:** All previous stages complete (or explicitly cut). |
| 256 | + |
| 257 | +--- |
| 258 | + |
| 259 | +## Day-by-Day Schedule |
| 260 | + |
| 261 | +### Day 2 (March 29) -- Target: ~9.5 hours |
| 262 | + |
| 263 | +| Time | Stage | Hours | |
| 264 | +|---|---|---| |
| 265 | +| Morning start | Stage 0: Tool setup | 0.25 | |
| 266 | +| +15min | Stage 1: Azure infra (start AKS, write Stage 2 code while it provisions) | 1.75 | |
| 267 | +| +2h | Stage 2: Redis Streams pipeline refactor | 2.5 | |
| 268 | +| Break | -- | -- | |
| 269 | +| Afternoon | Stage 3: K8s manifests | 2.0 | |
| 270 | +| +2h | Stage 4: Build, push, deploy | 1.5 | |
| 271 | +| +1.5h | Stage 5: KEDA autoscaling | 1.5 | |
| 272 | + |
| 273 | +**Day 2 gate:** Pipeline running on AKS. KEDA scaling workers up and down under load. |
| 274 | + |
| 275 | +### Day 3 (March 30) -- Target: ~7.5 hours |
| 276 | + |
| 277 | +| Time | Stage | Hours | |
| 278 | +|---|---|---| |
| 279 | +| Morning start | Stage 6: Grafana dashboard | 2.0 | |
| 280 | +| +2h | Stage 7: Chaos Mesh experiments | 1.0 | |
| 281 | +| +1h | Stage 8: Locust load testing | 1.0 | |
| 282 | +| Break | -- | -- | |
| 283 | +| Afternoon | Stage 9: CI/CD deploy workflow | 1.0 | |
| 284 | +| +1h | Stage 10: Rolling update demo | 0.5 | |
| 285 | +| +30min | Stage 11: Polish + demo rehearsal | 2.0 | |
| 286 | + |
| 287 | +**Day 3 gate:** Full demo rehearsed end-to-end against live cluster. All docs updated. |
| 288 | + |
| 289 | +--- |
| 290 | + |
| 291 | +## Risk Mitigations |
| 292 | + |
| 293 | +| Risk | Mitigation | Fallback | |
| 294 | +|---|---|---| |
| 295 | +| AKS provisioning slow | Start Stage 1 FIRST. Write Stage 2 code in parallel. | Use Minikube locally for Day 1-2 dev. | |
| 296 | +| Redis Streams integration bugs | Test locally with `docker compose up` before touching AKS. Keep sync fallback. | Demo with synchronous mode if pipeline isn't ready. | |
| 297 | +| KEDA Redis scaler misconfigured | Check `kubectl logs -n keda deployment/keda-operator`. Consumer group must exist first. | Fall back to HPA with CPU-based scaling. | |
| 298 | +| sentence-transformers OOM | Set memory limit to 768Mi or 1Gi. Model is ~80MB, process ~300-400MB total. | Use rule-based only, skip semantic. | |
| 299 | +| Helm chart conflicts | Pin chart versions in `helm-install.sh`. | Install components one at a time. | |
| 300 | +| Running out of time | Stages 0-5 + 11 are MUST. Cut 6-10 from the bottom. | Even without Grafana, `kubectl get pods -w` shows scaling live. | |
| 301 | + |
| 302 | +--- |
| 303 | + |
| 304 | +## New Files Summary (~28 files) |
| 305 | + |
| 306 | +**Infrastructure (3):** |
| 307 | +`infra/setup.sh`, `infra/teardown.sh`, `infra/helm-install.sh` |
| 308 | + |
| 309 | +**Worker code (7):** |
| 310 | +`src/worker/queue.py`, `src/worker/extract_runner.py`, `src/worker/classify_runner.py`, |
| 311 | +`src/worker/store.py`, `src/worker/store_runner.py`, `src/worker/schema.sql`, `src/worker/Dockerfile` |
| 312 | + |
| 313 | +**K8s base manifests (8):** |
| 314 | +`k8s/base/namespace.yaml`, `k8s/base/configmap.yaml`, `k8s/base/gateway-deployment.yaml`, |
| 315 | +`k8s/base/gateway-service.yaml`, `k8s/base/extract-deployment.yaml`, |
| 316 | +`k8s/base/classify-deployment.yaml`, `k8s/base/store-deployment.yaml`, |
| 317 | +`k8s/base/ingress.yaml`, `k8s/base/kustomization.yaml` |
| 318 | + |
| 319 | +**K8s scaling (3):** |
| 320 | +`k8s/scaling/extract-scaledobject.yaml`, `k8s/scaling/classify-scaledobject.yaml`, |
| 321 | +`k8s/scaling/store-scaledobject.yaml` |
| 322 | + |
| 323 | +**K8s chaos (3):** |
| 324 | +`k8s/chaos/pod-kill.yaml`, `k8s/chaos/network-delay.yaml`, `k8s/chaos/cpu-stress.yaml` |
| 325 | + |
| 326 | +**Observability (1):** |
| 327 | +`grafana/documentstream-dashboard.json` |
| 328 | + |
| 329 | +**Load testing (1):** |
| 330 | +`locust/locustfile.py` |
| 331 | + |
| 332 | +**CI/CD (1):** |
| 333 | +`.github/workflows/deploy.yml` |
| 334 | + |
| 335 | +**Tests (1):** |
| 336 | +`tests/test_queue.py` |
0 commit comments