Skip to content

Commit cf29d93

Browse files
Snoww3dclaude
Snoww3d
and
claude
authored
docs: ADR 0001 + Python backend scaffolding for single-backend migration (Phase 0) (#1623)
* docs: ADR 0001 + Python backend scaffolding for single-backend migration (Phase 0) Adds the architecture decision record to collapse the three-tier polyglot stack into a two-service architecture (React frontend + Python FastAPI single backend), deleting the .NET gateway over a phased strangler-fig migration. Phase 0 (additive, zero behavior change): - ADR docs/architecture/adr/0001-collapse-to-python-single-backend.md, linked from the architecture index and mkdocs nav. - Scaffolding packages under processing-engine/app/: auth/, db/, library/, jobs/. Empty routers wired into main.py to establish the import graph and OpenAPI tags without changing runtime behavior. - requirements.txt: motor, pyjwt, passlib[bcrypt], bcrypt for the upcoming persistence + auth phases. - AGENTS.md and system-overview.md note the in-progress target architecture. https://claude.ai/code/session_01MWMesVhoX13SVmCcgjWrDF * fix: sort import blocks in scaffolding routers (ruff I001) Claude-Session: https://claude.ai/code/session_019QhpARsCtfZrenjm4shej7 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent c38fa94 commit cf29d93

14 files changed

Lines changed: 203 additions & 0 deletions

File tree

‎AGENTS.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,13 @@ JWST Data Analysis Application — a microservices-based platform for analyzing
88

99
**Architecture**: Frontend (React TypeScript) → Backend (.NET 10 API) → MongoDB + Processing Engine (Python FastAPI) → MAST Portal (STScI)
1010

11+
> **Migration in progress (ADR 0001):** collapsing to a two-service architecture —
12+
> the React frontend will talk directly to the Python FastAPI backend (which absorbs
13+
> auth, MongoDB persistence, jobs, and WebSocket progress) and the `.NET` gateway is
14+
> removed. New backend code goes in `processing-engine/app/` (`auth/`, `db/`,
15+
> `library/`, `jobs/`). See
16+
> [`docs/architecture/adr/0001-collapse-to-python-single-backend.md`](docs/architecture/adr/0001-collapse-to-python-single-backend.md).
17+
1118
| Service | URL | Tech |
1219
| ----------------- | ------ | ------------------------- |
1320
| Frontend | :3000 | React + Vite + TypeScript |

‎docs/architecture/adr/0001-collapse-to-python-single-backend.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# ADR 0001 — Collapse to a Two-Service Architecture (Python single backend)
2+
3+
- **Status:** Accepted (in progress)
4+
- **Date:** 2026-06-07
5+
- **Supersedes:** the three-tier polyglot architecture described in `system-overview.md`
6+
7+
## Context
8+
9+
The application runs a three-tier polyglot stack:
10+
11+
```
12+
React frontend (31k LOC)
13+
│ only ever talks to :5001
14+
▼
15+
.NET gateway (21k LOC + 23k test) ──HTTP proxy──► Python engine (17.5k LOC + 16k test)
16+
auth · MongoDB · jobs · SignalR · ALL real compute (astropy):
17+
storage · + proxies Composite/Mosaic/ composite, mosaic, mast,
18+
Mast/Analysis/Semantic to Python analysis, discovery, semantic
19+
│ │
20+
▼ ▼
21+
MongoDB (2 collections) MAST + shared storage (SeaweedFS = 4 containers)
22+
```
23+
24+
A full-stack review surfaced that the `.NET` `CompositeService`, `MosaicService`,
25+
`MastService`, `AnalysisService`, and `SemanticSearchService` are **HTTP proxies**
26+
(`_httpClient` → Python). The Python engine already owns every one of those domains
27+
end-to-end. The `.NET` tier's only *non-duplicated* responsibilities are:
28+
29+
- **Auth** (JWT issue/validate, register/login/refresh)
30+
- **MongoDB persistence** — only two collections: `jwst_data` and `users`
31+
- **Job tracking + SignalR push** (`/hubs/job-progress`) and its background workers
32+
(composite/mosaic/thumbnail/embedding/scan/reaper)
33+
- **Storage abstraction** — already duplicated by Python's `app/storage/provider.py`
34+
35+
Astropy and the scientific stack must stay in Python. The middle tier therefore adds
36+
a second language, a second service, and a `snake_case ↔ camelCase` boundary while
37+
mostly forwarding requests.
38+
39+
## Decision
40+
41+
Make the **Python FastAPI service the single backend** and **delete the .NET gateway**.
42+
The Python backend absorbs auth, MongoDB persistence, job tracking, and real-time
43+
progress (WebSocket replacing SignalR). The React frontend talks to one backend over
44+
HTTP + WebSocket.
45+
46+
### Target architecture
47+
48+
```
49+
React frontend ──HTTP + WebSocket──► Python FastAPI (single backend)
50+
compute + auth + persistence + jobs + WS
51+
│
52+
▼
53+
MongoDB · storage (volume/real S3) · MAST
54+
```
55+
56+
## Consequences
57+
58+
**Positive**
59+
60+
- One backend instead of two; one language deleted (~44k lines of C# incl. tests).
61+
- No more cross-language DTO/casing boundary.
62+
- Far fewer containers and compose files; simpler local and prod topology.
63+
- Each ported responsibility carries its security fix from the review
64+
(JWT-secret-from-env, password complexity, no seed passwords, locked job state,
65+
`asyncio.to_thread` for blocking `fits.open`, no `str(e)` leakage).
66+
67+
**Negative / risks**
68+
69+
- Auth, persistence, and job orchestration must be reimplemented in Python.
70+
- SignalR is replaced by a WebSocket protocol; the frontend client is rewritten once.
71+
- A single frontend cutover (Phase 5) is the riskiest step; it is reversible by
72+
reverting the API base URL while both backends still run.
73+
74+
## Migration roadmap (strangler-fig)
75+
76+
The system stays shippable after every phase. Phases 1–4 add Python capability behind
77+
the scenes while .NET still serves the frontend; Phase 5 is the one switchover; Phase 6
78+
deletes the gateway.
79+
80+
| Phase | Scope |
81+
|-------|-------|
82+
| 0 | This ADR + Python backend package scaffolding (`auth/`, `db/`, `library/`, `jobs/`) + deps |
83+
| 1 | MongoDB (`motor`) + Auth (`pyjwt`, `passlib`) in Python |
84+
| 2 | Library / persistence endpoints (`jwst_data` CRUD, upload, scan) |
85+
| 3 | Jobs + WebSocket progress + background workers |
86+
| 4 | MAST import + discovery wired to persistence/jobs |
87+
| 5 | Frontend cutover to Python + WebSocket |
88+
| 6 | Delete the `.NET` tier (`backend/`, CI jobs, compose services) |
89+
| 7 | Infrastructure simplification (SeaweedFS → volume/S3, collapse compose files) |
90+
| 8 | In-tier cleanup (god-files, duplication) |
91+
92+
## References
93+
94+
- `CODEBASE_REVIEW.md` — full-stack review and issue inventory
95+
- `system-overview.md` — current (pre-migration) architecture

‎docs/architecture/index.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,17 @@
22

33
Architecture documentation for the JWST Data Analysis Application, organized using the [4+1 Architectural View Model](https://en.wikipedia.org/wiki/4%2B1_architectural_view_model).
44

5+
> **In progress:** the system is migrating from the three-tier polyglot stack to a
6+
> two-service architecture (Python single backend). See
7+
> [ADR 0001 — Collapse to a Python single backend](adr/0001-collapse-to-python-single-backend.md).
8+
> Documents below describe the current (pre-migration) state until each phase lands.
9+
10+
---
11+
12+
## Decisions (ADRs)
13+
14+
- **[ADR 0001 — Collapse to a Python single backend](adr/0001-collapse-to-python-single-backend.md)** — delete the .NET gateway; Python/FastAPI becomes the one backend
15+
516
---
617

718
## +1 Scenarios (Use Cases)

‎docs/architecture/system-overview.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22

33
High-level view of the microservices architecture and their communication patterns.
44

5+
> **Migration in progress (ADR 0001).** The diagram below is the current state. The
6+
> target is a two-service architecture where the React frontend talks directly to the
7+
> Python FastAPI backend (compute + auth + persistence + jobs + WebSocket) and the
8+
> `.NET` gateway is removed. See
9+
> [ADR 0001 — Collapse to a Python single backend](adr/0001-collapse-to-python-single-backend.md).
10+
511
```mermaid
612
flowchart TB
713
subgraph Client["Client Layer"]

‎mkdocs.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ nav:
3939
- Architecture:
4040
- Overview: architecture/index.md
4141
- System Overview: architecture/system-overview.md
42+
- Decisions (ADRs):
43+
- "ADR 0001 — Python Single Backend": architecture/adr/0001-collapse-to-python-single-backend.md
4244
- "+1 Scenarios":
4345
- Use Case Catalog: architecture/use-case-catalog.md
4446
- Quality Attributes: architecture/quality-attributes.md

‎processing-engine/app/auth/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
"""Authentication package — JWT issue/validate, register/login/refresh.
2+
3+
Scaffolding for the Python single-backend migration. Logic lands in Phase 1;
4+
see docs/architecture/adr/0001-collapse-to-python-single-backend.md.
5+
"""

‎processing-engine/app/auth/routes.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
"""Authentication routes (scaffold).
2+
3+
Endpoints land in Phase 1 of the single-backend migration
4+
(docs/architecture/adr/0001-collapse-to-python-single-backend.md):
5+
``/api/auth/register``, ``/api/auth/login``, ``/api/auth/refresh``.
6+
7+
The router is intentionally empty for now so the import graph and OpenAPI tag
8+
are established without changing runtime behavior.
9+
"""
10+
11+
from fastapi import APIRouter
12+
13+
14+
router = APIRouter(prefix="/api/auth", tags=["Auth"])

‎processing-engine/app/db/__init__.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
"""Database package — async MongoDB access (motor).
2+
3+
Scaffolding for the Python single-backend migration. The repository and client
4+
factory land in Phases 1-2; see
5+
docs/architecture/adr/0001-collapse-to-python-single-backend.md.
6+
"""

‎processing-engine/app/jobs/__init__.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
"""Jobs package — async job tracking, workers, and WebSocket progress.
2+
3+
Scaffolding for the Python single-backend migration. The tracker, queue workers,
4+
reaper, and WebSocket endpoint land in Phase 3; see
5+
docs/architecture/adr/0001-collapse-to-python-single-backend.md.
6+
"""

‎processing-engine/app/jobs/routes.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
"""Jobs routes (scaffold).
2+
3+
Endpoints land in Phase 3 of the single-backend migration
4+
(docs/architecture/adr/0001-collapse-to-python-single-backend.md):
5+
``/api/jobs`` status queries plus the ``/ws/jobs`` WebSocket that replaces the
6+
``.NET`` SignalR ``/hubs/job-progress`` hub.
7+
8+
The router is intentionally empty for now so the import graph and OpenAPI tag
9+
are established without changing runtime behavior.
10+
"""
11+
12+
from fastapi import APIRouter
13+
14+
15+
router = APIRouter(prefix="/api/jobs", tags=["Jobs"])

0 commit comments

Comments
 (0)