Docs

Author: jeffmaki

.claude/settings.json

@@ -10,7 +10,9 @@
       "Bash(git config *)",
       "Bash(command -v gh)",
       "Bash(gh api *)",
-      "Bash(python3 -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\('protected:', d.get\\('protected'\\)\\); p=d.get\\('protection',{}\\); print\\('required_pull_request_reviews:', 'yes' if p.get\\('required_pull_request_reviews'\\) else 'no'\\); print\\('enforce_admins:', \\(p.get\\('enforce_admins'\\) or {}\\).get\\('enabled'\\)\\)\")"
+      "Bash(python3 -c \"import sys,json; d=json.load\\(sys.stdin\\); print\\('protected:', d.get\\('protected'\\)\\); p=d.get\\('protection',{}\\); print\\('required_pull_request_reviews:', 'yes' if p.get\\('required_pull_request_reviews'\\) else 'no'\\); print\\('enforce_admins:', \\(p.get\\('enforce_admins'\\) or {}\\).get\\('enabled'\\)\\)\")",
+      "Bash(grep -nA6 \"TENANT_BYPASSES: list\" api/main.py)",
+      "Bash(grep -nA10 \"STRIP_REQUEST_HEADERS = \" api/main.py)"
     ]
   }
 }

CLAUDE.md

@@ -116,6 +116,84 @@ enforced at this layer; if required, it must be enforced downstream
 (`workspaces-openstreetmap-website/`, `workspaces-cgimap/`) — that has not been
 audited here.
 
+## The OSM proxy layer: routing, auth, and user provisioning
+
+This service is a reverse proxy in front of the OSM website (`osm-rails`) and
+cgimap. The non-obvious parts, learned the hard way:
+
+### Deployment routing (workspaces-stack)
+
+In `workspaces-stack`, the **api container (this backend) serves the public OSM
+host** (`osm.workspaces-<env>...`) alongside the API hosts — its traefik router
+rule is `Host(new-api) || Host(api) || Host(osm)`, and the `osm-web` /
+`osm-log-proxy` routers are commented out. So **every OSM call the frontend
+makes routes through this backend**: `validate_token`, the `X-Workspace` gate,
+and the CORS middleware all apply. The backend then proxies `/api/0.6/*` to
+`WS_OSM_HOST` (config default `http://osm-web` — the internal service, *not* the
+public domain). `osm-web`'s nginx splits traffic: changeset read/write subpaths
+to cgimap, everything else to osm-rails.
+
+The frontend (`services/osm.ts`) calls the OSM host directly with
+`credentials: 'include'` + a Bearer TDEI token. Because it's *credentialed*
+CORS, `CORS_ORIGINS` must list the exact frontend origin (no `*`;
+`allow_credentials` is on). The stack supplies `WS_API_CORS_ORIGINS` as a **JSON
+array** (`["https://..."]`), while older config comma-split it — a JSON array
+would then become one malformed origin. `api/main.py` therefore reads
+`settings.cors_origins_list`, which parses **both** a JSON array and a
+comma-separated string (and `*`); set `CORS_ORIGINS` in either form.
+
+### Two databases; `users` is owned by OSM Rails
+
+`TASK_DATABASE_URL` (`alembic_task` tree) holds workspaces / tasking-manager
+tables; `OSM_DATABASE_URL` (`alembic_osm` tree) holds OSM data, `users`, and the
+`tasking_*` tables. The **`users` table is owned by the OSM Rails website**, not
+either alembic tree — the trees only FK to it, and integration tests stub it
+(`tests/integration/conftest.py`). The backend provisions `users` rows itself
+via raw SQL with `auth_provider='TDEI'` and `auth_uid = str(<JWT sub>)` (the OSM
+`auth_uid` **is** the token's `sub` claim).
+
+### How OSM authenticates, and the TDEI token bridge
+
+* **osm-rails** authenticates API calls *only* via **doorkeeper OAuth2**: it
+  looks the bearer token up in `oauth_access_tokens` (`token` →
+  `resource_owner_id` → `users.id`). Doorkeeper stores tokens **plaintext**
+  (`SecretStoring::Plain`), so the raw JWT matches directly. It has **no**
+  TDEI/JWT auth path.
+* **cgimap** has both: the oauth2 lookup *and* a custom TDEI path
+  (`get_user_id_for_tdei_token`, which verifies the JWT and matches
+  `users.auth_uid`).
+
+To make TDEI tokens work against osm-rails without forking it, the **token
+bridge** (`_bridge_token_to_osm` in `api/core/security.py`) mirrors a validated
+TDEI JWT into `oauth_access_tokens` — on the cache-miss path of `validate_token`
+(so ~once per token, not per request). Gated by `WS_OSM_TOKEN_BRIDGE_ENABLED`.
+It auto-provisions everything: a dedicated **system user** (`WS_OSM_SYSTEM_USER_*`)
+to own the doorkeeper `oauth_applications` row it creates (keyed by
+`WS_OSM_OAUTH_CLIENT_UID`; `oauth_applications.owner_id` is a NOT-NULL FK to
+`users`, hence the system user), the caller's `users` row, and the plaintext
+`oauth_access_tokens` row (`expires_in` from the JWT `exp`;
+`ON CONFLICT (token) DO UPDATE` so re-presenting reactivates it). On token
+rotation (new `jti`) the superseded token is revoked. Since the token then lives
+in `oauth_access_tokens`, both osm-rails and cgimap authenticate it via plain
+OAuth2 — cgimap's custom TDEI path becomes redundant.
+
+### Provisioning gotcha: `pass_crypt` must be 8..255 chars
+
+The OSM `User` model has `validates :pass_crypt, :length => 8..255`. When the
+backend provisions a `users` row, **`pass_crypt` must be 8–255 chars** (use a
+random throwaway — TDEI manages auth, so it's never used to log in). A too-short
+value (the old `'none'`) makes the user *invalid*. This is silent for **cgimap**
+operations — cgimap runs no Rails validations — but breaks any **osm-rails**
+operation that re-validates the author via `validates :author, :associated =>
+true`: notably `POST /api/0.6/changeset/{id}/comment` and note comments. The
+comment fails to save (no `id`), then rendering it throws *"Unable to serialize
+… without an id"*. Both provisioning paths (`_ensure_osm_user`,
+`_provision_users_from_tdei`) set a valid `pass_crypt`; migration
+`alembic_osm/versions/f3a7b9c1d2e4` heals legacy short rows. General principle:
+a backend-provisioned `users` row must satisfy OSM's `User` validations (also
+`display_name` 3..255 + unique, `email` present + unique) or Rails operations
+that touch it will fail even though cgimap operations succeed.
+
 ## Testing
 
 Two layers, both fast and dependency-free (no Postgres, PostGIS, Docker, or

README.md

@@ -6,6 +6,48 @@ This is a combination API backend for workspaces, providing /workspaces* methods
 the OSM API ("openstreetmap website") plus OSM CGI-map (the C-accelerated methods) that enforces 
 authorization and authentication based on a TDEI/Keycloak JWT token (see main.py for this proxy logic). 
 
+## What the proxy must provide for osm-rails / osm-web
+
+This backend is the **only entry point** to the OSM tier (osm-rails + cgimap,
+behind `osm-web`): in the deployment, the public OSM host routes to this
+container, which proxies `/api/0.6/*` to `WS_OSM_HOST` (default
+`http://osm-web`). For the OSM services to work, the proxy must uphold the
+following contract. `CLAUDE.md` has the full rationale.
+
+1. **Bridge TDEI auth into OSM's OAuth2.** osm-rails authenticates the API *only*
+   via doorkeeper OAuth2 (`oauth_access_tokens`); it has no TDEI/JWT path. So on
+   token validation the backend mirrors the TDEI JWT into `oauth_access_tokens`
+   in the OSM DB (the "token bridge" in `api/core/security.py`), and forwards the
+   incoming `Authorization: Bearer <token>` header unchanged. Then osm-rails and
+   cgimap authenticate the token via plain OAuth2. Controlled by
+   `WS_OSM_TOKEN_BRIDGE_ENABLED` (on by default) — with it off, osm-rails returns
+   **401** for TDEI tokens. The backend auto-creates the doorkeeper application
+   (and a system user to own it), so no manual OSM setup is required.
+
+2. **Provision valid OSM `users` rows.** The backend creates OSM `users` rows for
+   TDEI users (`auth_provider='TDEI'`, `auth_uid` = the JWT `sub`). These must
+   satisfy OSM's `User` validations — in particular `pass_crypt` length 8..255.
+   A too-short value is invisible to cgimap but makes osm-rails operations that
+   re-validate the user fail (e.g. posting a changeset comment or a note),
+   surfacing as *"Unable to serialize … without an id"*. The
+   `alembic_osm` migration `*_heal_short_tdei_pass_crypt` repairs legacy rows.
+
+3. **Carry workspace tenancy.** Workspace-scoped OSM requests must include an
+   `X-Workspace: <id>` header. The proxy authorizes it against the caller's
+   workspaces and forwards it (it is *not* stripped) so cgimap/osm-rails scope to
+   the `workspace-<id>` schema. A few paths are exempt (`TENANT_BYPASSES` in
+   `api/main.py`): workspace create/delete (`PUT`/`DELETE /api/0.6/workspaces/{id}`)
+   and user provisioning during sign-in (`PUT /api/0.6/user/{uid}`).
+
+4. **Set the proxy headers.** The proxy rewrites `Host` to the OSM host and sets
+   `X-Real-IP` / `X-Forwarded-For` / `-Host` / `-Proto`, while stripping
+   hop-by-hop headers and any spoofed forwarding headers from the client. It does
+   *not* strip `Authorization` or `X-Workspace`.
+
+5. **Connectivity.** `WS_OSM_HOST` must reach `osm-web`, and the backend needs
+   **both** `OSM_DATABASE_URL` and `TASK_DATABASE_URL` — the token bridge and
+   user provisioning write to the OSM database.
+
 ## Branch Index
 
 * ```develop``` merge your work here; keep this up to date with the "development" environment / dev tag