Skip to content

Commit 815b9e1

Browse files
authored
Merge pull request #30 from TaskarCenterAtUW/main
Main -> Develop
2 parents 09d1ed5 + 8c4c4ae commit 815b9e1

60 files changed

Lines changed: 5074 additions & 492 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 20 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -6,77 +6,37 @@ on:
66
pull_request:
77
branches: [ main, develop ]
88

9+
# Cancel any in-progress run for the same branch/PR when a newer commit lands.
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
914
jobs:
10-
lint:
15+
ci:
1116
runs-on: ubuntu-latest
1217
strategy:
1318
matrix:
1419
python-version: ["3.12"]
1520

1621
steps:
17-
- uses: actions/checkout@v4
22+
- uses: actions/checkout@v5
1823

1924
- name: Install uv
20-
uses: astral-sh/setup-uv@v4
25+
uses: astral-sh/setup-uv@v6
2126
with:
2227
version: "0.5.8"
23-
24-
- name: Set up Python
25-
uses: actions/setup-python@v5
26-
with:
27-
python-version: "3.12"
28-
29-
- name: Install the project
30-
run: uv sync --all-extras
31-
32-
- name: Check imports with isort
33-
run: uv run isort --check-only --diff .
34-
35-
- name: Check code formatting with black
36-
run: uv run black --check .
28+
enable-cache: true
3729

38-
test:
39-
needs: lint
40-
runs-on: ubuntu-latest
41-
strategy:
42-
matrix:
43-
python-version: ["3.12"]
44-
45-
steps:
46-
- uses: actions/checkout@v4
47-
48-
- name: Install uv
49-
uses: astral-sh/setup-uv@v4
50-
with:
51-
version: "0.5.8"
52-
5330
- name: Set up Python ${{ matrix.python-version }}
54-
uses: actions/setup-python@v5
31+
uses: actions/setup-python@v6
5532
with:
56-
python-version: "3.12"
57-
58-
- name: Install the project
59-
run: uv sync --all-extras
60-
61-
- name: Run tests
62-
run: uv run pytest tests
63-
env:
64-
TASK_DATABASE_URL: "postgresql+asyncpg://postgres:postgres@localhost:5432/test_db"
65-
OSM_DATABASE_URL: "postgresql+asyncpg://postgres:postgres@localhost:5432/test_db"
66-
JWT_SECRET: "test_secret_key"
67-
JWT_ALGORITHM: "HS256"
68-
69-
services:
70-
postgres:
71-
image: postgres:16
72-
env:
73-
POSTGRES_USER: postgres
74-
POSTGRES_PASSWORD: postgres
75-
POSTGRES_DB: test_db
76-
ports:
77-
- 5432:5432
78-
options: >-
79-
--health-cmd pg_isready
80-
--health-interval 10s
81-
--health-timeout 5s
82-
--health-retries 5
33+
python-version: ${{ matrix.python-version }}
34+
35+
# Runs the same checks locals get from scripts/ci.sh: uv sync, isort,
36+
# black, pyright, and pytest. `--integration` additionally runs the
37+
# PostGIS/testcontainers suite (`pytest -m integration`); the ubuntu-latest
38+
# runner ships a running Docker daemon, which testcontainers needs to boot
39+
# the database. The script runs every check and exits non-zero if any fail,
40+
# so a single red step still lists all failures.
41+
- name: Run CI checks
42+
run: ./scripts/ci.sh --integration

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ repos:
1212
name: isort (python)
1313

1414
- repo: https://github.com/psf/black
15-
rev: 24.1.1
15+
rev: 24.10.0
1616
hooks:
1717
- id: black
1818
language_version: python3.12

.vscode/settings.json

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,16 @@
33
"alembic"
44
],
55
"python.testing.unittestEnabled": false,
6-
"python.testing.pytestEnabled": true
6+
"python.testing.pytestEnabled": true,
7+
"[python]": {
8+
"editor.defaultFormatter": "ms-python.black-formatter",
9+
"editor.formatOnSave": true,
10+
"editor.codeActionsOnSave": {
11+
"source.organizeImports": "explicit"
12+
}
13+
},
14+
"isort.args": [
15+
"--profile",
16+
"black"
17+
]
718
}

CLAUDE.md

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# CLAUDE.md
2+
3+
Guidance for working in this repo. Focused on the test infrastructure and
4+
conventions established for it; see `README.md` for app setup.
5+
6+
## Permission Structure
7+
8+
Project Group Admin ("POC")
9+
* Superuser for the whole project group
10+
* Implied by "poc" role in TDEI
11+
12+
Lead/Owner/Workspace Admin
13+
* Admin-level access for a workspace
14+
* Configures workspace settings and quest definitions
15+
* Assigns users to workspace teams
16+
* Ability to merge changes from other workspace
17+
* Exports data to TDEI (with appropriate TDEI core roles)
18+
* Granted by Workspaces setting.
19+
20+
Contributor/Data Generator
21+
* Modifies workspace data--all modifications need validation
22+
* Implied by membership in TDEI project group
23+
24+
Validator
25+
* Modifies workspace data and approves changes from contributors
26+
* Granted by Workspaces setting.
27+
28+
Viewer/Member/Everyone Else
29+
* Read-only access to workspace data
30+
* With express TDEI sign-up, the need for this access level diminishes greatly
31+
* Granted by Workspaces setting.
32+
33+
## What Each Role Can Do
34+
35+
Project Lead
36+
* Edit Metadata
37+
* Edit Longform Quests
38+
* Toggle App-Enabled Flag
39+
* Delete Workspace
40+
* Define User Teams
41+
* Define Groups or Roles
42+
* Export to TDEI
43+
* Validate Changeset
44+
* Move Workspace from Project Group to Project Group
45+
* Edit POSM Element
46+
47+
Validator
48+
* Export to TDEI
49+
* Validate Changeset
50+
* Edit POSM Element
51+
52+
Contributor
53+
* Edit POSM Element
54+
55+
Authenticated User With PG/Workspace Association
56+
* Edit POSM Element
57+
58+
### What this backend actually enforces (vs. the matrix above)
59+
60+
The matrix above is the intended product model. It is only **partially**
61+
enforced in `api/` — this service is a proxy in front of the OSM website,
62+
cgimap, and TDEI, so several capabilities are enforced downstream (or not yet
63+
at all). Validated against the code:
64+
65+
**Enforced here, Lead-gated (`isWorkspaceLead` → 403).** POC inherits these
66+
(POC on the owning project group satisfies `isWorkspaceLead`):
67+
68+
| Capability | Endpoint |
69+
|---|---|
70+
| Edit Metadata | PATCH `/workspaces/{id}` |
71+
| Edit Longform Quests | PATCH `/workspaces/{id}/quests/long/settings` |
72+
| Toggle App-Enabled Flag | PATCH `/workspaces/{id}` (`externalAppAccess`) |
73+
| Delete Workspace | DELETE `/workspaces/{id}` |
74+
| Define User Teams | `/workspaces/{id}/teams...` (create/update/delete/members) |
75+
| Define Groups or Roles | PUT/DELETE `/workspaces/{id}/users/{user_id}...` |
76+
77+
**Not enforced / not present here:**
78+
79+
* **Export to TDEI** — no endpoint exists in this backend.
80+
* **Move Workspace PG→PG** — not possible; `WorkspacePatch` has no
81+
`tdeiProjectGroupId` field, so no route can change a workspace's project group.
82+
* **Validate Changeset** and **Edit POSM Element** — these go through the OSM
83+
proxy catch-all (`api/main.py`), which gates *every* proxied operation on
84+
`isWorkspaceContributor` alone. There is no Validator- or Lead-level check on
85+
proxied traffic.
86+
87+
**The Validator role grants nothing extra at this layer.**
88+
`isWorkspaceValidator` exists in `api/core/security.py` but no endpoint
89+
authorizes on it — it only appears in the `role` field of `WorkspaceResponse`.
90+
A Validator and a Contributor have identical permissions in this backend.
91+
92+
**"Contributor" and "Authenticated User With PG/Workspace Association" are the
93+
same gate.** `isWorkspaceContributor` simply checks whether the workspace is in
94+
one of the user's project groups (`accessibleWorkspaceIds`), i.e. PG/workspace
95+
association — so both rows collapse to the same check.
96+
97+
If the Validator/Lead distinctions for changeset validation and TDEI export are
98+
required, they must be enforced downstream (`workspaces-openstreetmap-website/`,
99+
`workspaces-cgimap/`) — that has not been audited here.
100+
101+
## Testing
102+
103+
Two layers, both fast and dependency-free (no Postgres, PostGIS, Docker, or
104+
network). `tests/README.md` has the full reference; the essentials:
105+
106+
* **Unit** (`tests/unit/`) — pure logic and individual classes (permission
107+
rules, schema/DTO behavior, a repository in isolation).
108+
* **Integration** (`tests/integration/`) — real HTTP requests driven through
109+
the real FastAPI app: routing, auth wiring, repositories, serialization.
110+
111+
### The mocking boundary is the "data fetcher", not the repository
112+
113+
Integration tests run the **real** routes and repositories. Only three things
114+
are swapped out, via `app.dependency_overrides` and a fake:
115+
116+
1. `get_task_session` / `get_osm_session` → a `FakeSession` (in
117+
`tests/support/fakes.py`) that returns pre-programmed `FakeResult`s instead
118+
of running SQL. This is the data-fetcher boundary: everything above the
119+
`AsyncSession` runs for real.
120+
2. `validate_token` → a real `UserInfo` built by `tests/support/factories.py`
121+
(skips JWT decode + the TDEI call; the permission logic is still real).
122+
3. `api.main._osm_client` → a streamable mock transport (proxy tests only;
123+
`tests/support/http.py`).
124+
125+
Because the mock is at the session level, **queue results in the order the
126+
repository issues queries**. Routes that touch both DBs queue on both
127+
`task_session` and `osm_session`. Builders: `rows()`, `empty()`, `affected(n)`,
128+
`mappings()`, `scalar(v)`, and `raises(exc)` (drives 500 paths). The
129+
`error_client` fixture turns unhandled exceptions into 500 responses (httpx's
130+
ASGI transport re-raises by default).
131+
132+
### `@test:` comment outlines
133+
134+
Modules carry `# @test:` comments describing intended coverage. They are the
135+
spec for the test suite; when adding behavior, add matching `@test:` lines and
136+
tests. Treat the docstring/attribute comments as authoritative when they and
137+
the code disagree — file a fix rather than silently matching the code.
138+
139+
### Known behavior discrepancy: read endpoints return 404, not 403
140+
141+
Several `@test:` comments on read endpoints (get workspace, list teams, quest
142+
and imagery GETs) specify a **403** when the caller lacks access. The code
143+
enforces access via `WorkspaceRepository.getById`, which raises **404
144+
NotFound** when the workspace is missing *or* inaccessible — so "not a member"
145+
currently surfaces as 404 on those routes. The tests assert the actual 404
146+
behavior and flag this in their docstrings. If 403 is the intended contract,
147+
that is a code change in the read routes, not a test change.
148+
149+
### SQLModel + Pyright
150+
151+
SQLModel declares columns as plain annotations (e.g. `id: int | None`) rather
152+
than `Mapped[int]`, so Pyright reads `Column == value` as `bool` and flags
153+
`where()`/`exec()`/`select()`/`selectinload` calls and `result.rowcount`. These
154+
are framework false positives. Suppress them with **targeted, inline**
155+
`# pyright: ignore[<rule>]` comments at the specific offending call sites (e.g.
156+
`# pyright: ignore[reportArgumentType]` on a `.where(Column == value)` line) —
157+
not blanket file-level `# pyright:` directives, which would hide genuine errors
158+
of those rules elsewhere in the file. Note Black may wrap a long query line and
159+
move a trailing comment off the flagged line; place the ignore on the line
160+
Pyright actually reports (often the inner `== value` line) so it survives
161+
formatting. Keep `api/` and `tests/` at zero Pyright errors.
162+
163+
### Alembic enum migrations
164+
165+
Postgres `ENUM` types must be created/dropped idempotently. Declare the enum
166+
with `create_type=False` and manage it explicitly with
167+
`enum.create(op.get_bind(), checkfirst=True)` / `enum.drop(..., checkfirst=True)`
168+
so a migration is safe whether or not the type already exists (and never
169+
double-creates it via implicit table DDL).
170+

README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,23 @@ uv sync
1414
uv run uvicorn api.main:app
1515
```
1616

17+
## Running the tests
18+
19+
Tests are fast and require no database, Docker, or network (see
20+
`tests/README.md` for the design, and `CLAUDE.md` for conventions).
21+
22+
```
23+
uv run pytest # full suite with coverage (configured in pyproject.toml)
24+
uv run pytest --no-cov -q # quick run, no coverage
25+
uv run pytest tests/unit # unit tests only
26+
uv run pytest tests/integration # integration tests only
27+
uv run pytest -k workspaces # filter by keyword
28+
```
29+
30+
Type-check and format (matches the pre-commit hooks):
31+
32+
```
33+
uvx pyright --pythonpath .venv/bin/python api tests
34+
uv run black api tests && uv run isort api tests
35+
```
36+

0 commit comments

Comments
 (0)