test: mirror app/ layout and rebuild the integration test suite

- Move unit tests into app-mirroring sub-packages (storage/, crates/, etc.) and switch pytest to importlib import mode with pythonpath=".".
- Rewrite the integration test for the new stack
- Update the integration workflow deps

Closes #184

Author: alexhambley

.github/workflows/test_docker.yml

@@ -20,16 +20,15 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
-          pip install pytest requests minio docker
+          pip install pytest requests boto3
 
-      - name: Build Docker Compose Containers
+      - name: Run integration tests (brings up the compose stack)
         run: |
           cp example.env .env
-          docker compose -f docker-compose-develop.yml build 
+          pytest -s -v tests/test_integration.py
 
-      - name: Spin Up Docker Compose and Run Tests
-        run: pytest -s -v tests/test_integration.py
-
-      - name: Ensure that Docker Compose is Shutdown
+      - name: Ensure Docker Compose is shut down
         if: always()
-        run: docker compose down
+        run: >
+          docker compose -f docker-compose-develop.yml -p cratey_integration
+          --profile objectstore down -v || true

app/crates/resolver.py

@@ -44,9 +44,7 @@ def _object_exists(storage: StorageBackend, key: str) -> bool:
         return False
 
 
-def resolve_crate(
-    storage: StorageBackend, crate_id: str, crate_prefix: str
-) -> ResolvedCrate:
+def resolve_crate(storage: StorageBackend, crate_id: str, crate_prefix: str) -> ResolvedCrate:
     """Resolve ``crate_id`` to a concrete crate object.
 
     :raises InvalidCrateId: If the ID is malformed.

docker-compose-develop.yml

@@ -23,6 +23,10 @@ services:
       - PROFILES_PATH=/app/profiles
     depends_on:
       - redis
+    # Metadata validation runs synchronously in this process, so the flask
+    # service needs the custom profiles mounted too (not just the worker).
+    volumes:
+      - ./tests/data/rocrate_validator_profiles:/app/profiles:ro
 
   celery_worker:
     build:

pyproject.toml

@@ -37,6 +37,8 @@ dev = [
 include = ["app*"]
 
 [tool.pytest.ini_options]
+addopts = "--import-mode=importlib"
+pythonpath = ["."]
 log_format = "%(asctime)s %(levelname)s %(message)s"
 log_date_format = "%Y-%m-%d %H:%M:%S"