Add script to bootstrap SQLite & Mongo from files

Signed-off-by: Allen Rose <RoseAllenM@gmail.com>

Author: RoseAllenM

QUICKSTART.md

@@ -29,7 +29,7 @@ cp example.docker-compose.local.vexa.yml docker-compose.local.vexa.yml
 
 Edit `docker-compose.local.yml` with your credentials.
 
-**Production tracking (ShotGrid):** To run without a ShotGrid seat, set **`PRODTRACK_PROVIDER=mock`** in `docker-compose.local.yml`. The mock provider uses read-only SQLite with pre-seeded data. To use real ShotGrid, set `PRODTRACK_PROVIDER=shotgrid` (or leave it unset) and add `SHOTGRID_URL`, `SHOTGRID_SCRIPT_NAME`, and `SHOTGRID_API_KEY`. See [Mock setup](#mock-production-tracking-setup) below for how to refresh or customize the mock data.
+**Production tracking (ShotGrid):** To run without a ShotGrid seat, set **`PRODTRACK_PROVIDER=mock`** in `docker-compose.local.yml`. Set **`MOCK_PRODTRACK_DB_PATH=src/dna/prodtrack_providers/mock_data/mock.db`** to make the active mock DB explicit and easy to change later. The mock provider uses read-only SQLite with pre-seeded data. To use real ShotGrid, set `PRODTRACK_PROVIDER=shotgrid` (or leave it unset) and add `SHOTGRID_URL`, `SHOTGRID_SCRIPT_NAME`, and `SHOTGRID_API_KEY`. See [Mock setup](#mock-production-tracking-setup) below for how to refresh or customize the mock data.
 
 **LLM provider:** Set `LLM_PROVIDER` to choose which backend LLM integration to use.
 
@@ -140,6 +140,7 @@ The React app will be available at `http://localhost:5173`.
 | `SHOTGRID_API_KEY` | Yes\* | - | ShotGrid API key (required when using ShotGrid) |
 | `SHOTGRID_SCRIPT_NAME` | Yes\* | - | ShotGrid script name (required when using ShotGrid) |
 | `PRODTRACK_PROVIDER` | No | `shotgrid` | `shotgrid` or `mock`; set to `mock` to use the read-only mock DB without ShotGrid |
+| `MOCK_PRODTRACK_DB_PATH` | No | bundled `mock.db` | Path to the SQLite DB used when `PRODTRACK_PROVIDER=mock` |
 | `MONGODB_URL` | No | `mongodb://mongo:27017` | MongoDB connection string |
 | `STORAGE_PROVIDER` | No | `mongodb` | Storage provider type |
 | `VEXA_API_KEY` | Yes | - | API key for Vexa transcription service |
@@ -257,12 +258,13 @@ The DNA API serves as the central hub:
 
 ## Mock Production Tracking Setup
 
-When you set **`PRODTRACK_PROVIDER=mock`**, the backend uses a read-only mock provider backed by SQLite (`backend/src/dna/prodtrack_providers/mock_data/mock.db`). The app runs normally with this data so you can develop and test the UI without a ShotGrid seat.
+When you set **`PRODTRACK_PROVIDER=mock`**, the backend uses a read-only mock provider backed by SQLite. By default, set **`MOCK_PRODTRACK_DB_PATH=src/dna/prodtrack_providers/mock_data/mock.db`** in `docker-compose.local.yml` so the active DB is explicit and easy to change. The app runs normally with this data so you can develop and test the UI without a ShotGrid seat.
 
 ### Using the mock provider
 
-- In `docker-compose.local.yml`, set **`PRODTRACK_PROVIDER=mock`**. You do not need to set any ShotGrid variables when using the mock.
+- In `docker-compose.local.yml`, set **`PRODTRACK_PROVIDER=mock`** and **`MOCK_PRODTRACK_DB_PATH=src/dna/prodtrack_providers/mock_data/mock.db`**.
 - The mock provider is used only when explicitly set; there is no automatic fallback if ShotGrid credentials are missing.
+- To switch the backend to a bootstrapped local DB, change `MOCK_PRODTRACK_DB_PATH` to `/app/.local/mock.db`.
 
 ### Refreshing or customizing mock data from ShotGrid
 
@@ -283,10 +285,34 @@ docker-compose -f docker-compose.yml -f docker-compose.local.yml run --rm api \
   --api-key 'YOUR_API_KEY'
 ```
 
-- This overwrites `mock_data/mock.db` with entities (projects, users, shots, assets, tasks, versions, playlists, notes) from the given ShotGrid project.
+- To bootstrap sample review data, first start Mongo, then run `bootstrap_dataset`.
+- By default, `bootstrap_dataset` writes to `backend/.local/mock.db`, so it does not modify the checked-in mock DB.
+- If you want the backend to use that bootstrapped DB, set `MOCK_PRODTRACK_DB_PATH=/app/.local/mock.db` in `docker-compose.local.yml`.
+- `seed-mock-db` overwrites `mock_data/mock.db` with entities (projects, users, shots, assets, tasks, versions, playlists, notes) from the given ShotGrid project.
 - Use `--skip-thumbnails` to skip downloading version thumbnails (faster seed; thumbnails will not work after signed URLs expire).
 - Without `--skip-thumbnails`, thumbnails are downloaded to `mock_data/thumbnails/` and served by the API at `/api/mock-thumbnails/{version_id}` so they keep working after ShotGrid signed URLs expire.
 
+Example sample bootstrap workflow:
+
+```bash
+cd backend
+
+# Start only Mongo first
+make start-mongo
+
+# See available arguments
+python -m dna.devtools.bootstrap_dataset --help
+
+# Preview the import without writing anything
+python -m dna.devtools.bootstrap_dataset ../sample_dailies_dataset --dry-run
+
+# Seed SQLite + Mongo using the default local SQLite output
+python -m dna.devtools.bootstrap_dataset ../sample_dailies_dataset
+
+# Then start the full backend stack so the API comes up with the seeded data
+make start-local
+```
+
 The mock provider is **read-only**: it does not write to ShotGrid or to the SQLite file at runtime. Writes such as publishing notes will raise an error when using the mock provider.
 
 ## Docker Compose Files

backend/.gitignore

@@ -42,3 +42,4 @@ backend/docker-compose.vexa.yml
 # Local environment files
 docker-compose.local.yml
 docker-compose.local.vexa.yml
+.local/

backend/docker-compose.yml

@@ -29,6 +29,7 @@ services:
       - ./pytest.ini:/app/pytest.ini
       - ./.coveragerc:/app/.coveragerc
       - ./htmlcov:/app/htmlcov
+      - ./.local:/app/.local
     environment:
       - PYTHONUNBUFFERED=1
       - SHOTGRID_URL=https://your-shotgrid-url.com

backend/example.docker-compose.local.yml

@@ -1,6 +1,8 @@
 # Local development override. Copy to docker-compose.local.yml and fill in secrets.
 # Auth: AUTH_PROVIDER=none uses the noop provider (sign in with any email; no token validation).
 # Prodtrack: Set PRODTRACK_PROVIDER=mock to use the read-only mock (no ShotGrid needed).
+# Mock DB path: change MOCK_PRODTRACK_DB_PATH to /app/.local/mock.db if you
+# want the backend to use a locally bootstrapped DB instead of the checked-in fixture.
 # To use real ShotGrid, set PRODTRACK_PROVIDER=shotgrid and SHOTGRID_URL, SHOTGRID_SCRIPT_NAME, SHOTGRID_API_KEY.
 services:
   api:
@@ -10,6 +12,7 @@ services:
       - SHOTGRID_API_KEY=************
       - SHOTGRID_SCRIPT_NAME=DNA_local_testing
       - PRODTRACK_PROVIDER=mock
+      - MOCK_PRODTRACK_DB_PATH=src/dna/prodtrack_providers/mock_data/mock.db
       - VEXA_API_KEY=**********
       - VEXA_API_URL=http://vexa:8056
       - OPENAI_API_KEY=your-openai-api-key

backend/makefile

@@ -44,4 +44,7 @@ format-python: venv-lint
 	.venv-lint/bin/isort src/ tests/
 
 seed-mock-db:
-	$(DOCKER_COMPOSE) -f docker-compose.yml -f docker-compose.local.yml run --rm api python -m dna.prodtrack_providers.mock_data.seed_db --project-id 124 --url https://aswf.shotgrid.autodesk.com --script-name DNA_local_testing --api-key '$(SHOTGRID_API_KEY)'
+	$(DOCKER_COMPOSE) -f docker-compose.yml -f docker-compose.local.yml run --rm api python -m dna.prodtrack_providers.mock_data.seed_db --project-id 124 --url https://aswf.shotgrid.autodesk.com --script-name DNA_local_testing --api-key '$(SHOTGRID_API_KEY)'
+
+start-mongo:
+	$(DOCKER_COMPOSE) -f docker-compose.yml -f docker-compose.local.yml up -d mongo

backend/src/dna/devtools/__init__.py

@@ -0,0 +1 @@
+"""Developer tooling utilities for backend workflows."""