Skip to content

Commit b95d69b

Browse files
committed
RHINENG-26546: move dev files into one folder
1 parent 719fdd6 commit b95d69b

6 files changed

Lines changed: 16 additions & 16 deletions

File tree

conf/workspace_backfill.env

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
LOG_LEVEL=info
22

33
# Local Docker only: 1000 rows per job run (re-run job until complete). Production uses 50000 via WORKSPACE_BACKFILL_CONFIG.
4-
POD_CONFIG=update_users;update_db_config;wait_for_db=empty;use_testing_db;workspace_backfill_batch_size=1000;workspace_backfill_max_rows_per_run=1000
4+
POD_CONFIG=update_users;update_db_config;wait_for_db=empty;use_testing_db;workspace_backfill_batch_size=1000;workspace_backfill_max_rows_per_run=10000
File renamed without changes.
File renamed without changes.
File renamed without changes.

dev/workspace_backfill.md renamed to dev/workspace_backfill/workspace_backfill.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,19 @@ Backfill copies `workspaces` JSON into `workspace_id` and `workspace_name` on `s
88

99
| File | Purpose |
1010
|------|---------|
11-
| [`docker-compose.workspace-backfill.yml`](../docker-compose.workspace-backfill.yml) | `db` + optional one-shot e2e runner |
12-
| [`conf/workspace_backfill.env`](../conf/workspace_backfill.env) | Local Docker `POD_CONFIG` (**1000** rows per job run) |
13-
| [`dev/test_generate_system_inventory.sql`](test_generate_system_inventory.sql) | Fast load: accounts + inventory + patch only |
14-
| [`dev/prepare_workspace_backfill_test.sql`](prepare_workspace_backfill_test.sql) | Clear `workspace_id` / `workspace_name` without triggers |
15-
| [`dev/verify_workspace_backfill.sql`](verify_workspace_backfill.sql) | Check pending / mismatched rows |
16-
| [`scripts/workspace_backfill_e2e.sh`](../scripts/workspace_backfill_e2e.sh) | Automated pipeline: setup + **one** job run + verify |
11+
| [`docker-compose.workspace-backfill.yml`](../../docker-compose.workspace-backfill.yml) | `db` + optional one-shot e2e runner |
12+
| [`conf/workspace_backfill.env`](../../conf/workspace_backfill.env) | Local Docker `POD_CONFIG` (**1000** rows per job run) |
13+
| [`test_generate_system_inventory.sql`](test_generate_system_inventory.sql) | Fast load: accounts + inventory + patch only |
14+
| [`prepare_workspace_backfill_test.sql`](prepare_workspace_backfill_test.sql) | Clear `workspace_id` / `workspace_name` without triggers |
15+
| [`verify_workspace_backfill.sql`](verify_workspace_backfill.sql) | Check pending / mismatched rows |
16+
| [`scripts/workspace_backfill_e2e.sh`](../../scripts/workspace_backfill_e2e.sh) | Automated pipeline: setup + **one** job run + verify |
1717

1818
## Configuration (`POD_CONFIG`)
1919

20-
| Key | Local Docker ([`conf/workspace_backfill.env`](../conf/workspace_backfill.env)) | Code / production default |
20+
| Key | Local Docker ([`conf/workspace_backfill.env`](../../conf/workspace_backfill.env)) | Code / production default |
2121
|-----|----------------------------------------------------------------------------------|---------------------------|
2222
| `workspace_backfill_batch_size` | `1000` | `1000` |
23-
| `workspace_backfill_max_rows_per_run` | **`1000`** | **`50000`** ([`tasks/config.go`](../tasks/config.go), [`deploy/clowdapp.yaml`](../deploy/clowdapp.yaml)) |
23+
| `workspace_backfill_max_rows_per_run` | **`1000`** | **`50000`** ([`tasks/config.go`](../../tasks/config.go), [`deploy/clowdapp.yaml`](../../deploy/clowdapp.yaml)) |
2424
| `workspace_backfill_batch_sleep_ms` | `0` | `0` |
2525

2626
Local compose loads `workspace_backfill.env` so each job invocation updates at most **1000** rows. Re-run the job manually until logs say `Workspace backfill complete`.
@@ -56,8 +56,8 @@ unset WAIT_FOR_EMPTY_DB
5656
export WAIT_FOR_FULL_DB=1
5757
./dev/scripts/wait-for-services.sh true
5858
unset WAIT_FOR_FULL_DB
59-
PGPASSWORD=passwd psql -h db -p 5432 -U admin -d patchman -f dev/test_generate_system_inventory.sql
60-
PGPASSWORD=passwd psql -h db -p 5432 -U admin -d patchman -f dev/prepare_workspace_backfill_test.sql
59+
PGPASSWORD=passwd psql -h db -p 5432 -U admin -d patchman -f dev/workspace_backfill/test_generate_system_inventory.sql
60+
PGPASSWORD=passwd psql -h db -p 5432 -U admin -d patchman -f dev/workspace_backfill/prepare_workspace_backfill_test.sql
6161
'
6262
```
6363

@@ -87,7 +87,7 @@ PGPASSWORD=passwd psql -h localhost -p 5433 -U admin -d patchman -c \
8787
Verify when `pending` is 0:
8888

8989
```bash
90-
PGPASSWORD=passwd psql -h localhost -p 5433 -U admin -d patchman -f dev/verify_workspace_backfill.sql
90+
PGPASSWORD=passwd psql -h localhost -p 5433 -U admin -d patchman -f dev/workspace_backfill/verify_workspace_backfill.sql
9191
```
9292

9393
Expect `pending = 0` and `mismatched = 0`.
@@ -124,4 +124,4 @@ The job connects as the **admin** user (`core.ConfigureAdminApp()`), same creden
124124

125125
## Production
126126

127-
CronJob `workspace-backfill` in [`deploy/clowdapp.yaml`](../deploy/clowdapp.yaml) uses `WORKSPACE_BACKFILL_CONFIG` with `workspace_backfill_max_rows_per_run=50000` (suspended by default). Run on a schedule until pending rows are gone. Admin DB credentials are provided by the platform (Clowder), not component secrets.
127+
CronJob `workspace-backfill` in [`deploy/clowdapp.yaml`](../../deploy/clowdapp.yaml) uses `WORKSPACE_BACKFILL_CONFIG` with `workspace_backfill_max_rows_per_run=50000` (suspended by default). Run on a schedule until pending rows are gone. Admin DB credentials are provided by the platform (Clowder), not component secrets.

scripts/workspace_backfill_e2e.sh

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,16 +28,16 @@ export WAIT_FOR_FULL_DB=1
2828
unset WAIT_FOR_FULL_DB
2929

3030
echo "==> Loading system_inventory test data"
31-
psql_admin -f dev/test_generate_system_inventory.sql
31+
psql_admin -f dev/workspace_backfill/test_generate_system_inventory.sql
3232

3333
echo "==> Clearing workspace_id / workspace_name (triggers disabled per transaction)"
34-
psql_admin -f dev/prepare_workspace_backfill_test.sql
34+
psql_admin -f dev/workspace_backfill/prepare_workspace_backfill_test.sql
3535

3636
echo "==> Running workspace_backfill job"
3737
./scripts/entrypoint.sh job workspace_backfill
3838

3939
echo "==> Verifying backfill"
40-
mapfile -t _wb_counts < <(psql_admin -t -A -f dev/verify_workspace_backfill.sql)
40+
mapfile -t _wb_counts < <(psql_admin -t -A -f dev/workspace_backfill/verify_workspace_backfill.sql)
4141
pending="${_wb_counts[0]}"
4242
mismatched="${_wb_counts[1]}"
4343

0 commit comments

Comments
 (0)