Commit 18fde26
Refresh worker: fix Workday 0-jobs + Ashby chunk timeouts
Two targeted fixes diagnosed against the live VPS this evening
after the cache layer was activated.
Workday — `_PAGE_SIZE = 50 → 20`
Workday's CXS API silently tightened its per-page limit. Probed
all 11 configured tenants (nvidia.wd5, micron.wd1, hpe.wd5,
citi.wd5, walmart.wd5, hpe.wd5, adobe.wd5, boeing.wd1,
disney.wd5, hp.wd5, blackrock.wd1, workday.wd5):
limit=5/10/20 → 200 OK
limit=50 → 400 with body
{"errorCode":"HTTP_400", "message":""}
The old `_PAGE_SIZE = 50` made the FIRST page POST fail for
every tenant, so `_fetch_board_jobs` raised, fetch_all_postings
yielded ('error', ...) for all 11 boards, and the cache worker
ended up with zero workday postings to upsert. ~13 K Fortune-
500 jobs were silently missing from the index.
20 matches the browser default and is the largest size every
validated tenant currently accepts.
Ashby — per-source chunk_size override
Ashby postings carry significantly larger description bodies
than Greenhouse/Lever/Workday. The cache table's `search_tsv`
is a GENERATED STORED tsvector that gets re-derived on every
insert, and Supabase's default `statement_timeout` for the
`service_role` REST path is 60 s. At chunk_size=100 we observed
five consecutive `canceling statement due to statement timeout`
failures per refresh, silently dropping ~500 rows.
Lowered Ashby's chunk_size to 30; left other sources at 100.
Per refresh: ashby goes from ~18 requests to ~60 but each
finishes in well under 60 s, and every row lands.
Other sources unchanged. The fix only kicks in on the next CI
deploy (which this push triggers — the api container will
recreate from the new image and the next cron tick at HH:30 / :00
will exercise both paths).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 630b6e8 commit 18fde26
2 files changed
Lines changed: 20 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
146 | | - | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
147 | 157 | | |
148 | | - | |
| 158 | + | |
149 | 159 | | |
150 | 160 | | |
151 | 161 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
61 | | - | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
62 | 68 | | |
63 | 69 | | |
64 | 70 | | |
| |||
0 commit comments