Commit c975a72
authored
fix(kiloclaw): reverse region list on capacity recovery (#1059)
## Summary
- **Fix capacity recovery region fallback**: `replaceStrandedVolume`
used `deprioritizeRegion` to push the failed region to the back of the
list, but this compared the concrete Fly region (e.g. `ord`) against
meta-regions (`us`, `eu`) — which never matched. The recovery volume
landed back in the same exhausted region, causing the retry
`createNewMachine` to fail with the same capacity error. Axiom logs
confirmed a **76% failure rate** (136/178) on capacity recovery
attempts, with replacement volumes landing in `ord` 113/178 times. Fix:
reverse the configured region list on recovery so the opposite
geographic region is always tried first.
## Verification
- [x] `pnpm typecheck` — pass
- [x] `pnpm test` — 537 tests pass (28 test files)
- [x] Verified via Axiom log analysis that the capacity recovery was
firing but replacement volumes were landing back in the same region
(`ord` 113/178 times)
- [ ] Additional verification after deploy
## Visual Changes
N/A
## Reviewer Notes
- The `deprioritizeRegion` function and its tests are kept (still
exported) but no longer used in the recovery path. It could be cleaned
up separately if desired.
- Initial provision and `ensureVolume` still use `shuffleRegions` for
randomized first-attempt region selection — only the capacity recovery
path is changed to deterministic reverse order.
- With `FLY_REGION=us,eu`, recovery now always tries `[eu, us]`,
ensuring the replacement volume targets the opposite continent first.2 files changed
Lines changed: 11 additions & 13 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2128 | 2128 | | |
2129 | 2129 | | |
2130 | 2130 | | |
2131 | | - | |
| 2131 | + | |
2132 | 2132 | | |
2133 | 2133 | | |
2134 | 2134 | | |
| |||
2150 | 2150 | | |
2151 | 2151 | | |
2152 | 2152 | | |
2153 | | - | |
| 2153 | + | |
2154 | 2154 | | |
2155 | 2155 | | |
2156 | 2156 | | |
2157 | 2157 | | |
2158 | 2158 | | |
2159 | 2159 | | |
2160 | | - | |
2161 | | - | |
2162 | | - | |
| 2160 | + | |
| 2161 | + | |
2163 | 2162 | | |
2164 | 2163 | | |
2165 | 2164 | | |
| |||
2197 | 2196 | | |
2198 | 2197 | | |
2199 | 2198 | | |
2200 | | - | |
| 2199 | + | |
2201 | 2200 | | |
2202 | 2201 | | |
2203 | 2202 | | |
2204 | 2203 | | |
2205 | 2204 | | |
2206 | 2205 | | |
2207 | 2206 | | |
2208 | | - | |
2209 | | - | |
| 2207 | + | |
| 2208 | + | |
2210 | 2209 | | |
2211 | 2210 | | |
2212 | 2211 | | |
| |||
2270 | 2269 | | |
2271 | 2270 | | |
2272 | 2271 | | |
2273 | | - | |
2274 | | - | |
| 2272 | + | |
| 2273 | + | |
2275 | 2274 | | |
2276 | 2275 | | |
2277 | 2276 | | |
| |||
Lines changed: 2 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
68 | | - | |
69 | | - | |
| 68 | + | |
70 | 69 | | |
71 | 70 | | |
72 | 71 | | |
| |||
0 commit comments