Skip to content

Commit 0ecf53e

Browse files
vishal-balaclaude
andauthored
ci: pin RediSearch WORKERS to 0 to fix flaky redis:latest tests (#641)
## Motivation Our `redis:latest` CI matrix has been flaking intermittently, and we traced it to a real change in Redis 8.8. That release turned on a multithreaded background executor for search by default — the `WORKERS` setting went from `0` to a nonzero value to align with Valkey. It's a deliberate performance improvement, but it exposed a race: when a document's key or an indexed hash field expires (via TTL or `HPEXPIRE`) at the exact moment an `FT.SEARCH` is running in the background, the server returns the matched document's id with a `nil` in place of its field array instead of dropping the expired doc. Clients that reasonably assume every matched doc carries a field array then break, which is what our suite was hitting at random. This was confirmed on the Redis side: the same reproduction runs cleanly on 8.2/8.4/8.6 and only fails on 8.8, and forcing `WORKERS > 0` on 8.6 reproduces it too. The underlying server bug is being fixed upstream — this change just stabilizes our CI in the meantime. ## Fix We pin the search worker pool back to `0` on the test Redis container, so searches run in the foreground and the expiry race can't occur. The setting lives in the shared `tests/docker-compose.yml` that drives every local and CI run, so it applies uniformly across the matrix. Because `0` is already the default on the pinned 8.2/8.4 images, this is a no-op there and only changes behavior for `redis:latest`. It's also overridable through the `REDIS_SEARCH_WORKERS` environment variable, so we can flip multithreading back on to validate the upstream fix once it lands. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Test-only Docker Compose configuration with no application or production runtime impact. > > **Overview** > **Stabilizes CI** for `redis:latest` by forcing RediSearch to run searches on the main thread instead of Redis 8.8’s default multithreaded background executor. > > In `tests/docker-compose.yml`, `REDIS_ARGS` gains `--search-workers ${REDIS_SEARCH_WORKERS:-0}` alongside the existing persistence flags. That avoids a race where background `FT.SEARCH` can return matched doc IDs with `nil` field arrays when keys or hash fields expire mid-query. Pinned 8.2/8.4 images already default workers to `0`, so behavior is unchanged there; only newer images are affected unless you set `REDIS_SEARCH_WORKERS`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c08c456. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 80fa258 commit 0ecf53e

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

tests/docker-compose.yml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,13 @@ services:
44
ports:
55
- "6379"
66
environment:
7-
- "REDIS_ARGS=--save '' --appendonly no"
7+
# Pin the RediSearch worker pool to 0 (single-threaded, foreground
8+
# execution). Redis 8.8 changed the default from 0 to a multithreaded
9+
# background executor, which surfaces a race where FT.SEARCH can return
10+
# nil fields for docs that expire mid-query -- the source of flaky
11+
# redis:latest CI runs. WORKERS=0 avoids the race and is already the
12+
# default on the pinned 8.2/8.4 images, so this is a no-op there.
13+
- "REDIS_ARGS=--save '' --appendonly no --search-workers ${REDIS_SEARCH_WORKERS:-0}"
814
deploy:
915
replicas: 1
1016
restart_policy:

0 commit comments

Comments
 (0)