Commit 21cf2c9
authored
perf(redis): stream entry-per-key layout for O(new) XREAD (#620)
## Summary
- Store each Redis stream entry under its own key
(`!stream|entry|<userKey><StreamID(16B)>`) with a separate meta key
(`!stream|meta|<userKey>` -> `Length | LastMs | LastSeq`); XREAD /
XRANGE become bounded range scans that unmarshal only the selected
entries, XADD is two small writes, XLEN is a single meta read.
- Migration is dual-read with write-time rewrite: reads bump
`elastickv_stream_legacy_format_reads_total` when they fall through to
the legacy blob; writes convert the legacy blob to the new layout in the
same transaction and delete the blob, so XLEN never double-counts.
- StreamID suffix is binary big-endian `ms || seq` so lex order over
entry keys matches the numeric `(ms, seq)` order the client sees;
parsing the "ms-seq" string form for range bounds is done once up front
to compute the scan bounds.
## Motivation
Incident 2026-04-24: one client doing 11 XREAD/s on a large stream
consumed 14 CPU cores on the leader, starving raft and Lua paths. Per
the CPU profile, `proto.Unmarshal` alone took 59% of 14 cores because
`loadStreamAt` read the entire stream as a single protobuf blob and
`unmarshalStreamValue` re-parsed every entry on every read.
## Design
- Layout: meta key holds a fixed 24-byte binary record (`Length(8) |
LastMs(8) | LastSeq(8)`); entry keys embed the StreamID in big-endian
binary so a prefix range scan is enough to serve XRANGE / XREAD without
loading the whole stream. `LastMs` / `LastSeq` track the highest ID ever
assigned, so XADD `*` stays monotonic even after XTRIM removes the
current tail.
- Commands: XADD / XREAD / XRANGE / XREVRANGE / XLEN / XTRIM all
rewritten to use the new layout. XADD with MAXLEN trims the head by
scanning exactly `count` entry keys and emitting Dels in the same txn as
the new Put.
- Migration: reads try the new meta first; on miss, fall back to the
legacy blob and increment the counter. Writes that observe the legacy
blob re-emit every legacy entry as a `!stream|entry|…` put, emit the new
meta, and delete the legacy blob, all in a single coordinator-dispatched
transaction. The legacy blob and the new layout never coexist in a
committed state. `redisStreamKey` is marked Deprecated and kept only for
the dual-read fallback.
## Commands touched
`XADD`, `XREAD`, `XRANGE`, `XREVRANGE`, `XLEN`, `XTRIM`.
`deleteLogicalKeyElems` also learns to clean up the new meta + every
entry key so `DEL` and overwrite paths stay correct.
## Test plan
New tests in `adapter/redis_compat_commands_stream_test.go`:
- [x] `TestRedis_StreamXAddXReadRoundTrip` — XADD then XREAD `0` returns
all entries.
- [x] `TestRedis_StreamXReadLatencyIsConstant` — 10 000 XADDs, then 100
XREADs from `$` must not grow beyond `2 * first + 10ms`. On the old blob
path this failed catastrophically.
- [x] `TestRedis_StreamXTrimMaxLen` — XADD 100, XTRIM MAXLEN=10,
XLEN==10, XRANGE returns the last 10.
- [x] `TestRedis_StreamXRangeBounds` — inclusive / exclusive (`(`)
bounds on both sides, plus XREVRANGE.
- [x] `TestRedis_StreamMigrationFromLegacyBlob` — seed a legacy blob
directly, XREAD bumps the counter, XADD migrates and deletes the blob,
subsequent XREAD hits the new layout and does NOT bump the counter, XLEN
is original + new (no double count), XADD `*` is > the pre-migration
last ID.
- [x] `TestRedis_StreamAutoIDMonotonicAfterTrim` — XTRIM that empties
the stream must not rewind XADD `*`.
`go build ./... && go vet ./... && go test -short ./adapter/...
./store/...` all pass locally.
## Follow-ups (not in this PR)
- Migrate hash / set / zset single-blob fallbacks off the entry-per-key
layouts the same way (Layer 4 design doc already lists these).
- XINFO, XACK, consumer groups — none exist today.
- Delete the `redisStreamKey` helper + legacy blob fallback once
`elastickv_stream_legacy_format_reads_total` stays at zero across
production nodes for long enough to guarantee no surviving legacy
streams.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved stream add semantics with stricter ID validation and
guaranteed monotonic auto-IDs.
* Multikey stream deletions now reliably remove all stream data within
transactions.
* **Bug Fixes**
* Corrected XRANGE/XREVRANGE inclusive/exclusive boundaries and COUNT
clamping.
* Ensured XLEN/XTRIM reflect actual entries and legacy-format remnants
are cleaned on write.
* **Performance**
* Reduced blocked read latency and stabilized BLOCK timeout behavior to
return null on timeout.
* **Tests**
* Added extensive end-to-end and unit tests covering
XADD/XREAD/XTRIM/XRANGE, deletion, and edge cases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->10 files changed
Lines changed: 2572 additions & 132 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1496 | 1496 | | |
1497 | 1497 | | |
1498 | 1498 | | |
| 1499 | + | |
| 1500 | + | |
| 1501 | + | |
| 1502 | + | |
| 1503 | + | |
| 1504 | + | |
| 1505 | + | |
| 1506 | + | |
| 1507 | + | |
1499 | 1508 | | |
1500 | 1509 | | |
1501 | 1510 | | |
| |||
1518 | 1527 | | |
1519 | 1528 | | |
1520 | 1529 | | |
1521 | | - | |
| 1530 | + | |
| 1531 | + | |
| 1532 | + | |
| 1533 | + | |
| 1534 | + | |
| 1535 | + | |
| 1536 | + | |
| 1537 | + | |
| 1538 | + | |
| 1539 | + | |
| 1540 | + | |
| 1541 | + | |
| 1542 | + | |
| 1543 | + | |
| 1544 | + | |
| 1545 | + | |
| 1546 | + | |
| 1547 | + | |
| 1548 | + | |
| 1549 | + | |
| 1550 | + | |
1522 | 1551 | | |
1523 | 1552 | | |
1524 | 1553 | | |
| |||
1664 | 1693 | | |
1665 | 1694 | | |
1666 | 1695 | | |
| 1696 | + | |
| 1697 | + | |
| 1698 | + | |
1667 | 1699 | | |
1668 | 1700 | | |
1669 | 1701 | | |
| 1702 | + | |
| 1703 | + | |
| 1704 | + | |
| 1705 | + | |
| 1706 | + | |
| 1707 | + | |
| 1708 | + | |
| 1709 | + | |
| 1710 | + | |
| 1711 | + | |
| 1712 | + | |
| 1713 | + | |
| 1714 | + | |
1670 | 1715 | | |
1671 | 1716 | | |
1672 | 1717 | | |
| |||
1863 | 1908 | | |
1864 | 1909 | | |
1865 | 1910 | | |
1866 | | - | |
| 1911 | + | |
| 1912 | + | |
| 1913 | + | |
| 1914 | + | |
| 1915 | + | |
| 1916 | + | |
| 1917 | + | |
1867 | 1918 | | |
1868 | 1919 | | |
1869 | 1920 | | |
| |||
1918 | 1969 | | |
1919 | 1970 | | |
1920 | 1971 | | |
1921 | | - | |
| 1972 | + | |
| 1973 | + | |
1922 | 1974 | | |
1923 | 1975 | | |
1924 | 1976 | | |
| |||
2459 | 2511 | | |
2460 | 2512 | | |
2461 | 2513 | | |
2462 | | - | |
| 2514 | + | |
2463 | 2515 | | |
2464 | 2516 | | |
2465 | 2517 | | |
| |||
2473 | 2525 | | |
2474 | 2526 | | |
2475 | 2527 | | |
| 2528 | + | |
| 2529 | + | |
| 2530 | + | |
| 2531 | + | |
| 2532 | + | |
| 2533 | + | |
| 2534 | + | |
| 2535 | + | |
| 2536 | + | |
| 2537 | + | |
| 2538 | + | |
| 2539 | + | |
| 2540 | + | |
2476 | 2541 | | |
2477 | 2542 | | |
2478 | 2543 | | |
| |||
2553 | 2618 | | |
2554 | 2619 | | |
2555 | 2620 | | |
| 2621 | + | |
| 2622 | + | |
| 2623 | + | |
| 2624 | + | |
| 2625 | + | |
| 2626 | + | |
| 2627 | + | |
| 2628 | + | |
| 2629 | + | |
| 2630 | + | |
| 2631 | + | |
| 2632 | + | |
| 2633 | + | |
2556 | 2634 | | |
2557 | 2635 | | |
2558 | 2636 | | |
| 2637 | + | |
2559 | 2638 | | |
2560 | 2639 | | |
2561 | 2640 | | |
| |||
2571 | 2650 | | |
2572 | 2651 | | |
2573 | 2652 | | |
2574 | | - | |
2575 | | - | |
2576 | 2653 | | |
2577 | 2654 | | |
2578 | 2655 | | |
| |||
2795 | 2872 | | |
2796 | 2873 | | |
2797 | 2874 | | |
| 2875 | + | |
| 2876 | + | |
| 2877 | + | |
| 2878 | + | |
| 2879 | + | |
| 2880 | + | |
| 2881 | + | |
| 2882 | + | |
| 2883 | + | |
| 2884 | + | |
| 2885 | + | |
| 2886 | + | |
| 2887 | + | |
| 2888 | + | |
| 2889 | + | |
| 2890 | + | |
| 2891 | + | |
| 2892 | + | |
| 2893 | + | |
| 2894 | + | |
| 2895 | + | |
| 2896 | + | |
| 2897 | + | |
| 2898 | + | |
| 2899 | + | |
| 2900 | + | |
| 2901 | + | |
| 2902 | + | |
| 2903 | + | |
| 2904 | + | |
| 2905 | + | |
| 2906 | + | |
| 2907 | + | |
2798 | 2908 | | |
2799 | 2909 | | |
2800 | 2910 | | |
| |||
2827 | 2937 | | |
2828 | 2938 | | |
2829 | 2939 | | |
2830 | | - | |
2831 | | - | |
2832 | | - | |
2833 | | - | |
2834 | | - | |
2835 | | - | |
2836 | | - | |
| 2940 | + | |
| 2941 | + | |
| 2942 | + | |
| 2943 | + | |
| 2944 | + | |
| 2945 | + | |
| 2946 | + | |
| 2947 | + | |
2837 | 2948 | | |
2838 | 2949 | | |
2839 | 2950 | | |
| |||
0 commit comments