RI-8219: Add Array read endpoints + key-info strategy#6064
Conversation
🛡️ Jit Security Scan Results
✅ No security findings were detected in this PR
Security scan by Jit
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe56b30c97
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 76a49d6fee
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Code Coverage - Backend unit tests
Test suite run success3580 tests passing in 317 suites. Report generated by 🧪jest coverage report action from d3cdb3f |
Code Coverage - Integration Tests
|
Expose the seven read-only endpoints for the Redis array data type
(POST /databases/:id/array/{get-range,scan,get-length,get-count,
get-next-index,get-element,get-elements}).
Indexes are handled as decimal strings end-to-end (validated via
IsArrayIndex) so unsigned 64-bit array indexes survive the wire and
controller layer without precision loss. ARGETRANGE enforces a hard
1,000,000-element span guard per the Redis docs; ARSCAN is exempt
because it already skips empty slots.
Adds 33 unit tests covering happy paths, WrongType, missing keys,
and ACL failures.
References: RI-8219
Adds a Bruno collection folder for the new Array endpoints with one preset per route plus: - Seed Sample Data — uploads fixtures/readings.txt via the existing POST /bulk-actions/import endpoint, seeding the sparse 'readings' array (indexes 0, 1, 5) that the happy-path presets target. - Get Element (empty slot) — documents the gap-preserving semantics (200 OK with value: null on an unset index, not 404). Requires Redis 8.8+ on the target database for the AR* commands. References: RI-8219
- redis-string-to-buffer transformer: pass null/undefined through rather than throwing, matching the ASCII/UTF8 sibling transformers. - ARSCAN pairing: accept both flat [idx, val, ...] and nested [[idx, val], ...] reply shapes; drop pairs with null/undefined halves so the populated-only contract is honored end-to-end.
Adds ArrayKeyInfoStrategy that pipelines TTL + ARLEN + ARCOUNT + MEMORY USAGE so the array length and populated count appear in the key details header (mirroring vector-set's vectorDim/quantType). Registers the ar* commands as ioredis built-ins so they can be issued via sendPipeline (sendCommand already handled them). - GetKeyInfoResponse: + count?: number - ArrayKeyInfoStrategy: pipelined fetch with size-skip when length exceeds MAX_KEY_SIZE (mirrors list/vector-set patterns) - KeyInfoProvider / KeysModule: register the new strategy - ioredis client: addBuiltinCommand for arget/armget/arlen/arcount/ argetrange/arscan/arnext
pawelangelow
force-pushed
the
be/RI-8219/add-api-endpoints
branch
from
76a49d6 to
f51748f
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3cf97b588b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f65e26d12f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca96558052
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
ARGETRANGE / ARSCAN require start <= end. Match that contract at the API boundary: validate the order in the service and return 400 if reversed, instead of silently normalizing. Aligns with how every other Redis range command behaves and removes ambiguity about how a reversed reply would be ordered or paginated.
pawelangelow
force-pushed
the
be/RI-8219/add-api-endpoints
branch
from
ca96558 to
9186271
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9186271634
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 74bb22a61a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ccd26c98e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Addresses two review issues on the array read endpoints: - toIndexString silently turned nil replies into the literal strings "null" / "undefined". Only getNextIndex guarded the helper; getLength, getCount, and ArrayKeyInfoStrategy fed nil straight through. Fixed at the helper: toIndexString now returns string | null (passing nil through as null), and a new toRequiredIndexString throws on nil for callers where the upstream key/type check guarantees a value. - GetArrayMultiElementsDto accepted an unbounded indexes array. ARMGET is O(N) per the Redis docs, so a large list with the app's 512MB request body limit could tie up Redis. Capped at the same 1,000,000 per-call limit ARGETRANGE/ARSCAN use, with the same 400 error path. Bruno Get Elements doc updated to reflect the new cap. New unit tests cover the nil pass-through (the exact regression codex flagged) and the strict variant's throw-on-nil contract.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 42ddc13ebe
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Covers the seven Array read endpoints landed in PR #6064: - POST /array/get-range (ARGETRANGE) - POST /array/scan (ARSCAN) - POST /array/get-length (ARLEN) - POST /array/get-count (ARCOUNT) - POST /array/get-next-index (ARNEXT) - POST /array/get-element (ARGET) - POST /array/get-elements (ARMGET) Locks in on the wire format the unit specs can't reach: - decimal-string contract for length/count/scan-index/next-index (Joi.string().pattern(/^\d+$/) plus a u64 boundary case at index 9223372036854775818 to catch silent numeric coercion); - gap-preserving semantics — empty slots surface as JSON null in get-range / get-element / get-elements (and are skipped by scan); - RedisStringToBufferTransformer null passthrough — encoding=buffer on an empty slot must stay JSON null, not a zero-length Buffer; - explicit limit:null on scan is treated as omitted (regression guard for the `typeof limit === 'number'` gate); - 400 cases hardened in PR #6064: ARRAY_RANGE_REVERSED, range > 1M (ARRAY_RANGE_TOO_LARGE), ARMGET @ArrayMinSize(1) and the @ArrayMaxSize(ARRAY_RANGE_MAX_ELEMENTS) cap, non-decimal and out-of-u64 indexes; - WrongType, missing key, missing instance per endpoint; - per-command ACL denials (-arget, -arlen, -arcount, -arnext, -armget, -argetrange, -arscan) gated by requirements('rte.acl'). Also extends keys/POST-databases-id-keys-get_info.test.ts with an Array describe block (gated by rte.version>=8.8) that seeds a dense and a sparse Array key and pins the new GetArrayKeyInfoResponse oneOf branch — length === count for dense, length !== count for sparse, both as decimal strings.
Covers the seven Array read endpoints landed in PR #6064: - POST /array/get-range (ARGETRANGE) - POST /array/scan (ARSCAN) - POST /array/get-length (ARLEN) - POST /array/get-count (ARCOUNT) - POST /array/get-next-index (ARNEXT) - POST /array/get-element (ARGET) - POST /array/get-elements (ARMGET) Locks in on the wire format the unit specs can't reach: - decimal-string contract for length/count/scan-index/next-index (Joi.string().pattern(/^\d+$/) plus a u64 boundary case at index 9223372036854775818 to catch silent numeric coercion); - gap-preserving semantics — empty slots surface as JSON null in get-range / get-element / get-elements (and are skipped by scan); - RedisStringToBufferTransformer null passthrough — encoding=buffer on an empty slot must stay JSON null, not a zero-length Buffer; - explicit limit:null on scan is treated as omitted (regression guard for the `typeof limit === 'number'` gate); - 400 cases hardened in PR #6064: ARRAY_RANGE_REVERSED, range > 1M (ARRAY_RANGE_TOO_LARGE), ARMGET @ArrayMinSize(1) and the @ArrayMaxSize(ARRAY_RANGE_MAX_ELEMENTS) cap, non-decimal and out-of-u64 indexes; - WrongType, missing key, missing instance per endpoint; - per-command ACL denials (-arget, -arlen, -arcount, -arnext, -armget, -argetrange, -arscan) gated by requirements('rte.acl'). Also extends keys/POST-databases-id-keys-get_info.test.ts with an Array describe block (gated by rte.version>=8.8) that seeds a dense and a sparse Array key and pins the new GetArrayKeyInfoResponse oneOf branch — length === count for dense, length !== count for sparse, both as decimal strings.
| hasLimit ? [...baseArgs, 'LIMIT', limit] : [...baseArgs], | ||
| )) as unknown[]; | ||
|
|
||
| // ARSCAN's wire shape varies by client: some clients surface a flat |
There was a problem hiding this comment.
is this correct? from the docs - https://redis.io/docs/latest/commands/arscan/
it seems to me like only the flat array structure is returned? I might be missing something
There was a problem hiding this comment.
Good catch - I had to verify empirically because the docs and the wire don't agree. Tested against redis:8.8-alpine (same image as CI's oss-st-8 #6078):
RESP wire (source of truth) - redis-cli with --no-raw so it doesn't collapse nested arrays:
$ printf 'ARSCAN k 0 100\r\n' | redis-cli --no-raw
1) 1) (integer) 0
2) "a"
2) 1) (integer) 1
2) "b"
3) 1) (integer) 10
2) "x"
That's *N of *2 — nested at the protocol level. Same on RESP2 and RESP3 (redis-cli -3). The plain redis-cli ARSCAN ... output in the docs (each value on its own line) is just redis-cli's default flat-print rendering — it walks any nested-array reply and prints each leaf on its own line, which makes it look flat in the "Examples" section.
ioredis (what our service sees) — c.sendCommand(new Command('ARSCAN', [key, '0', '100'])):
[ [ 0, <Buffer 61> ], [ 1, <Buffer 62> ], [ 10, <Buffer 78> ] ]
Also nested. That matched the CI failure exactly — the flat parser was building index: '0,20.1' strings from String(['0','20.1']) and value was undefined on every odd pair, which is why 6 of the scan tests collapsed to 1 element.
The Return-info section of the docs (Array reply: Flat array of index-value pairs) is wrong/outdated.
ARSCAN's per-command page documents start > end as reverse iteration, not invalid input. Split the range guard so getRange still rejects reversed ranges (ARGETRANGE doesn't support them) but scan only enforces the abs-span cap and forwards start/end unchanged.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2ccfdf31ba
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Code Coverage - Frontend unit tests
Test suite run success7182 tests passing in 816 suites. Report generated by 🧪jest coverage report action from d3cdb3f |
Empirical probe against Redis 8.8.0 confirms ARGETRANGE returns elements in reverse index order when start > end (matching ARSCAN and the /data-types/arrays page). Drop the pre-flight reversal rejection so both endpoints share a single span-cap validator and forward start/end to Redis unchanged.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79b21819e3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Covers the seven Array read endpoints landed in PR #6064: - POST /array/get-range (ARGETRANGE) - POST /array/scan (ARSCAN) - POST /array/get-length (ARLEN) - POST /array/get-count (ARCOUNT) - POST /array/get-next-index (ARNEXT) - POST /array/get-element (ARGET) - POST /array/get-elements (ARMGET) Locks in on the wire format the unit specs can't reach: - decimal-string contract for length/count/scan-index/next-index (Joi.string().pattern(/^\d+$/) plus a u64 boundary case at index 9223372036854775818 to catch silent numeric coercion); - gap-preserving semantics — empty slots surface as JSON null in get-range / get-element / get-elements (and are skipped by scan); - RedisStringToBufferTransformer null passthrough — encoding=buffer on an empty slot must stay JSON null, not a zero-length Buffer; - explicit limit:null on scan is treated as omitted (regression guard for the `typeof limit === 'number'` gate); - 400 cases hardened in PR #6064: ARRAY_RANGE_REVERSED, range > 1M (ARRAY_RANGE_TOO_LARGE), ARMGET @ArrayMinSize(1) and the @ArrayMaxSize(ARRAY_RANGE_MAX_ELEMENTS) cap, non-decimal and out-of-u64 indexes; - WrongType, missing key, missing instance per endpoint; - per-command ACL denials (-arget, -arlen, -arcount, -arnext, -armget, -argetrange, -arscan) gated by requirements('rte.acl'). Also extends keys/POST-databases-id-keys-get_info.test.ts with an Array describe block (gated by rte.version>=8.8) that seeds a dense and a sparse Array key and pins the new GetArrayKeyInfoResponse oneOf branch — length === count for dense, length !== count for sparse, both as decimal strings.
Redis 8.8 returns ARSCAN results as [[index, value], ...] nested entries, while some earlier builds surface a flat [index, value, ...] reply. Detect the shape by sniffing the first element and normalize both into the same populated-only contract.
Covers the seven Array read endpoints landed in PR #6064: - POST /array/get-range (ARGETRANGE) - POST /array/scan (ARSCAN) - POST /array/get-length (ARLEN) - POST /array/get-count (ARCOUNT) - POST /array/get-next-index (ARNEXT) - POST /array/get-element (ARGET) - POST /array/get-elements (ARMGET) Locks in on the wire format the unit specs can't reach: - decimal-string contract for length/count/scan-index/next-index (Joi.string().pattern(/^\d+$/) plus a u64 boundary case at index 9223372036854775818 to catch silent numeric coercion); - gap-preserving semantics — empty slots surface as JSON null in get-range / get-element / get-elements (and are skipped by scan); - RedisStringToBufferTransformer null passthrough — encoding=buffer on an empty slot must stay JSON null, not a zero-length Buffer; - explicit limit:null on scan is treated as omitted (regression guard for the `typeof limit === 'number'` gate); - 400 cases hardened in PR #6064: ARRAY_RANGE_REVERSED, range > 1M (ARRAY_RANGE_TOO_LARGE), ARMGET @ArrayMinSize(1) and the @ArrayMaxSize(ARRAY_RANGE_MAX_ELEMENTS) cap, non-decimal and out-of-u64 indexes; - WrongType, missing key, missing instance per endpoint; - per-command ACL denials (-arget, -arlen, -arcount, -arnext, -armget, -argetrange, -arscan) gated by requirements('rte.acl'). Also extends keys/POST-databases-id-keys-get_info.test.ts with an Array describe block (gated by rte.version>=8.8) that seeds a dense and a sparse Array key and pins the new GetArrayKeyInfoResponse oneOf branch — length === count for dense, length !== count for sparse, both as decimal strings.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit fd4df8d. Configure here.
Covers the seven Array read endpoints landed in PR #6064: - POST /array/get-range (ARGETRANGE) - POST /array/scan (ARSCAN) - POST /array/get-length (ARLEN) - POST /array/get-count (ARCOUNT) - POST /array/get-next-index (ARNEXT) - POST /array/get-element (ARGET) - POST /array/get-elements (ARMGET) Locks in on the wire format the unit specs can't reach: - decimal-string contract for length/count/scan-index/next-index (Joi.string().pattern(/^\d+$/) plus a u64 boundary case at index 9223372036854775818 to catch silent numeric coercion); - gap-preserving semantics — empty slots surface as JSON null in get-range / get-element / get-elements (and are skipped by scan); - RedisStringToBufferTransformer null passthrough — encoding=buffer on an empty slot must stay JSON null, not a zero-length Buffer; - explicit limit:null on scan is treated as omitted (regression guard for the `typeof limit === 'number'` gate); - 400 cases hardened in PR #6064: ARRAY_RANGE_REVERSED, range > 1M (ARRAY_RANGE_TOO_LARGE), ARMGET @ArrayMinSize(1) and the @ArrayMaxSize(ARRAY_RANGE_MAX_ELEMENTS) cap, non-decimal and out-of-u64 indexes; - WrongType, missing key, missing instance per endpoint; - per-command ACL denials (-arget, -arlen, -arcount, -arnext, -armget, -argetrange, -arscan) gated by requirements('rte.acl'). Also extends keys/POST-databases-id-keys-get_info.test.ts with an Array describe block (gated by rte.version>=8.8) that seeds a dense and a sparse Array key and pins the new GetArrayKeyInfoResponse oneOf branch — length === count for dense, length !== count for sparse, both as decimal strings.
2 participants
What
Backend slice of the Array data type read vertical (RI-8219).
ARGETRANGEcap: 1,000,000-element span guard so astart..endrange can't be used to pull arbitrary memory;ARSCANis exempt because it already skips empty slots.ARSCANreply normalisation: handles both the flat[idx, val, idx, val, …]and nested[[idx, val], …]shapes some clients surface, and drops malformed pairs soJSON.stringifynever reaches the client as{ index }with no value.ArrayKeyInfoStrategyfor the key-info pipeline: returns the standard TTL / size triple pluslength(ARLEN, total addressable slots including gaps) andcount(ARCOUNT, populated slots only). The two diverge for sparse arrays and the View tab surfaces both.Testing
Seed Sample Datarequest that uploadsfixtures/readings.txtthrough the existing bulk-import endpoint, populating the sparsereadingsarray (indexes 0, 1, 5) that the happy-path presets target.Get Element (empty slot)documents the gap-preserving semantics (200 OK withvalue: null, not 404).Note
Medium Risk
Large new browser surface against Redis with a validation breaking change on max index; mitigated by caps, tests, and read-only paths with standard ACL/404 handling.
Overview
Adds Browser Array read APIs (
get-range,scan,get-length,get-count,get-next-index,get-element,get-elements) wired to RedisAR*commands, with DTOs that return u64 metrics as strings and gap slots asnullon dense reads.Safety and wire behavior:
ARGETRANGErejects spans over 1M elements;ARMGET/ARSCAN LIMITuse the same cap.ARSCANnormalizes flat vs nested Redis replies and drops incomplete pairs.RedisStringToBufferTransformernow passesnull/undefinedthrough so empty slots serialize correctly.Key browser: New
ArrayKeyInfoStrategyexposeslength(ARLEN) andcount(ARCOUNT) as strings;get-infoOpenAPI is aoneOfwithGetArrayKeyInfoResponse. IORedis registers the new array builtins for pipelines.Breaking alignment: Valid array index max is
2^64 − 2(not full u64) in API/UI validators, Bruno create docs, and tests. Bruno presets plus areadingsbulk-import fixture exercise sparse arrays and empty-slot200+nullsemantics.Reviewed by Cursor Bugbot for commit d3cdb3f. Bugbot is set up for automated code reviews on this repo. Configure here.