Optimized sliding-window

[SomajitDey]

Implemented the sliding-window ratelimiter in a more efficient way, requiring less number of commands. The new command count matrix is:

Cache Result	Algorithm State	Command Count	Command
Hit/Miss	First	3	EVAL, INCRBY, SET
Hit/Miss	Intermediate	2	EVAL, INCRBY
Miss	First Rate-Limited	3	EVAL, INCRBY, SET
Miss	Rate-Limited	2	EVAL, INCRBY
Hit	Rate-Limited	0	utilized cache

So, for most cases (i.e. Intermediate state), the proposed implementation requires half (2/4) of the number of commands required by the existing implementation.

Other comparisons

	Proposed	Existing
Memory/Storage	64-bit: Single integer encodes #requests from current & previous window	2x64-bit: Uses 2 separate integers
Key reuse / Key-space size	First request in every window persists key for next window too. Smaller key space: Key is mapped to identifier only.	Keys don't live much beyond 2 windows. Larger key space: Key is mapped to identifier and timestamp.
Resetting used tokens	Fast [O(1)], requires single command: DEL	Slow [O(N)], requires multiple commands: SCAN, DEL
Stale key TTL	TTL = 2 * window, if last window doesn't see a request (i.e. expires soon when unused)	TTL = 2 * window + 1 second, (i.e. lingers)
Cache miss overhead	Once ratelimit is hit, returns early for all further requests in that window	Executes similar computations for all requests in a window

[vercel]

@SomajitDey is attempting to deploy a commit to the Upstash Team on Vercel.

A member of the Team first needs to authorize it.

[SomajitDey]

@CahidArda Might I request a review of this PR 🙏

If the proposed implementation is unclear from the comments in the script, please let me know. To understand the mathematical equivalence between the proposed and the existing implementations, the following notes might help.

• Whereas the existing implementation uses an ordered pair [requestsInPreviousWindow, requestsInCurrentWindow], which is an element of a tokens X tokens matrix, the proposed implementation linearizes the 2-D pair into a 1-D integer with range [ 0 - tokens X tokens). It actually does more (see below), but this is the basic idea / inspiration it builds on.

• For any given window, the existing implementation stores the ordered requests pair in timestamped variables (actually 2 variables, each containing an element from the pair). On the other hand, the proposed implementation reuses the same variable by storing the window index (currWinIndex) along with its linearized requests pair. Note here that, since requests from only two consecutive windows are needed, we do not actually need to store the complete window index but only currWinIndex % 2, i.e. whether the window is even or odd! The proposed implementation actually stores the 1-D linearized value of the 2 X tokens X tokens ranked ordered triplet: [currWinMod2, requestsInPreviousWindow, requestsInCurrentWindow].

• Once requests, upon increment, becomes >= tokens, the proposed implementation resets the variable type from integer to string, so that subsequent integer increments using INCRBY throws a type-mismatch error. It also expires the variable right after the current window. This effectively blocks the current window for all further requests, helping early detection of its blocked status based on a failed INCRBY. Note that the requests = tokens case needs special attention as the linearization mentioned above only works for 0 <= requests < tokens. To deal with this boundary problem, once the requests count reaches or exceeds tokens, the implementation returns with a success, i.e. lets the request pass, only if requests = tokens, while also blocking the window for further requests with the mentioned type-conversion.

[CahidArda]

Hi @SomajitDey,

Thanks for the PR, I will review and get back to you soon

[SomajitDey]

Hi @SomajitDey,

Thanks for the PR, I will review and get back to you soon

Hello @CahidArda, any update on this PR?