Skip to content

Commit ba04dd1

Browse files
cplaursencplaursen
committed
CP-312094: Update rate limiting design document
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
1 parent b43eb5b commit ba04dd1

1 file changed

Lines changed: 81 additions & 30 deletions

File tree

‎doc/content/design/rate_limit.md‎

Lines changed: 81 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
---
2-
title: XAPI Rate Limiting
2+
title: Rate Limiting
33
layout: default
44
design_doc: true
5-
revision: 1
5+
revision: 2
66
status: draft
77
---
88

@@ -11,12 +11,19 @@ status: draft
1111
- [Approach](#approach)
1212
- [Rate limiting](#rate-limiting)
1313
- [Client classification](#client-classification)
14+
- [Statistics](#statistics)
1415
- [API design](#api-design)
16+
- [Caller Datamodel](#caller-datamodel)
17+
- [API functions](#api-functions)
18+
- [Matching semantics](#matching-semantics)
19+
- [Caller lifecycle](#caller-lifecycle)
1520
- [XAPI integration](#xapi-integration)
1621
- [Library](#library)
1722
- [Token bucket](#token-bucket)
1823
- [Rate limit queue](#rate-limit-queue)
1924
- [Client table](#client-table)
25+
- [Operational Description](#operational-description)
26+
- [Pool Member / Multi-Host Considerations](#pool-member-multi-host-considerations)
2027
<!--toc:end-->
2128

2229
## Overview
@@ -103,42 +110,86 @@ The caller classification allows wildcards for any field, though we require
103110
that at least one field be specified. This lets us, for example, combine all
104111
accesses from the xapi python API by specifying the user-agent .
105112

106-
In order to assist with rate limiting, we can store statistics about callers:
107-
- Last request timestamp
113+
A rate limiter can be associated with any number of callers, and the parameters
114+
of the rate limiter can either be derived from the usage patterns of the
115+
callers or selected from a number of preset profiles.
116+
117+
### Statistics
118+
In order to assist with rate limiting, we can store statistics about callers.
119+
We identify two kinds of statistics: volatile and stable. Volatile statistics
120+
change over time without any input, e.g. sliding windows. These will be stored
121+
in RRDs. By contrast, stable statistics only vary at most once per request, and
122+
so are safe to store in the main database.
123+
124+
**Volatile statistics:**
108125
- Tokens used over the last (5 minutes/hour/day).
109-
- Most common API requests.
126+
- Most common requests over the last (5 minutes/hour/day).
110127

111-
A rate limiter can then be attached to a particular caller, and the parameters
112-
of the rate limiter can either be derived from the usage patterns of the caller
113-
or selected from a number of preset profiles.
128+
**Stable statistics:**
129+
- Last request timestamp
114130

115131
## API design
116-
We propose two new datamodels: **Caller** and **Rate_limit**.
132+
133+
### Caller Datamodel
134+
We propose two new datamodel tables: **Caller**, which stores the data associated with each caller, and **Rate limit**, which identifies one or more callers with a rate limiter
117135

118136
Caller:
119137
| Mutability | Name | Type | Description |
120138
| ---------: | ----------- | -------- | ---------------------------------------------------|
121-
| RW | name_label | String | User-assigned label for the caller |
122-
| RO | user_agent | String | User agent of throttled client; use "*" for "any" |
123-
| RO | host_ip | String | IP address of throttled client; use "*" for "any" |
124-
| RO | last_access | DateTime | Last time the caller made a request |
125-
| RO | burst_size | Float | Amount of tokens that can be consumed in one burst; -1 if no rate limit is applied |
126-
| RO | fill_rate | Float | Tokens added to the bucket per second; -1 if no rate limit is applied |
127-
128-
129-
Matching semantics for `Caller` fields:
130-
- `user_agent` and `host_ip` are treated as match patterns, not just stored values.
131-
- The star (`"*"`) is a wildcard for that field (equivalent to “any”).
132-
- A star can be appended to the end of a field to match any string that starts with the prefix. We only allow prefix matching.
133-
- An incoming call is assigned to a `Caller` only if **all non-wildcard fields in that
134-
caller record match** (logical AND across fields).
135-
- Examples:
136-
- `{user_agent = "*", host_ip = "10.1.2.3"}` matches any user-agent from `10.1.2.3`.
137-
- `{user_agent = "Python-xmlrpc/*", host_ip = "*"}` matches any Python
138-
XML-RPC client from any IP.
139-
- `{user_agent = "Python-xmlrpc/3.11", host_ip = "10.1.2.3"}` matches only calls where both fields match.
140-
141-
Rate limits: We provide calls to set and remove a rate limiter. When no rate limiter is set, both parameters are negative.
139+
| RW | name_label | String | User-assigned label for the caller |
140+
| RO | user_agent | String | user agent matching pattern |
141+
| RO | host_ip | String | IP address matching pattern |
142+
| RO | last_access | DateTime | Last time the caller made a request |
143+
| RO | rate_limit | Ref Rate_limit | Associated rate limiter - can be null |
144+
145+
Rate limit:
146+
| Mutability | Name | Type | Description |
147+
|--------: | --------------|-----------|------------------|
148+
| RW | name_label | String | User-assigned label for the rate limiter |
149+
| SRO | callers | Set (Ref Caller) | Callers associated with this rate limiter |
150+
| RO | burst_size | Float | Amount of tokens that can be consumed in one burst |
151+
| RO | fill_rate | Float | Tokens added to the bucket per second |
152+
153+
### API functions
154+
We define the following API functions on the caller datamodel:
155+
- `Caller.create(name_label, user_agent, host_ip)`: Create a new caller.
156+
- `Caller.set_name_label(caller, name_label)`: Set name label on the caller
157+
- `Caller.destroy(caller)`: Destroy the caller
158+
159+
And the following functions for the rate limiter datamodel:
160+
- `Rate_limit.create(name_label, callers, burst_size, fill_rate)`: Create a
161+
rate limiter with the supplied parameters.
162+
- `Rate_limit.add_caller(rate_limit, caller)`: Add a caller to the callers set
163+
- `Rate_limit.remove_caller(rate_limit, caller)`: Remove a caller from the callers set
164+
- `Rate_limit.set_burst_size(rate_limit, burst_size)`: Set the burst size for the
165+
rate limiter
166+
- `Rate_limit.set_fill_rate(rate_limit, burst_size)`: Set the fill rate for the
167+
rate limiter
168+
- `Rate_limit.destroy(rate_limit)`: Destroy the rate limiter
169+
170+
171+
### Matching semantics
172+
When a request arrives, xapi matches the request's metadata against the
173+
`Caller` table. Requests match a caller record iff all the request fields match the
174+
corresponding patterns in the record.
175+
176+
We treat logging and rate limiting differently:
177+
- **Logging**: All caller records that match with an incoming request track the
178+
request.
179+
- **Rate limiting**: Only the most specific match (which has rate limiting
180+
enabled) for any given request will trigger rate limiting and deduct tokens
181+
from its token bucket.
182+
183+
This results in the statistics being stored by callers tracking everything that
184+
matches, but only the most specific rate limiter to any given request will be
185+
triggered.
186+
187+
### Caller lifecycle
188+
- When a call comes in, a new Caller is automatically created if no existing record matches.
189+
- Users can proactively create callers with wildcards to pre-configure rate limiting rules.
190+
- Toolstack startup behavior: On toolstack startup, an in-memory data structure
191+
is created from the database fields which stores all the rate limiters and
192+
callers, as described in the implementation section.
142193

143194
## XAPI integration
144195
Calls into xapi are intercepted at two points:

0 commit comments

Comments
 (0)