Skip to content

Commit 24b724c

Browse files
yhl25vigith
yhl25
and
vigith
authored
chore: e2e tests for distributed throttling (numaproj#2926)
Signed-off-by: Yashash H L <yashashhl25@gmail.com> Signed-off-by: Vigith Maurice <vigith@gmail.com> Co-authored-by: Vigith Maurice <vigith@gmail.com>
1 parent 3914905 commit 24b724c

34 files changed

Lines changed: 557 additions & 179 deletions

‎api/json-schema/schema.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21445,7 +21445,7 @@
2144521445
},
2144621446
"rateLimit": {
2144721447
"$ref": "#/definitions/io.numaproj.numaflow.v1alpha1.RateLimit",
21448-
"description": "RateLimit is used to define the rate limit for all the vertices in the pipeline, it could be overridden by the vertex's limit settings. It will be applied to all non-source vertices. This is because for source vertices, the rate limit is defined by how many times the `Read` is called per second multiplied by the `readBatchSize`. Reduce does not support RateLimit."
21448+
"description": "RateLimit is used to define the rate limit for all the vertices in the pipeline, it could be overridden by the vertex's limit settings. For source vertices, it will be set to rate divided by readBatchSize because for source vertices, the rate limit is defined by how many times the `Read` is called per second Reduce does not support RateLimit."
2144921449
},
2145021450
"readBatchSize": {
2145121451
"description": "Read batch size for all the vertices in the pipeline, can be overridden by the vertex's limit settings.",

‎api/openapi-spec/swagger.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21431,7 +21431,7 @@
2143121431
"format": "int64"
2143221432
},
2143321433
"rateLimit": {
21434-
"description": "RateLimit is used to define the rate limit for all the vertices in the pipeline, it could be overridden by the vertex's limit settings. It will be applied to all non-source vertices. This is because for source vertices, the rate limit is defined by how many times the `Read` is called per second multiplied by the `readBatchSize`. Reduce does not support RateLimit.",
21434+
"description": "RateLimit is used to define the rate limit for all the vertices in the pipeline, it could be overridden by the vertex's limit settings. For source vertices, it will be set to rate divided by readBatchSize because for source vertices, the rate limit is defined by how many times the `Read` is called per second Reduce does not support RateLimit.",
2143521435
"$ref": "#/definitions/io.numaproj.numaflow.v1alpha1.RateLimit"
2143621436
},
2143721437
"readBatchSize": {

‎config/base/crds/full/numaflow.numaproj.io_monovertices.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2549,7 +2549,6 @@ spec:
25492549
rateLimit:
25502550
properties:
25512551
max:
2552-
default: 10
25532552
format: int64
25542553
type: integer
25552554
min:

‎config/base/crds/full/numaflow.numaproj.io_pipelines.yaml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,6 @@ spec:
155155
rateLimit:
156156
properties:
157157
max:
158-
default: 10
159158
format: int64
160159
type: integer
161160
min:
@@ -7203,7 +7202,6 @@ spec:
72037202
rateLimit:
72047203
properties:
72057204
max:
7206-
default: 10
72077205
format: int64
72087206
type: integer
72097207
min:

‎config/base/crds/full/numaflow.numaproj.io_servingpipelines.yaml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,6 @@ spec:
134134
rateLimit:
135135
properties:
136136
max:
137-
default: 10
138137
format: int64
139138
type: integer
140139
min:
@@ -7182,7 +7181,6 @@ spec:
71827181
rateLimit:
71837182
properties:
71847183
max:
7185-
default: 10
71867184
format: int64
71877185
type: integer
71887186
min:

‎config/base/crds/full/numaflow.numaproj.io_vertices.yaml‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -789,7 +789,6 @@ spec:
789789
rateLimit:
790790
properties:
791791
max:
792-
default: 10
793792
format: int64
794793
type: integer
795794
min:
@@ -1010,7 +1009,6 @@ spec:
10101009
rateLimit:
10111010
properties:
10121011
max:
1013-
default: 10
10141012
format: int64
10151013
type: integer
10161014
min:
@@ -2181,7 +2179,6 @@ spec:
21812179
rateLimit:
21822180
properties:
21832181
max:
2184-
default: 10
21852182
format: int64
21862183
type: integer
21872184
min:
@@ -6063,7 +6060,6 @@ spec:
60636060
rateLimit:
60646061
properties:
60656062
max:
6066-
default: 10
60676063
format: int64
60686064
type: integer
60696065
min:
@@ -6284,7 +6280,6 @@ spec:
62846280
rateLimit:
62856281
properties:
62866282
max:
6287-
default: 10
62886283
format: int64
62896284
type: integer
62906285
min:

‎config/install.yaml‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4075,7 +4075,6 @@ spec:
40754075
rateLimit:
40764076
properties:
40774077
max:
4078-
default: 10
40794078
format: int64
40804079
type: integer
40814080
min:
@@ -8740,7 +8739,6 @@ spec:
87408739
rateLimit:
87418740
properties:
87428741
max:
8743-
default: 10
87448742
format: int64
87458743
type: integer
87468744
min:
@@ -15788,7 +15786,6 @@ spec:
1578815786
rateLimit:
1578915787
properties:
1579015788
max:
15791-
default: 10
1579215789
format: int64
1579315790
type: integer
1579415791
min:
@@ -21050,7 +21047,6 @@ spec:
2105021047
rateLimit:
2105121048
properties:
2105221049
max:
21053-
default: 10
2105421050
format: int64
2105521051
type: integer
2105621052
min:
@@ -28098,7 +28094,6 @@ spec:
2809828094
rateLimit:
2809928095
properties:
2810028096
max:
28101-
default: 10
2810228097
format: int64
2810328098
type: integer
2810428099
min:
@@ -35176,7 +35171,6 @@ spec:
3517635171
rateLimit:
3517735172
properties:
3517835173
max:
35179-
default: 10
3518035174
format: int64
3518135175
type: integer
3518235176
min:
@@ -35397,7 +35391,6 @@ spec:
3539735391
rateLimit:
3539835392
properties:
3539935393
max:
35400-
default: 10
3540135394
format: int64
3540235395
type: integer
3540335396
min:
@@ -36568,7 +36561,6 @@ spec:
3656836561
rateLimit:
3656936562
properties:
3657036563
max:
36571-
default: 10
3657236564
format: int64
3657336565
type: integer
3657436566
min:
@@ -40450,7 +40442,6 @@ spec:
4045040442
rateLimit:
4045140443
properties:
4045240444
max:
40453-
default: 10
4045440445
format: int64
4045540446
type: integer
4045640447
min:
@@ -40671,7 +40662,6 @@ spec:
4067140662
rateLimit:
4067240663
properties:
4067340664
max:
40674-
default: 10
4067540665
format: int64
4067640666
type: integer
4067740667
min:

‎config/namespace-install.yaml‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4075,7 +4075,6 @@ spec:
40754075
rateLimit:
40764076
properties:
40774077
max:
4078-
default: 10
40794078
format: int64
40804079
type: integer
40814080
min:
@@ -8740,7 +8739,6 @@ spec:
87408739
rateLimit:
87418740
properties:
87428741
max:
8743-
default: 10
87448742
format: int64
87458743
type: integer
87468744
min:
@@ -15788,7 +15786,6 @@ spec:
1578815786
rateLimit:
1578915787
properties:
1579015788
max:
15791-
default: 10
1579215789
format: int64
1579315790
type: integer
1579415791
min:
@@ -21050,7 +21047,6 @@ spec:
2105021047
rateLimit:
2105121048
properties:
2105221049
max:
21053-
default: 10
2105421050
format: int64
2105521051
type: integer
2105621052
min:
@@ -28098,7 +28094,6 @@ spec:
2809828094
rateLimit:
2809928095
properties:
2810028096
max:
28101-
default: 10
2810228097
format: int64
2810328098
type: integer
2810428099
min:
@@ -35176,7 +35171,6 @@ spec:
3517635171
rateLimit:
3517735172
properties:
3517835173
max:
35179-
default: 10
3518035174
format: int64
3518135175
type: integer
3518235176
min:
@@ -35397,7 +35391,6 @@ spec:
3539735391
rateLimit:
3539835392
properties:
3539935393
max:
35400-
default: 10
3540135394
format: int64
3540235395
type: integer
3540335396
min:
@@ -36568,7 +36561,6 @@ spec:
3656836561
rateLimit:
3656936562
properties:
3657036563
max:
36571-
default: 10
3657236564
format: int64
3657336565
type: integer
3657436566
min:
@@ -40450,7 +40442,6 @@ spec:
4045040442
rateLimit:
4045140443
properties:
4045240444
max:
40453-
default: 10
4045440445
format: int64
4045540446
type: integer
4045640447
min:
@@ -40671,7 +40662,6 @@ spec:
4067140662
rateLimit:
4067240663
properties:
4067340664
max:
40674-
default: 10
4067540665
format: int64
4067640666
type: integer
4067740667
min:

‎docs/APIs.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7774,11 +7774,11 @@ the vertex’s limit settings
77747774
<p>
77757775

77767776
RateLimit is used to define the rate limit for all the vertices in the
7777-
pipeline, it could be overridden by the vertex’s limit settings. It will
7778-
be applied to all non-source vertices. This is because for source
7779-
vertices, the rate limit is defined by how many times the
7780-
<code>Read</code> is called per second multiplied by the
7781-
<code>readBatchSize</code>. Reduce does not support RateLimit.
7777+
pipeline, it could be overridden by the vertex’s limit settings. For
7778+
source vertices, it will be set to rate divided by readBatchSize because
7779+
for source vertices, the rate limit is defined by how many times the
7780+
<code>Read</code> is called per second Reduce does not support
7781+
RateLimit.
77827782
</p>
77837783

77847784
</td>

‎pkg/apis/numaflow/v1alpha1/generated.proto‎

Lines changed: 2 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)