Skip to content

Commit 4a9aaab

Browse files
committed
BF: fix for gdi_request_limit and release note update
1 parent b41d82c commit 4a9aaab

8 files changed

Lines changed: 90 additions & 1375 deletions

File tree

doc/markdown/man/man5/sge_conf.md

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1128,13 +1128,12 @@ Each section of the limit rule that is separated by a colon allows you to filter
11281128

11291129
The following request characteristics can be used:
11301130

1131-
- *source* : The source of the request. Possible values are names of command line tools names such as *qsub*, *qstat*, *qalter*. *qconf*, *qhost*, ... that send GDI requests.
1132-
- *type* : The type of the request can be either *ADD*, *MOD*, *DEL* or *GET* for object specific CRUD operations. Trigger requests that are intended to trigger a specific action in the qmaster (such as startup/shutdown actions) are interpreted as *MOD* requests.
1131+
- *source* : The source of the request. Possible values are names of command line tools such as *qsub*, *qstat*, *qalter*, *qconf*, *qhost*, ... that send GDI requests.
1132+
- *type* : The type of the request can be either *ADD*, *MOD*, *DEL* or *GET* for object specific CRUD operations. Trigger requests that are intended to trigger a specific action in the qmaster (such as startup/shutdown actions) are interpreted as *MOD* requests. Stored procedure requests are interpreted as *GET* requests.
11331133
- *object* : The object of the request. Possible values are
11341134
- *JOB* : Job
11351135
- *CQUEUE* : Cluster Queue
11361136
- *AHOST*, *EHOST*, *SHOST* : Admin, Execution, Submission Host
1137-
- *SHARETREE* : Share Tree
11381137
- *ECLIENT* : Event Clients
11391138
- *CPLX* : Complex
11401139
- *CONF* : Configuration
@@ -1150,28 +1149,32 @@ The following request characteristics can be used:
11501149
- *HGROUP* : Host Group
11511150
- *RQS* : Resource Quota Set
11521151
- *AR* : Advance Reservation
1153-
- *host* : The name of the host (or host group) from which the request is being sent.
1152+
- *PROC* : Procedure requests for any object.
11541153
- *user* : The username of the user sending the request or the name of a user list.
1154+
- *host* : The name of the host (or host group) from which the request is being sent.
11551155

11561156
For all filter sections, fmatch(1) patterns are allowed as long as they do not contain a colon character or an equal sign.
11571157

1158-
If *host* and/or *user* specifies a host or user group, the limit applies to all hosts or users in the group combined. Also in case of patterns, the limit applies to all matching objects combined.
1158+
If *host* and/or *user* specifies a host or user group, the limit applies to all hosts or users in the group combined. Also, in the case of patterns, the limit applies to all matching objects combined.
11591159
Only the first matching rule is used to decide whether to accept the request. Therefore, more specific rules should be placed before more general rules.
11601160

1161-
Note that command line clients may send multiple GDI requests to the xxqs_name_sxx_qmaster(8) process for a single command call. Reducing a limit below a certain threshold (e.g. 15 for qstat) may lead to scenarios where the command line tool is no longer able to trigger GDI requests anymore for regular users without manager privileges.
1161+
GDI request limits are not respected for manager requests. This means that users with manager privileges can bypass these limits and issue as many GDI requests as they want.
11621162

11631163
To enforce the limits, time is divided into 1-second intervals. The algorithm tracks the number of requests in the current second and the previous second. Each time a request arrives, the number of requests within a sliding 1-second window is calculated. This window includes the requests from the current second and a proportionate number of requests from the previous second. The difference between the number of requests in this window and the limit determines whether a new request can be accepted or rejected. The algorithm assumes that requests were evenly distributed in the previous second to prevent spikes at the start of a new interval.
11641164

11651165
For example:
11661166

11671167
gdi_request_limits=*:add:job:john:*=500,
11681168
*:add:job:*:*=50,
1169-
qstat:get:*:*:*=60000
1169+
qstat:*:proc:*:*=4000
11701170

11711171
The first rule allows user *john* to submit 500 jobs per second. The second rule allows all (remaining) users to post 50 jobs per second. Both rules are submit client independent. This means that users can use any submit client (qsub, qrsh, DRMAA client or GUI) to submit jobs. Any attempt to submit more jobs will be rejected and the user will see the limit rule that has been violated.
11721172

1173-
The third rule allows the *qstat* commands to query the status via *qstat* for 60000 requests per second. Please note that one *qstat* command can trigger multiple GDI requests at once. For example, *qstat -f* will query up to 15 different objects (job, queue, ehost, cplx etc.) in one command. Therefore, the limit should be set to a value that is high enough to allow users to get all the information they need in one command, or in other words, the
1174-
limit of 60000 get requests will allow about 4000 *qstat -f* commands per second or 60000 *qstat -j* commands per second.
1173+
The third rule limits the number of *qstat* status queries to 4000 requests per second. Note that starting with Gridware Cluster Scheduler 9.2, *qstat* no longer issues multiple individual GDI GET requests to collect job, queue, and host information. Instead, it sends a single stored procedure request that triggers a stored procedure on the qmaster to gather all required data. As a result, each *qstat* invocation corresponds to exactly one GDI request, and the limit value maps directly to the maximum number of *qstat* commands per second without any multiplier.
1174+
1175+
*qhost*, *qquota* and *qrstat* also trigger a single *GET_PROCEDURE* request for each command call, but they are not limited by the *gdi_request_limits* setting in the example above.
1176+
1177+
This setting is available in Gridware Cluster Scheduler only.
11751178

11761179
## binding_params
11771180

doc/markdown/manual/release-notes/02_os_matrix.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,9 @@ the symbols/letter used in the table.
3333
| CentOS Linux | 9 | x86-64 | x | x | x |
3434
| Free BSD | 13 | x86-64 | a | x | x |
3535
| Free BSD | 14 | x86-64 | a | x | x |
36+
| Oracle Linux | 8 | x86-64 | x | x | x |
37+
| Oracle Linux | 9 | x86-64 | x | x | x |
38+
| Oracle Linux | 10 | x86-64 | x | x | x |
3639
| Redhat Linux | 7 | x86-64 | a | x | x |
3740
| Redhat Linux | 8 | x86-64 | x | x | x |
3841
| Redhat Linux | 9 | x86-64 | x | x | x |

0 commit comments

Comments
 (0)