Fix rate limiter on number of requests & connections

[mariacarmina]

Fixes #981 .

Changes proposed in this PR:

• implement logic regarding rate limiter for no. of requests per minute
• implement logic regarding rate limiter for no. of global connections per minute
• integrate for HTTP and P2P protocols
• keep only one interface of CommonValidation

[mariacarmina]

Tested with MAX_REQ_PER_MINUTE=3 and MAX_CONNECTIONS_PER_MINUTE=1:

• when more than 3 requests are sent to node, this is the output:

❯ date; curl <node_url>/api/services/computeEnvironments
Thu Jul 17 16:16:13 EEST 2025
[{"id":"0x3f648280afd2f64d470fe3b252bdb0df4c24db1faaf43303fc71a61f859fdad1-0x6d8aa2ca9189dda9984f15c6175de1012816fc760daf78b678dfd18264b01d4e","runningJobs":0,"consumerAddress":"0xe2DD09d719Da89e5a3D0F2549c7E24566e947260","platform":{"architecture":"x86_64","os":"Ubuntu 24.04.2 LTS"},"fees":{"8996":[{"prices":[{"id":"cpu","price":1},{"id":"myGPU","price":1}],"feeToken":"0x2473f4F7bf40ed9310838edFCA6262C17A59DF64"}]},"storageExpiry":604800,"maxJobDuration":7200,"resources":[{"id":"cpu","total":4,"max":4,"min":1,"inUse":0},{"id":"ram","total":27311357952,"max":27311357952,"min":1000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"max":1,"min":0,"inUse":0},{"id":"disk","total":21000000000,"max":21000000000,"min":0,"inUse":0}],"free":{"maxJobDuration":7200,"maxJobs":3,"resources":[{"id":"cpu","max":1,"inUse":0},{"id":"ram","max":21000000000,"inUse":0},{"id":"disk","max":21000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"inUse":0}]},"runningfreeJobs":0}]%                                                                                                                                                                                     ❯ date; curl <node_url>/api/services/computeEnvironments
Thu Jul 17 16:16:14 EEST 2025
[{"id":"0x3f648280afd2f64d470fe3b252bdb0df4c24db1faaf43303fc71a61f859fdad1-0x6d8aa2ca9189dda9984f15c6175de1012816fc760daf78b678dfd18264b01d4e","runningJobs":0,"consumerAddress":"0xe2DD09d719Da89e5a3D0F2549c7E24566e947260","platform":{"architecture":"x86_64","os":"Ubuntu 24.04.2 LTS"},"fees":{"8996":[{"prices":[{"id":"cpu","price":1},{"id":"myGPU","price":1}],"feeToken":"0x2473f4F7bf40ed9310838edFCA6262C17A59DF64"}]},"storageExpiry":604800,"maxJobDuration":7200,"resources":[{"id":"cpu","total":4,"max":4,"min":1,"inUse":0},{"id":"ram","total":27311357952,"max":27311357952,"min":1000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"max":1,"min":0,"inUse":0},{"id":"disk","total":21000000000,"max":21000000000,"min":0,"inUse":0}],"free":{"maxJobDuration":7200,"maxJobs":3,"resources":[{"id":"cpu","max":1,"inUse":0},{"id":"ram","max":21000000000,"inUse":0},{"id":"disk","max":21000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"inUse":0}]},"runningfreeJobs":0}]%                                                                                                                                                                                     ❯ date; curl <node_url>/api/services/computeEnvironments
Thu Jul 17 16:16:15 EEST 2025
[{"id":"0x3f648280afd2f64d470fe3b252bdb0df4c24db1faaf43303fc71a61f859fdad1-0x6d8aa2ca9189dda9984f15c6175de1012816fc760daf78b678dfd18264b01d4e","runningJobs":0,"consumerAddress":"0xe2DD09d719Da89e5a3D0F2549c7E24566e947260","platform":{"architecture":"x86_64","os":"Ubuntu 24.04.2 LTS"},"fees":{"8996":[{"prices":[{"id":"cpu","price":1},{"id":"myGPU","price":1}],"feeToken":"0x2473f4F7bf40ed9310838edFCA6262C17A59DF64"}]},"storageExpiry":604800,"maxJobDuration":7200,"resources":[{"id":"cpu","total":4,"max":4,"min":1,"inUse":0},{"id":"ram","total":27311357952,"max":27311357952,"min":1000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"max":1,"min":0,"inUse":0},{"id":"disk","total":21000000000,"max":21000000000,"min":0,"inUse":0}],"free":{"maxJobDuration":7200,"maxJobs":3,"resources":[{"id":"cpu","max":1,"inUse":0},{"id":"ram","max":21000000000,"inUse":0},{"id":"disk","max":21000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"inUse":0}]},"runningfreeJobs":0}]%                                                                                                                                                                                     ❯ date; <node_url>/api/services/computeEnvironments
Thu Jul 17 16:16:16 EEST 2025
Rate limit exceeded. Try again in 58 seconds.%                                                                                                                                                                                                         ❯ date; curl http://3.c2d.nodes.oceanprotocol.com:8000/api/services/computeEnvironments
Thu Jul 17 16:16:17 EEST 2025
Rate limit exceeded. Try again in 56 seconds.%

• when number of connections exceeded MAX_CONNECTIONS:

❯ date; curl <node_url>/api/services/computeEnvironments
Thu Jul 17 18:18:14 EEST 2025
Too many active connections (2/1) in the last minute.%                                                                                                                                                                                                 ❯ date; curl <node_url>/api/services/computeEnvironments
Thu Jul 17 18:18:15 EEST 2025
Too many active connections (2/1) in the last minute.%

After 2 mins:

❯ date; curl <node_url>/api/services/computeEnvironments
Thu Jul 17 18:20:27 EEST 2025
[{"id":"0x3f648280afd2f64d470fe3b252bdb0df4c24db1faaf43303fc71a61f859fdad1-0x6d8aa2ca9189dda9984f15c6175de1012816fc760daf78b678dfd18264b01d4e","runningJobs":0,"consumerAddress":"0xe2DD09d719Da89e5a3D0F2549c7E24566e947260","platform":{"architecture":"x86_64","os":"Ubuntu 24.04.2 LTS"},"fees":{"8996":[{"prices":[{"id":"cpu","price":1},{"id":"myGPU","price":1}],"feeToken":"0x2473f4F7bf40ed9310838edFCA6262C17A59DF64"}]},"storageExpiry":604800,"maxJobDuration":7200,"resources":[{"id":"cpu","total":4,"max":4,"min":1,"inUse":0},{"id":"ram","total":27311357952,"max":27311357952,"min":1000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"max":1,"min":0,"inUse":0},{"id":"disk","total":21000000000,"max":21000000000,"min":0,"inUse":0}],"free":{"maxJobDuration":7200,"maxJobs":3,"resources":[{"id":"cpu","max":1,"inUse":0},{"id":"ram","max":21000000000,"inUse":0},{"id":"disk","max":21000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"inUse":0}]},"runningfreeJobs":0}]%                                                                                                                                                                                     ❯ date; curl <node_url>/api/services/computeEnvironments
Thu Jul 17 18:20:28 EEST 2025
[{"id":"0x3f648280afd2f64d470fe3b252bdb0df4c24db1faaf43303fc71a61f859fdad1-0x6d8aa2ca9189dda9984f15c6175de1012816fc760daf78b678dfd18264b01d4e","runningJobs":0,"consumerAddress":"0xe2DD09d719Da89e5a3D0F2549c7E24566e947260","platform":{"architecture":"x86_64","os":"Ubuntu 24.04.2 LTS"},"fees":{"8996":[{"prices":[{"id":"cpu","price":1},{"id":"myGPU","price":1}],"feeToken":"0x2473f4F7bf40ed9310838edFCA6262C17A59DF64"}]},"storageExpiry":604800,"maxJobDuration":7200,"resources":[{"id":"cpu","total":4,"max":4,"min":1,"inUse":0},{"id":"ram","total":27311357952,"max":27311357952,"min":1000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"max":1,"min":0,"inUse":0},{"id":"disk","total":21000000000,"max":21000000000,"min":0,"inUse":0}],"free":{"maxJobDuration":7200,"maxJobs":3,"resources":[{"id":"cpu","max":1,"inUse":0},{"id":"ram","max":21000000000,"inUse":0},{"id":"disk","max":21000000000,"inUse":0},{"id":"myGPU","description":"NVIDIA Tesla T4","type":"gpu","total":1,"init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":[""],"Capabilities":[["gpu"]]}},"inUse":0}]},"runningfreeJobs":0}]%

[giurgiur99]

Can we follow this standard and add headers in response? It will help clients know when to retry connections

X-RateLimit-Limit: The maximum number of requests you're permitted to make per hour.
X-RateLimit-Remaining: The number of requests remaining in the current rate limit window.
X-RateLimit-Reset: the time at which the current rate limit window resets in UTC epoch seconds

Ref: https://www.ietf.org/archive/id/draft-polli-ratelimit-headers-02.html

[mariacarmina]

Can we follow this standard and add headers in response? It will help clients know when to retry connections

X-RateLimit-Limit: The maximum number of requests you're permitted to make per hour. X-RateLimit-Remaining: The number of requests remaining in the current rate limit window. X-RateLimit-Reset: the time at which the current rate limit window resets in UTC epoch seconds

Ref: https://www.ietf.org/archive/id/draft-polli-ratelimit-headers-02.html

Hello!
I included in the logs when the user can retry to send requests to node when rate limiter is exceeded.
Also those 2 util functions for checking rate limiting work for both protocol HTTP and P2P, RateLimit headers are specific to the HTTP and are not a native concept in P2P, so I suggest to continue with the PR's code to be compatible with both protocols for now.

[giurgiur99]

LGTM!