|
| 1 | +[中文](../zh/overload_control_filter_limiter.md) |
| 2 | + |
| 3 | +# Overview |
| 4 | + |
| 5 | +In the tRPC-Cpp framework, under the `Fiber (M:N)` thread model (refer to the [Fiber User Guide](./fiber_user_guide.md)), when resource exhaustion occurs in high-concurrency scenarios, it can lead to program freezing. Therefore, it is necessary to impose limits on the number of Fibers to prevent program freezing. This article primarily introduces an overload protection plugin based on the number of Fibers in concurrency. |
| 6 | + |
| 7 | +# Principle |
| 8 | + |
| 9 | +## Principle diagram of overload protection based on the number of Fibers in concurrency |
| 10 | + |
| 11 | + |
| 12 | +The core point in the diagram is the `FiberLimiter`, which is primarily responsible for comparing the current number of concurrent Fibers (`curr_fiber_count`) in the framework with the maximum configured number of concurrent Fibers (`max_fiber_count`). Its main purpose is to determine whether to reject processing the current request (`curr_req`). |
| 13 | + |
| 14 | +## Implementation code |
| 15 | + |
| 16 | +The current `FiberLimiter` supports **both the client and the server**. Since the framework's overload protection is implemented based on filters (refer to [filter]((./filter.md))), in order to ensure flow control protection as early as possible, this filter is placed before the request is enqueued. The specific implementation of this filter is as follows: |
| 17 | + |
| 18 | +```cpp |
| 19 | +// Client side |
| 20 | +std::vector<FilterPoint> FiberLimiterClientFilter::GetFilterPoint() { |
| 21 | + return { |
| 22 | + FilterPoint::CLIENT_PRE_SEND_MSG, |
| 23 | + ... |
| 24 | + }; |
| 25 | +} |
| 26 | + |
| 27 | +// Server side |
| 28 | +std::vector<FilterPoint> FiberLimiterServerFilter::GetFilterPoint() { |
| 29 | + return { |
| 30 | + FilterPoint::SERVER_PRE_SCHED_RECV_MSG, |
| 31 | + ... |
| 32 | + }; |
| 33 | +} |
| 34 | +``` |
| 35 | + |
| 36 | +Source code reference: [fiber_limiter](../../trpc/overload_control/fiber_limiter/) |
| 37 | + |
| 38 | +# Usage example |
| 39 | + |
| 40 | +The overload protection filter based on the number of Fibers in concurrency supports both the client and the server. It must be used in the Fiber thread model. Users do not need to modify any code; they only need to modify the configuration, making it very convenient to use. |
| 41 | + |
| 42 | +## Compilation options |
| 43 | + |
| 44 | +Add the following line to the `.bazelrc` file. |
| 45 | + |
| 46 | +```sh |
| 47 | +build --define trpc_include_overload_control=true |
| 48 | +``` |
| 49 | + |
| 50 | +## Configuration file |
| 51 | + |
| 52 | +The configuration for the Fiber concurrency filter is as follows (for detailed configuration, refer to [fibers_overload_ctrl.yaml](../../trpc/overload_control/fiber_limiter/fibers_overload_ctrl.yaml)) |
| 53 | + |
| 54 | +```yaml |
| 55 | +#Global configuration (required) |
| 56 | +global: |
| 57 | + local_ip: 0.0.0.0 #Local IP, used for: not affecting the normal operation of the framework, used to obtain the local IP from the framework configuration. |
| 58 | + coroutine: #Coroutine configuration. |
| 59 | + enable: true #false: means not using coroutine; true: means using coroutine. |
| 60 | + threadmodel: |
| 61 | + fiber: |
| 62 | + - instance_name: fiber_instance |
| 63 | + concurrency_hint: 1 |
| 64 | + scheduling_group_size: 1 |
| 65 | + reactor_num_per_scheduling_group: 1 |
| 66 | + |
| 67 | +#Server configuration |
| 68 | +server: |
| 69 | + app: test #Business name |
| 70 | + server: route #Module name of the business |
| 71 | + admin_port: 18888 # Admin port |
| 72 | + admin_ip: 0.0.0.0 #Admin ip |
| 73 | + service: #Business service, can have multiple. |
| 74 | + - name: trpc.test.route.Forward #Service name, needs to be filled in according to the format, the first field is default to trpc, the second and third fields are the app and server configurations above, and the fourth field is the user-defined service_name. |
| 75 | + network: tcp #Network listening type: for example: TCP, UDP. |
| 76 | + ip: 0.0.0.0 #Listen ip |
| 77 | + port: 11112 #Listen port |
| 78 | + protocol: trpc #Service application layer protocol, for example: trpc, http. |
| 79 | + accept_thread_num: 1 #Number of threads for binding ports. |
| 80 | + filter: |
| 81 | + - fiber_limiter # Name of filter for this service |
| 82 | + |
| 83 | +#Client configuration. |
| 84 | +client: |
| 85 | + service: |
| 86 | + - name: trpc.test.helloworld.Greeter |
| 87 | + target: 127.0.0.1:32345 |
| 88 | + protocol: trpc #Service application layer protocol, for example: trpc, http. |
| 89 | + network: tcp #Network listening type: for example: TCP, UDP. |
| 90 | + selector_name: direct #Name service used for route selection, "direct" for direct connection. |
| 91 | + filter: |
| 92 | + - fiber_limiter # Name of filter for this client |
| 93 | + |
| 94 | +#Plugin configuration. |
| 95 | +plugins: |
| 96 | +# metrics: |
| 97 | +# prometheus: |
| 98 | + # ... |
| 99 | + overload_control: |
| 100 | + fiber_limiter: |
| 101 | + max_fiber_count: 20 # Maximum number of fibers. Here, for unit testing purposes, it is set relatively small. In normal business scenarios, this value should be much larger than 20. |
| 102 | + is_report: true # Whether to report overload information. |
| 103 | + # ... |
| 104 | +``` |
| 105 | + |
| 106 | +The key points of the configuration are as follows： |
| 107 | + |
| 108 | +- fiber_limiter: The name of the overload protection for Fiber concurrency. |
| 109 | +- max_fiber_count: The maximum number of concurrent requests configured by the user. When the current concurrent requests are greater than or equal to this value, the requests will be intercepted. |
| 110 | +- is_report: Whether to report monitoring data to the monitoring plugin. **Note that this configuration must be used together with a monitoring plugin (for example, configuring plugins->metrics->prometheus will report to Prometheus). If no monitoring plugin is configured, this option is meaningless**. The monitored data includes: |
| 111 | + - `max_fiber_count`: Reports the maximum Fiber concurrency configured by the user. It is a fixed value used to check if the configured maximum request concurrency is effective in the program. |
| 112 | + - `fibers_count`: Reports the current number of concurrent Fibers in the system. |
| 113 | + - `/{callee_name}/{method}`: The format for reporting RPC method monitoring name. It is a fixed value and consists of the callee service (callee_name) and the method name (method). For example: /trpc.test.helloworld.Greeter/SayHello. This format is **only effective for server-side rate limiting**. |
| 114 | + - `/{ip}/{port}`: The format for reporting monitoring items based on the IP and port of the callee service. For example: /127.0.0.1/22345. This format is **only effective for client-side rate limiting**. In the case of a client, when there are multiple nodes for the downstream service, it is not possible to differentiate the monitoring information for different nodes based on the callee service RPC. Therefore, the monitoring item is based on the combination of IP and port. |
| 115 | + - `Pass`：The pass status for an individual request: 0 means intercepted, and 1 means passed. |
| 116 | + - `Limited`：The interception status for an individual request: 1 means intercepted, and 0 means passed. It is the opposite of the `Pass` monitoring attribute mentioned above. |
| 117 | + |
| 118 | +# FAQ |
| 119 | + |
| 120 | +## Why is the Fiber overload protection filter not effective even after configuration? |
| 121 | + |
| 122 | +Check if the Fiber thread model is being used. |
| 123 | + |
| 124 | +## What are differences between overload protectors based on concurrent requests and overload protectors based on Fiber concurrency count? |
| 125 | + |
| 126 | +In the tRPC-Cpp framework, the Fiber scheduling group is a global configuration. Therefore, in a relay scenario, whether acting as a client or a server, the same Fiber concurrency count can be obtained, allowing for rate limiting. However, the overload protector based on concurrent requests calculates the current concurrent requests based on the received requests. It is not reasonable to use this value when accessing backend nodes as a client. Therefore, the Fiber concurrency count overload protector supports both server-side and client-side, while the request concurrency overload protector is only applied to the server-side. |
| 127 | + |
| 128 | +## Why is the concurrency still limited even when the number of concurrent requests is less than the maximum parallelism of Fiber? |
| 129 | + |
| 130 | +The number of concurrent requests is not equivalent to the current parallelism of Fiber because some requests may be processed using multiple Fibers. |
0 commit comments