Skip to content

Commit 83ebd10

Browse files
committed
Add runtime io_context_options for scheduler and service tuning
Introduce io_context_options struct with six configurable knobs: max_events_per_poll, inline_budget_initial/max, unassisted_budget, gqcs_timeout_ms, and thread_pool_size. All defaults match existing hardcoded values for zero behavior change on unconfigured contexts.
1 parent adca257 commit 83ebd10

10 files changed

Lines changed: 529 additions & 33 deletions

File tree

doc/modules/ROOT/nav.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
** xref:4.guide/4a.tcp-networking.adoc[TCP/IP Networking]
2424
** xref:4.guide/4b.concurrent-programming.adoc[Concurrent Programming]
2525
** xref:4.guide/4c.io-context.adoc[I/O Context]
26+
*** xref:4.guide/4c2.configuration.adoc[Configuration]
2627
** xref:4.guide/4d.sockets.adoc[Sockets]
2728
** xref:4.guide/4e.tcp-acceptor.adoc[Acceptors]
2829
** xref:4.guide/4f.endpoints.adoc[Endpoints]
Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
= Configuration
2+
:navtitle: Configuration
3+
4+
The `io_context_options` struct provides runtime tuning knobs for the
5+
I/O context and its backend scheduler. All defaults match the
6+
library's built-in values, so an unconfigured context behaves
7+
identically to previous releases.
8+
9+
[source,cpp]
10+
----
11+
#include <boost/corosio/io_context.hpp>
12+
13+
corosio::io_context_options opts;
14+
opts.max_events_per_poll = 256;
15+
opts.inline_budget_max = 32;
16+
17+
corosio::io_context ioc(opts);
18+
----
19+
20+
Both `io_context` and `native_io_context` accept options:
21+
22+
[source,cpp]
23+
----
24+
#include <boost/corosio/native/native_io_context.hpp>
25+
26+
corosio::io_context_options opts;
27+
opts.max_events_per_poll = 512;
28+
29+
corosio::native_io_context<corosio::epoll> ioc(opts);
30+
----
31+
32+
== Available Options
33+
34+
[cols="1,1,1,3"]
35+
|===
36+
| Option | Default | Backends | Description
37+
38+
| `max_events_per_poll`
39+
| 128
40+
| epoll, kqueue
41+
| Number of events fetched per reactor poll call. Larger values
42+
reduce syscall frequency under high load; smaller values improve
43+
fairness between connections.
44+
45+
| `inline_budget_initial`
46+
| 2
47+
| epoll, kqueue, select
48+
| Starting inline completion budget per handler chain. After a
49+
posted handler executes, the reactor grants this many speculative
50+
inline completions before forcing a re-queue.
51+
52+
| `inline_budget_max`
53+
| 16
54+
| epoll, kqueue, select
55+
| Hard ceiling on adaptive inline budget ramp-up. The budget
56+
doubles each cycle it is fully consumed, up to this limit.
57+
58+
| `unassisted_budget`
59+
| 4
60+
| epoll, kqueue, select
61+
| Inline budget when no other thread is running the event loop.
62+
Prevents a single-threaded context from starving connections.
63+
64+
| `gqcs_timeout_ms`
65+
| 500
66+
| IOCP
67+
| Maximum `GetQueuedCompletionStatus` blocking time in
68+
milliseconds. Lower values improve timer responsiveness at the
69+
cost of more syscalls.
70+
71+
| `thread_pool_size`
72+
| 1
73+
| POSIX (epoll, kqueue, select)
74+
| Number of worker threads in the shared thread pool used for
75+
blocking file I/O and DNS resolution. Ignored on IOCP where
76+
file I/O uses native overlapped I/O.
77+
|===
78+
79+
Options that do not apply to the active backend are silently ignored.
80+
81+
== Tuning Guidelines
82+
83+
=== Event Buffer Size (`max_events_per_poll`)
84+
85+
The event buffer controls how many I/O events are fetched in a single
86+
`epoll_wait()` or `kevent()` call.
87+
88+
* *High-throughput streaming* (few connections, high bandwidth):
89+
increase to 256-512 to reduce syscall overhead.
90+
* *Many idle connections* (chat servers, WebSocket hubs):
91+
keep at 128 or lower for better fairness.
92+
93+
=== Inline Completion Budget
94+
95+
The inline budget controls how many I/O completions the reactor
96+
completes speculatively within a single handler chain before forcing
97+
a re-queue through the scheduler.
98+
99+
* *Streaming workloads* (file transfer, video):
100+
`inline_budget_max = 32` or higher reduces context switches.
101+
* *Request-response workloads* (HTTP, RPC):
102+
keep at 16 to prevent one connection from monopolizing a thread.
103+
* *Single-threaded contexts*:
104+
`unassisted_budget` caps the budget when only one thread is
105+
running the event loop, preserving fairness.
106+
107+
=== IOCP Timeout (`gqcs_timeout_ms`)
108+
109+
On Windows, the IOCP scheduler periodically wakes to recheck timers.
110+
The default 500ms balances responsiveness with efficiency.
111+
112+
* *Sub-second timer precision*: reduce to 50-100ms.
113+
* *Minimal syscall overhead*: increase to 1000ms or higher.
114+
115+
=== Thread Pool Size (`thread_pool_size`)
116+
117+
On POSIX platforms, file I/O (`stream_file`, `random_access_file`)
118+
and DNS resolution use a shared thread pool.
119+
120+
* *Concurrent file operations*: increase to match expected
121+
parallelism (e.g. 4 for four concurrent file reads).
122+
* *No file I/O*: leave at 1 (the pool is created lazily).

include/boost/corosio/io_context.hpp

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,79 @@
2727

2828
namespace boost::corosio {
2929

30+
/** Runtime tuning options for @ref io_context.
31+
32+
All fields have defaults that match the library's built-in
33+
values, so constructing a default `io_context_options` produces
34+
identical behavior to an unconfigured context.
35+
36+
Options that apply only to a specific backend family are
37+
silently ignored when the active backend does not support them.
38+
39+
@par Example
40+
@code
41+
io_context_options opts;
42+
opts.max_events_per_poll = 256; // larger batch per syscall
43+
opts.inline_budget_max = 32; // more speculative completions
44+
opts.thread_pool_size = 4; // more file-I/O workers
45+
46+
io_context ioc(opts);
47+
@endcode
48+
49+
@see io_context, native_io_context
50+
*/
51+
struct io_context_options
52+
{
53+
/** Maximum events fetched per reactor poll call.
54+
55+
Controls the buffer size passed to `epoll_wait()` or
56+
`kevent()`. Larger values reduce syscall frequency under
57+
high load; smaller values improve fairness between
58+
connections. Ignored on IOCP and select backends.
59+
*/
60+
unsigned max_events_per_poll = 128;
61+
62+
/** Starting inline completion budget per handler chain.
63+
64+
After a posted handler executes, the reactor grants this
65+
many speculative inline completions before forcing a
66+
re-queue. Applies to reactor backends only.
67+
*/
68+
unsigned inline_budget_initial = 2;
69+
70+
/** Hard ceiling on adaptive inline budget ramp-up.
71+
72+
The budget doubles each cycle it is fully consumed, up to
73+
this limit. Applies to reactor backends only.
74+
*/
75+
unsigned inline_budget_max = 16;
76+
77+
/** Inline budget when no other thread assists the reactor.
78+
79+
When only one thread is running the event loop, this
80+
value caps the inline budget to preserve fairness.
81+
Applies to reactor backends only.
82+
*/
83+
unsigned unassisted_budget = 4;
84+
85+
/** Maximum `GetQueuedCompletionStatus` timeout in milliseconds.
86+
87+
Bounds how long the IOCP scheduler blocks between timer
88+
rechecks. Lower values improve timer responsiveness at the
89+
cost of more syscalls. Applies to IOCP only.
90+
*/
91+
unsigned gqcs_timeout_ms = 500;
92+
93+
/** Thread pool size for blocking I/O (file I/O, DNS resolution).
94+
95+
Sets the number of worker threads in the shared thread pool
96+
used by POSIX file services and DNS resolution. Must be at
97+
least 1. Applies to POSIX backends only; ignored on IOCP
98+
where file I/O uses native overlapped I/O.
99+
*/
100+
unsigned thread_pool_size = 1;
101+
};
102+
30103
namespace detail {
31104
struct timer_service_access;
32105
} // namespace detail
@@ -64,6 +137,12 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
64137
{
65138
friend struct detail::timer_service_access;
66139

140+
/// Pre-create services that depend on options (before construct).
141+
void apply_options_pre_(io_context_options const& opts);
142+
143+
/// Apply runtime tuning to the scheduler (after construct).
144+
void apply_options_post_(io_context_options const& opts);
145+
67146
protected:
68147
detail::scheduler* sched_;
69148

@@ -81,6 +160,17 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
81160
*/
82161
explicit io_context(unsigned concurrency_hint);
83162

163+
/** Construct with runtime tuning options and platform backend.
164+
165+
@param opts Runtime options controlling scheduler and
166+
service behavior.
167+
@param concurrency_hint Hint for the number of threads
168+
that will call `run()`.
169+
*/
170+
explicit io_context(
171+
io_context_options const& opts,
172+
unsigned concurrency_hint = std::thread::hardware_concurrency());
173+
84174
/** Construct with an explicit backend tag.
85175
86176
@param backend The backend tag value selecting the I/O
@@ -100,6 +190,30 @@ class BOOST_COROSIO_DECL io_context : public capy::execution_context
100190
sched_ = &Backend::construct(*this, concurrency_hint);
101191
}
102192

193+
/** Construct with an explicit backend tag and runtime options.
194+
195+
@param backend The backend tag value selecting the I/O
196+
multiplexer (e.g. `corosio::epoll`).
197+
@param opts Runtime options controlling scheduler and
198+
service behavior.
199+
@param concurrency_hint Hint for the number of threads
200+
that will call `run()`.
201+
*/
202+
template<class Backend>
203+
requires requires { Backend::construct; }
204+
explicit io_context(
205+
Backend backend,
206+
io_context_options const& opts,
207+
unsigned concurrency_hint = std::thread::hardware_concurrency())
208+
: capy::execution_context(this)
209+
, sched_(nullptr)
210+
{
211+
(void)backend;
212+
apply_options_pre_(opts);
213+
sched_ = &Backend::construct(*this, concurrency_hint);
214+
apply_options_post_(opts);
215+
}
216+
103217
~io_context();
104218

105219
io_context(io_context const&) = delete;

include/boost/corosio/native/detail/epoll/epoll_scheduler.hpp

Lines changed: 33 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@
3333
#include <chrono>
3434
#include <cstdint>
3535
#include <mutex>
36+
#include <vector>
3637

3738
#include <errno.h>
3839
#include <sys/epoll.h>
@@ -86,6 +87,13 @@ class BOOST_COROSIO_DECL epoll_scheduler final : public reactor_scheduler_base
8687
/// Shut down the scheduler, draining pending operations.
8788
void shutdown() override;
8889

90+
/// Apply runtime configuration, resizing the event buffer.
91+
void configure_reactor(
92+
unsigned max_events,
93+
unsigned budget_init,
94+
unsigned budget_max,
95+
unsigned unassisted) noexcept override;
96+
8997
/** Return the epoll file descriptor.
9098
9199
Used by socket services to register file descriptors
@@ -130,12 +138,17 @@ class BOOST_COROSIO_DECL epoll_scheduler final : public reactor_scheduler_base
130138

131139
// Set when the earliest timer changes; flushed before epoll_wait
132140
mutable std::atomic<bool> timerfd_stale_{false};
141+
142+
// Event buffer sized from max_events_per_poll_ (set at construction,
143+
// resized by configure_reactor via io_context_options).
144+
std::vector<epoll_event> event_buffer_;
133145
};
134146

135147
inline epoll_scheduler::epoll_scheduler(capy::execution_context& ctx, int)
136148
: epoll_fd_(-1)
137149
, event_fd_(-1)
138150
, timer_fd_(-1)
151+
, event_buffer_(max_events_per_poll_)
139152
{
140153
epoll_fd_ = ::epoll_create1(EPOLL_CLOEXEC);
141154
if (epoll_fd_ < 0)
@@ -217,6 +230,18 @@ epoll_scheduler::shutdown()
217230
interrupt_reactor();
218231
}
219232

233+
inline void
234+
epoll_scheduler::configure_reactor(
235+
unsigned max_events,
236+
unsigned budget_init,
237+
unsigned budget_max,
238+
unsigned unassisted) noexcept
239+
{
240+
reactor_scheduler_base::configure_reactor(
241+
max_events, budget_init, budget_max, unassisted);
242+
event_buffer_.resize(max_events_per_poll_);
243+
}
244+
220245
inline void
221246
epoll_scheduler::register_descriptor(int fd, descriptor_state* desc) const
222247
{
@@ -307,8 +332,9 @@ epoll_scheduler::run_task(std::unique_lock<std::mutex>& lock, context_type* ctx)
307332
if (timerfd_stale_.exchange(false, std::memory_order_acquire))
308333
update_timerfd();
309334

310-
epoll_event events[128];
311-
int nfds = ::epoll_wait(epoll_fd_, events, 128, timeout_ms);
335+
int nfds = ::epoll_wait(
336+
epoll_fd_, event_buffer_.data(),
337+
static_cast<int>(event_buffer_.size()), timeout_ms);
312338

313339
if (nfds < 0 && errno != EINTR)
314340
detail::throw_system_error(make_err(errno), "epoll_wait");
@@ -318,7 +344,7 @@ epoll_scheduler::run_task(std::unique_lock<std::mutex>& lock, context_type* ctx)
318344

319345
for (int i = 0; i < nfds; ++i)
320346
{
321-
if (events[i].data.ptr == nullptr)
347+
if (event_buffer_[i].data.ptr == nullptr)
322348
{
323349
std::uint64_t val;
324350
// NOLINTNEXTLINE(clang-analyzer-unix.BlockInCriticalSection)
@@ -327,7 +353,7 @@ epoll_scheduler::run_task(std::unique_lock<std::mutex>& lock, context_type* ctx)
327353
continue;
328354
}
329355

330-
if (events[i].data.ptr == &timer_fd_)
356+
if (event_buffer_[i].data.ptr == &timer_fd_)
331357
{
332358
std::uint64_t expirations;
333359
// NOLINTNEXTLINE(clang-analyzer-unix.BlockInCriticalSection)
@@ -337,8 +363,9 @@ epoll_scheduler::run_task(std::unique_lock<std::mutex>& lock, context_type* ctx)
337363
continue;
338364
}
339365

340-
auto* desc = static_cast<descriptor_state*>(events[i].data.ptr);
341-
desc->add_ready_events(events[i].events);
366+
auto* desc =
367+
static_cast<descriptor_state*>(event_buffer_[i].data.ptr);
368+
desc->add_ready_events(event_buffer_[i].events);
342369

343370
bool expected = false;
344371
if (desc->is_enqueued_.compare_exchange_strong(

0 commit comments

Comments
 (0)