perf(logger): allocation-free formatting and prompt async writer wakeup (#117)

Author: tkhquang

AGENTS.md

@@ -357,7 +357,7 @@ PATH="/c/msys64/mingw64/bin:$PATH" ./build/mingw-debug/tests/DetourModKit_tests.
 | Scanner | Stateless -- inherently safe | N/A (startup only) |
 | HookManager | SRWLOCK-backed reader/writer locks; two-phase shutdown (disable under shared lock, clear under exclusive lock), both phases walking `m_hook_creation_order` in reverse so inline/mid hooks layered on one address unwind newest-first instead of in bucket order; `m_mutator_gate` blocks new mutators (including all VMT operations) during teardown; CAS on `m_shutdown_called` serializes shutdown/remove_all_hooks; double-checked fast-fail on `m_shutdown_called` in all mutators; every public mutation or teardown entry point also fails closed (`HookError::ReentrantCallRejected`, a false/zero result, or a logged no-op for void lifecycle calls) when the per-thread reentrancy guard is set, so a call from inside a `with_*` callback returns or no-ops instead of recursively acquiring the non-recursive lock; the batch toggles (`enable_hooks`/`disable_hooks`/`*_all`) collect their log lines under the lock and emit them after release so a synchronous sink flush or a blocking async overflow never stalls an exclusive acquirer; destructor fallback (when `DMK_Shutdown()` was not called) acquires `m_mutator_gate` exclusively, flips `m_shutdown_called`, drains readers via exclusive `m_hooks_mutex`, then clears the maps -- under loader lock it pins the module and swaps each map's contents into heap storage allocated via `new (std::nothrow)` so the storage outlives the destructor without ever draining, mirroring the leak-on-loader-lock discipline used in `Logger::shutdown_internal` and `ConfigWatcher::~ConfigWatcher` | `shared_lock` SRWLOCK reader for `with_inline_hook()` |
 | Logger | `atomic<shared_ptr>` for lock-free async reads; `shutdown_internal` and `disable_async_mode` are safe across repeated shutdown / enable_async_mode cycles: when the writer thread has to be detached under loader lock, the module is pinned and the `shared_ptr<AsyncLogger>` is moved into a per-call permanent cell (normal path: `new (std::nothrow)`; fallback path: non-CRT permanent storage), so a heap allocation failure cannot drop the last handle while the writer may still be running | Single atomic load on log level check |
-| AsyncLogger | Lock-free MPMC queue (Vyukov-style); post-join drain on shutdown (at most one message per producer can be lost in the nanosecond race between drain and force-zero -- accepted trade-off to avoid atomic overhead on every enqueue); timestamp caching in write batches | Atomic sequence numbers per slot |
+| AsyncLogger | Lock-free MPMC queue (Vyukov-style); post-join drain on shutdown (at most one message per producer can be lost in the nanosecond race between drain and force-zero -- accepted trade-off to avoid atomic overhead on every enqueue); a producer wakes a parked writer through a seq_cst pending-count/flag handshake (`m_pending_messages` is made non-zero before the queue slot is published, the writer publishes `m_writer_waiting` before checking that count and blocking, and the producer notifies under `m_flush_mutex` only when the flag is set), so the busy-writer hot path stays lock-free and syscall-free yet a push can never strand a message until the flush-interval timeout; timestamp caching in write batches | Atomic sequence numbers per slot; flag-gated writer wakeup |
 | InputPoller | Atomic `m_active_states[]` array; the poll thread re-reserves its deferred-callback staging vector to the live binding count each cycle and stages it under a catch, so a runtime binding growth past the startup reserve cannot reallocate-then-throw out of the `jthread` body; a failed callback batch is dropped, not fatal | `shared_lock` (uncontended SRWLOCK reader) guarding the `m_active_states` pointer swap + `memory_order_relaxed` load per binding; keyboard/mouse reads route through a poll-thread-private per-cycle `KeyStateCache` so each distinct VK gets one coherent `GetAsyncKeyState` sample per cycle, not one call per binding reference |
 | InputManager | `mutex` for lifecycle, `atomic<shared_ptr<InputPoller>>` for reads | `atomic<shared_ptr<InputPoller>>` acquire-load, then the poller's `shared_lock` + relaxed load (not lock-free) |
 | InputIntercept (internal `src/input_intercept.*`) | File-scope atomics shared between the poll thread and the game's threads (XInput callers, window message thread); owns its safetyhook InlineHooks directly (not via HookManager) because the poll thread reads the trampoline and the hook lifetime is coupled to the poll thread; consume-until-release latch and wheel-pulse state are poll-thread-private; teardown skipped under loader lock (detours left installed against the pinned module) | Lock-free atomic loads in each detour; allocation-free, non-throwing detour bodies |

README.md

@@ -131,10 +131,10 @@ See the [Config Hot-Reload Guide](docs/config-hot-reload/README.md) for the thre
 <summary><strong>Async Logger</strong></summary>
 
 - Lock-free, bounded queue-based async logger decoupling log production from file I/O
-- Minimal latency on the producer side with batched writes on the consumer thread
+- Minimal latency on the producer side with batched writes on the consumer thread; a parked writer is woken promptly through a pending-count/flag handshake, so no message waits out the flush interval
 - Configurable overflow policies: DropNewest / DropOldest / Block / SyncFallback
 - Bounded Block policy with 16 ms default timeout (one frame at 60 fps) to prevent thread starvation
-- Inline buffer optimization for messages <= 512 bytes
+- Inline buffer optimization for messages <= 512 bytes; a formatted log line that fits is rendered into a stack buffer and never materializes a heap string
 - Message size validation with truncation for messages > 16 MB
 
 </details>

include/DetourModKit/async_logger.hpp

@@ -113,7 +113,7 @@ namespace DetourModKit
         std::chrono::system_clock::time_point timestamp;
         std::thread::id thread_id;
 
-        static constexpr size_t MAX_INLINE_SIZE = 512;
+        static constexpr size_t MAX_INLINE_SIZE = LOG_INLINE_MESSAGE_SIZE;
         static constexpr size_t MAX_VALID_LENGTH = MAX_MESSAGE_SIZE;
         std::array<char, MAX_INLINE_SIZE> buffer;
         size_t length{0};
@@ -343,6 +343,15 @@ namespace DetourModKit
 
         [[nodiscard]] bool is_running() const noexcept;
 
+        /**
+         * @brief Reports whether the writer thread is currently parked on the flush condition variable.
+         * @details Observability accessor for the idle-park state set by the writer immediately before it
+         *          blocks in wait_for and cleared when it wakes. Lets a test or diagnostic confirm the
+         *          writer has reached the parked path deterministically instead of relying on a fixed
+         *          sleep. The flag can flip at any time, so treat the result as a point-in-time snapshot.
+         */
+        [[nodiscard]] bool is_writer_waiting() const noexcept;
+
         [[nodiscard]] size_t queue_size() const noexcept;
 
         /**
@@ -371,6 +380,20 @@ namespace DetourModKit
 
         bool handle_overflow(LogMessage &&message) noexcept;
 
+        /**
+         * @brief Wakes the writer thread if it is parked on m_flush_cv after a successful push.
+         * @details A successful producer has already made m_pending_messages non-zero before publishing
+         *          a new queue slot, or preserved it while replacing an old slot. The writer publishes
+         *          m_writer_waiting before checking that same pending count and blocking. Those seq_cst
+         *          operations form a store/load handshake: if the writer misses the pending state, the
+         *          producer observes m_writer_waiting and notifies under m_flush_mutex; if the producer
+         *          observes false, the writer's pending-count predicate sees the work and does not block.
+         *          notify_all (not notify_one) is used so a flusher waiting on the same condition variable
+         *          cannot absorb the notification and leave the writer asleep. When the writer is
+         *          draining, this is a seq_cst flag load with no mutex or syscall.
+         */
+        void notify_writer() noexcept;
+
         DynamicMPMCQueue m_queue;
         AsyncLoggerConfig m_config;
 
@@ -383,6 +406,13 @@ namespace DetourModKit
 
         std::mutex m_flush_mutex;
         std::condition_variable m_flush_cv;
+
+        // Set true by the writer immediately before it parks on m_flush_cv and cleared when it wakes.
+        // Producers read it outside m_flush_mutex after a successful queue push; the seq_cst order shared
+        // with m_pending_messages makes a racing push either visible to the writer's wait predicate or
+        // visible here as a parked-writer wake.
+        std::atomic<bool> m_writer_waiting{false};
+
         std::atomic<size_t> m_pending_messages{0};
         std::atomic<size_t> m_dropped_messages{0};
     };

include/DetourModKit/logger.hpp

@@ -5,6 +5,8 @@
 #include <string_view>
 #include <mutex>
 #include <memory>
+#include <array>
+#include <cstddef>
 #include <format>
 #include <atomic>
 
@@ -55,6 +57,14 @@ namespace DetourModKit
     inline constexpr const char *DEFAULT_LOG_FILE_NAME = "DetourModKit_Log.txt";
     inline constexpr const char *DEFAULT_TIMESTAMP_FORMAT = "%Y-%m-%d %H:%M:%S";
 
+    // Upper bound, in bytes, on a log line that the formatted log() / try_log() fast path renders without
+    // a heap allocation. Those templates format into a stack buffer of this size and forward a view when
+    // the line fits, so a line up to this length never materializes a heap std::string. It mirrors the
+    // async sink's inline message buffer (LogMessage::MAX_INLINE_SIZE), which likewise stores a line of
+    // this size without touching the StringPool; longer lines take the documented overflow path on both
+    // sides.
+    inline constexpr std::size_t LOG_INLINE_MESSAGE_SIZE = 512;
+
     // Forward declarations
     struct AsyncLoggerConfig;
     class AsyncLogger;
@@ -202,7 +212,13 @@ namespace DetourModKit
         {
             if (level >= m_current_log_level.load(std::memory_order_acquire))
             {
-                log(level, std::format(fmt, std::forward<Args>(args)...));
+                // Format into a stack buffer instead of through a std::format temporary so a line that fits
+                // the inline buffer never heap-allocates. The view handed to the sink is consumed
+                // synchronously (the async path copies it into LogMessage, the sync path writes it) before
+                // this call returns, so it never outlives the stack buffer.
+                static_cast<void>(format_dispatch([this, level](std::string_view formatted)
+                                                  { return this->log(level, formatted); }, fmt,
+                                                  std::forward<Args>(args)...));
             }
         }
 
@@ -255,7 +271,11 @@ namespace DetourModKit
             }
             try
             {
-                return log_noexcept(level, std::format(fmt, std::forward<Args>(args)...));
+                // Same allocation-free formatting as log(); the try/catch preserves the no-throw contract
+                // because format_to_n and the std::format overflow fallback can both throw.
+                return format_dispatch([this, level](std::string_view formatted) noexcept
+                                       { return this->log_noexcept(level, formatted); }, fmt,
+                                       std::forward<Args>(args)...);
             }
             catch (...)
             {
@@ -296,6 +316,33 @@ namespace DetourModKit
         Logger(Logger &&) = delete;
         Logger &operator=(Logger &&) = delete;
 
+        /**
+         * @brief Formats one log line into a stack buffer and hands it to @p sink.
+         * @details Renders into a buffer the size of the async sink's inline message buffer
+         *          (LOG_INLINE_MESSAGE_SIZE). std::format_to_n reports the untruncated length, so a line
+         *          that fits is passed to @p sink as a view into the stack buffer with no heap allocation;
+         *          a std::format temporary would instead allocate for any result past the small-string
+         *          limit, which the async LogMessage then copies into its own inline buffer anyway. A line
+         *          longer than the inline buffer is the documented overflow case (the async sink stores it
+         *          via StringPool / heap up to MAX_MESSAGE_SIZE), so it is re-formatted once through
+         *          std::format to give @p sink the full line. The formatter only reads its arguments (it
+         *          never moves them), so forwarding the same pack to both format_to_n and the std::format
+         *          fallback is safe.
+         * @return Whatever @p sink returns for the line.
+         */
+        template <typename Sink, typename... Args>
+        static auto format_dispatch(Sink &&sink, std::format_string<Args...> fmt, Args &&...args)
+        {
+            std::array<char, LOG_INLINE_MESSAGE_SIZE> buffer;
+            const auto result = std::format_to_n(buffer.data(), buffer.size(), fmt, std::forward<Args>(args)...);
+            const auto formatted = static_cast<std::size_t>(result.size);
+            if (formatted <= buffer.size())
+            {
+                return sink(std::string_view(buffer.data(), formatted));
+            }
+            return sink(std::string_view(std::format(fmt, std::forward<Args>(args)...)));
+        }
+
         /**
          * @brief Shared shutdown logic used by both ~Logger() and shutdown().
          */