[SqlStatement] Add transparent block-prefetch for classic per-row fetch loops

Classic result iteration (while(cursor.FetchRow()){ GetColumn<T>(i) },
bound output columns, SqlRowIterator<T>, SqlVariantRowCursor) issued one
SQLFetch per row -- one network round-trip per row, which dominates
wall-clock for large result sets on TCP backends. The recent row-wise
array fetch only sped up the materializing DataMapper paths (All/Range/
First); the lazy cursor loops were left on the per-row path and cannot be
rewritten across all client code.

Back the classic cursor transparently with the existing RowArrayCursor:
on the first fetch of an eligible result set the statement arms a block
buffer (SQL_ATTR_ROW_ARRAY_SIZE) and serves FetchRow()/GetColumn<T>() and
bound-column scatters from it, cutting round-trips from N to
ceil(N/depth). On by default; depth is a single connection-level knob
(SqlConnection::SetDefaultPrefetchDepth, default PrefetchDepthDefault =
1000; <= 1 disables). Capability-gated by SupportsNativeRowArrayFetch().

Eligibility is restricted to fixed-width numeric, temporal and GUID
columns, whose block reconstruction is byte-identical to the per-row
binder on every backend. Result sets carrying character/text, NUMERIC,
TIME, binary or LOB columns transparently stay on the per-row path
(faithful materialization of those is not uniform across backends: MSSQL
returns narrow text in the client codepage, SQLite's dynamic typing
reports unreliable text sizes). A new SqlLogger::OnFetchBlock hook makes
the round-trip reduction observable/testable.

Performance: round-trips drop ~1000x at depth 1000 for eligible sets;
no regression on the per-row path (no allocation when disabled/ineligible).
Risk: an active cursor reads ahead up to one block (a few MB, budget-
clamped); the connection knob is the global escape hatch.

Tested: sqlite3, mssql2022 (Docker), postgres (Docker 16.4) -- full suite
shows no regression vs the pre-change baseline; new [prefetch] suite green
on all three. Build clean under clangcl-debug (PEDANTIC /WX).

Signed-off-by: Christian Parpart <c.parpart@lastrada.net>

Author: Christian Parpart

docs/best-practices.md

@@ -106,6 +106,25 @@ single response.
 This will help to reduce the response time and the load on the server, and improve the performance of your
 application.
 
+### Let block-prefetch cut network round-trips
+
+Per-row fetch loops issue one `SQLFetch` (one network round-trip) per row. Lightweight transparently
+fetches rows in blocks (ODBC row-array binding) so a large result set costs `ceil(rows / depth)`
+round-trips instead of one per row — see [Transparent block-prefetch](usage.md). It is on by default
+(`Lightweight::PrefetchDepthDefault`, 1000 rows) and tuned per connection:
+
+```cpp
+connection.SetDefaultPrefetchDepth(1000); // rows per SQLFetchScroll round-trip; <= 1 disables
+```
+
+Keep in mind:
+
+- It engages only for **fixed-width numeric/temporal** result sets; result sets with character,
+  `GUID`, `NUMERIC`, `TIME`, binary or LOB columns transparently stay on the per-row path.
+- An active cursor reads ahead up to one block and holds a few MB of buffers, so set the depth to `1`
+  on a connection used for cursors you intend to abandon early or where memory is tight.
+- It does not change results — values are identical to the per-row path.
+
 ## SQL Server Variation Challenges
 
 ### 64-bit Integer Handling in Oracle Database

docs/usage.md

@@ -32,6 +32,38 @@ while (stmt.FetchRow())
 }
 ```
 
+## Transparent block-prefetch (fewer network round-trips)
+
+Classic per-row fetch loops like the one above issue **one `SQLFetch` per row**, i.e. one network
+round-trip per row. On TCP-backed drivers (Microsoft SQL Server, PostgreSQL) that latency dominates the
+wall-clock time of large result sets.
+
+Lightweight transparently reduces these round-trips: on the first `FetchRow()` of a result set it
+inspects the columns and, when eligible, fetches whole **blocks** of rows per `SQLFetchScroll`
+round-trip (ODBC row-array binding) and serves your `FetchRow()` / `GetColumn<T>()` calls from that
+buffer. **No code change is required** — the loops above, `SqlRowIterator<T>`, `SqlVariantRowCursor`
+and the `DataMapper` all benefit automatically.
+
+The depth is a connection-level setting (default `Lightweight::PrefetchDepthDefault`, 1000 rows). A
+value `<= 1` disables prefetch and restores one `SQLFetch` per row:
+
+```cpp
+auto conn = SqlConnection {};
+conn.SetDefaultPrefetchDepth(2000); // request up to 2000 rows per SQLFetchScroll round-trip
+conn.SetDefaultPrefetchDepth(1);    // disable prefetch for this connection
+```
+
+Prefetch engages only for result sets whose columns are **fixed-width numeric, temporal, or `GUID`**
+types (integers, floating point, `DATE`, `TIMESTAMP`/`DATETIME`, and native `GUID`/`uniqueidentifier`/
+`uuid`) on drivers that support native row-array fetching (Microsoft SQL Server, PostgreSQL, SQLite).
+Result sets that contain character/text, `NUMERIC`/`DECIMAL`, `TIME`, binary or LOB columns transparently
+keep the per-row path: faithful block reconstruction of those is not achievable uniformly across backends
+(e.g. Microsoft SQL Server returns narrow text in the client codepage rather than UTF-8, and SQLite's
+dynamic typing reports text/`NUMERIC` columns with an unreliable, unenforced size), so the dedicated
+single-row binders handle them. Memory is bounded to a few MB per active cursor (the depth is auto-clamped
+to that budget), and prefetch reads ahead up to one block, so a loop that stops early over-reads at most
+one block.
+
 ## Prepared Statements
 
 You can also use prepared statements to execute queries, for example

src/Lightweight/SqlConnectInfo.hpp

@@ -5,6 +5,7 @@
 #include "Api.hpp"
 
 #include <chrono>
+#include <cstddef>
 #include <format>
 #include <map>
 #include <string>
@@ -13,6 +14,15 @@
 namespace Lightweight
 {
 
+/// @brief Default block-prefetch depth for new connections: the number of rows a classic per-row
+/// fetch loop requests per @c SQLFetchScroll round-trip on the transparent prefetch path.
+///
+/// Suffixed (not @c DefaultPrefetchDepth) so it does not collide with the
+/// @c SqlConnection::DefaultPrefetchDepth() accessor. A connection's depth can be overridden via
+/// @c SqlConnection::SetDefaultPrefetchDepth or @ref SqlConnectionDataSource::defaultPrefetchDepth;
+/// a value <= 1 disables prefetch.
+constexpr std::size_t PrefetchDepthDefault = 1000;
+
 /// Represents an ODBC connection string.
 struct SqlConnectionString
 {
@@ -62,6 +72,13 @@ struct [[nodiscard]] SqlConnectionDataSource
     std::string password;
     /// The connection timeout duration.
     std::chrono::seconds timeout { 5 };
+    /// @brief Default block-prefetch depth applied to statements created on the resulting connection
+    /// (rows requested per @c SQLFetchScroll round-trip on the transparent per-row fetch path).
+    ///
+    /// A value <= 1 disables prefetch (every classic loop keeps issuing one @c SQLFetch per row).
+    /// Defaults to @ref PrefetchDepthDefault. Has effect only on backends whose driver supports
+    /// native row-array fetching (see @c SqlConnection::SupportsNativeRowArrayFetch).
+    std::size_t defaultPrefetchDepth = PrefetchDepthDefault;
 
     /// Constructs a SqlConnectionDataSource from the given connection string.
     LIGHTWEIGHT_API static SqlConnectionDataSource FromConnectionString(SqlConnectionString const& value);

src/Lightweight/SqlConnection.cpp

@@ -54,7 +54,9 @@ struct SqlConnection::Data
     std::chrono::steady_clock::time_point lastUsed; // Last time the connection was used (mostly interesting for
                                                     // idle connections in the connection pool).
     SqlConnectionString connectionString;
-    std::unique_ptr<Async::IAsyncBackend> asyncBackend; // Async execution backend (null until EnableAsync()).
+    std::unique_ptr<Async::IAsyncBackend> asyncBackend;      // Async execution backend (null until EnableAsync()).
+    std::size_t defaultPrefetchDepth = PrefetchDepthDefault; // Rows requested per SQLFetchScroll on the
+                                                             // transparent per-row prefetch path (<= 1 disables).
 };
 
 SqlConnection::SqlConnection():
@@ -173,6 +175,16 @@ std::chrono::steady_clock::time_point SqlConnection::LastUsed() const noexcept
     return m_data->lastUsed;
 }
 
+std::size_t SqlConnection::DefaultPrefetchDepth() const noexcept
+{
+    return m_data->defaultPrefetchDepth;
+}
+
+void SqlConnection::SetDefaultPrefetchDepth(std::size_t depth) noexcept
+{
+    m_data->defaultPrefetchDepth = depth;
+}
+
 void SqlConnection::EnableAsync(Async::IExecutor& dbWorkers, Async::IResumeScheduler& resume)
 {
     // TODO(async): once the native event backend lands, select it here via a per-connection
@@ -219,6 +231,8 @@ bool SqlConnection::Connect(SqlConnectionDataSource const& info) noexcept
     ZoneScopedN("SqlConnection::Connect(DataSource)");
     EnsureHandlesAllocated();
 
+    m_data->defaultPrefetchDepth = info.defaultPrefetchDepth;
+
     if (m_hDbc)
         SQLDisconnect(m_hDbc);
 

src/Lightweight/SqlConnection.hpp

@@ -184,6 +184,22 @@ class SqlConnection final
     /// @return `true` if narrow character data round-trips byte-exact on that backend.
     [[nodiscard]] static bool RoundTripsNarrowTextByteExact(SqlServerType serverType) noexcept;
 
+    /// @brief The default block-prefetch depth applied to statements created on this connection.
+    ///
+    /// Classic per-row fetch loops (`while (cursor.FetchRow()) ...`, @c SqlRowIterator,
+    /// @c SqlVariantRowCursor) transparently request this many rows per @c SQLFetchScroll round-trip
+    /// instead of issuing one @c SQLFetch per row. A value <= 1 disables prefetch. Effective only when
+    /// @ref SupportsNativeRowArrayFetch is true; otherwise statements fall back to per-row fetching.
+    ///
+    /// @return The configured default prefetch depth (defaults to @ref PrefetchDepthDefault).
+    [[nodiscard]] LIGHTWEIGHT_API std::size_t DefaultPrefetchDepth() const noexcept;
+
+    /// @brief Sets the default block-prefetch depth for statements created on this connection.
+    ///
+    /// @param depth Rows to request per @c SQLFetchScroll round-trip on the transparent prefetch path;
+    ///              a value <= 1 disables prefetch (restoring one @c SQLFetch per row).
+    LIGHTWEIGHT_API void SetDefaultPrefetchDepth(std::size_t depth) noexcept;
+
     /// Creates a new query builder for the given table, compatible with the current connection.
     ///
     /// @param table The table to query.

src/Lightweight/SqlLogger.hpp

@@ -113,6 +113,15 @@ class SqlLogger
     /// Invoked when a row is fetched.
     virtual void OnFetchRow() = 0;
 
+    /// @brief Invoked once per block-prefetch round-trip (one @c SQLFetchScroll that materialized a
+    /// whole row block on the transparent prefetch path), with the number of rows the block yielded.
+    ///
+    /// Non-pure with an empty default so existing loggers need no change; override to observe how many
+    /// network round-trips a query actually cost (N rows at depth D collapse to @c ceil(N/D) blocks).
+    ///
+    /// @param rowsInBlock Number of rows materialized by this block fetch (0 at end of result set).
+    virtual void OnFetchBlock(std::size_t /*rowsInBlock*/) {}
+
     /// Invoked when fetching is done.
     virtual void OnFetchEnd() = 0;