Add ODBC-native row-wise array fetch for fast bulk retrieval (#511)

Customers on high-latency links wait the longest when retrieving many
rows because each row is its own `SQLFetch` round-trip — fetching 1000
rows costs 1000 round-trips. This adds the read-side counterpart of the
existing `CreateAll`/`UpdateAll` batch-write path:
`Query<Record>().All()`/`Range()` (and two-record JOIN tuples) now bind
result columns row-wise directly into the caller's `std::vector<Record>`
storage and pull whole row blocks per `SQLFetchScroll`, collapsing the
round-trips from N to ~`ceil(N/depth)`. Values land in place
(zero-copy); results are byte-identical to the per-row path.

The fast path is transparent and gated — a record qualifies when every
result column is row-bindable (the same `SqlRowBindableColumn` set the
write side uses: primitives, date/time/datetime, numeric, char
fixed-capacity strings, and non-numeric optionals of those) and the
driver supports row-array fetching. Anything else (growable
strings/binary, GUID, variant) transparently falls back to the unchanged
per-row path. Verified against SQLite, SQL Server 2022 and PostgreSQL.

## Changes

- **`SqlStatement::FetchAllRowWise`** — new low-level primitive
mirroring `ExecuteBatchNativeRowWise`: sets `SQL_ATTR_ROW_BIND_TYPE =
sizeof(Record)` + `SQL_ATTR_ROW_ARRAY_SIZE`, grows-and-rebinds the
destination per block, clamps depth to a memory budget, and restores
single-row statement state via a `Finally` guard on every exit
(including exceptions). Nullable columns use an over-allocated
row-strided NULL indicator; optionals are pre-engaged and reset on NULL.
Char fixed strings bind `SQL_C_CHAR` inline with a per-row length/trim
fixup.
- **`SqlConnection::SupportsNativeRowArrayFetch`** and
**`RoundTripsNarrowTextByteExact`** — driver capability checks kept on
the connection. The latter carves out PostgreSQL (whose psqlODBC
transcodes `SQL_C_CHAR` through the client codepage), so records
carrying a fixed-capacity string fall back to the per-row wide path
there rather than risk mangling non-ASCII bytes.
- **`DataMapper` retrieval wiring** — compile-time eligibility
(`CanRowWiseFetchRecord`/`CanRowWiseFetchTuple` plus the narrow-text
carve-out) and the `ReadResults` branch for both single-record and
two-record tuple result sets.
- **`RowWiseFetchTests`** — covers fixed/nullable/temporal/fixed-string
columns, NULL/empty/full-capacity values, multi-block boundaries, empty
results, `Where`/`Range`, statement reuse and the `std::string`
fallback, asserting via a block-fetch-counting logger that the fast path
actually ran (and that fixed-string records fall back on PostgreSQL). A
hidden opt-in benchmark (`[.rowwisefetchbench]`) reproduces the
comparison below.

## Performance

The structural win is round-trip count: for 200k rows the row-wise path
issues **~196 `SQLFetchScroll` calls instead of 200,000 `SQLFetch`
calls** (~1000x fewer round-trips). Measured end-to-end (release build,
median of 5 reps, `Query<>().All()` vs the per-row fallback, both
materializing the same `std::vector<Record>`):

| Backend | Transport | per-row `SQLFetch` | row-wise block fetch |
speedup |
|---|---|---|---|---|
| SQLite | in-process | 643 ms / 500k | 541 ms | **1.19x** |
| PostgreSQL | localhost TCP | 167 ms / 200k | 109 ms | **1.53x** |
| SQL Server 2022 | localhost TCP | 134 ms / 200k | 67 ms | **2.01x** |

The speedup scales with per-round-trip transport cost. In-process SQLite
has no socket, so its ~1.2x is purely reduced per-call CPU/ODBC overhead
— the floor, and a latency-independent constant. Over a localhost socket
(sub-millisecond RTT) it is already 1.5-2x.

On a high-latency link the round-trip term dominates: wall-clock ≈
round-trips × RTT, so the per-row path pays ~N × RTT while the block
path pays ~`ceil(N/depth)` × RTT, and the speedup approaches the
effective array depth (up to **~1000x** for narrow records, less for
wide rows that clamp to a smaller depth, and divided by whatever the
driver already prefetches per fetch). For example, modelled at 50 ms RTT
for 100k rows: ~83 min (per-row) vs ~5 s (block). The local numbers
above are a conservative lower bound demonstrating the mechanism; the
round-trip-count reduction is what carries the win to the WAN case.

Author: christianparpart

docs/best-practices.md

@@ -106,6 +106,25 @@ single response.
 This will help to reduce the response time and the load on the server, and improve the performance of your
 application.
 
+### Let block-prefetch cut network round-trips
+
+Per-row fetch loops issue one `SQLFetch` (one network round-trip) per row. Lightweight transparently
+fetches rows in blocks (ODBC row-array binding) so a large result set costs `ceil(rows / depth)`
+round-trips instead of one per row — see [Transparent block-prefetch](usage.md). It is on by default
+(`Lightweight::PrefetchDepthDefault`, 1000 rows) and tuned per connection:
+
+```cpp
+connection.SetDefaultPrefetchDepth(1000); // rows per SQLFetchScroll round-trip; <= 1 disables
+```
+
+Keep in mind:
+
+- It engages only for **fixed-width numeric/temporal** result sets; result sets with character,
+  `GUID`, `NUMERIC`, `TIME`, binary or LOB columns transparently stay on the per-row path.
+- An active cursor reads ahead up to one block and holds a few MB of buffers, so set the depth to `1`
+  on a connection used for cursors you intend to abandon early or where memory is tight.
+- It does not change results — values are identical to the per-row path.
+
 ## SQL Server Variation Challenges
 
 ### 64-bit Integer Handling in Oracle Database

docs/usage.md

@@ -32,6 +32,38 @@ while (stmt.FetchRow())
 }
 ```
 
+## Transparent block-prefetch (fewer network round-trips)
+
+Classic per-row fetch loops like the one above issue **one `SQLFetch` per row**, i.e. one network
+round-trip per row. On TCP-backed drivers (Microsoft SQL Server, PostgreSQL) that latency dominates the
+wall-clock time of large result sets.
+
+Lightweight transparently reduces these round-trips: on the first `FetchRow()` of a result set it
+inspects the columns and, when eligible, fetches whole **blocks** of rows per `SQLFetchScroll`
+round-trip (ODBC row-array binding) and serves your `FetchRow()` / `GetColumn<T>()` calls from that
+buffer. **No code change is required** — the loops above, `SqlRowIterator<T>`, `SqlVariantRowCursor`
+and the `DataMapper` all benefit automatically.
+
+The depth is a connection-level setting (default `Lightweight::PrefetchDepthDefault`, 1000 rows). A
+value `<= 1` disables prefetch and restores one `SQLFetch` per row:
+
+```cpp
+auto conn = SqlConnection {};
+conn.SetDefaultPrefetchDepth(2000); // request up to 2000 rows per SQLFetchScroll round-trip
+conn.SetDefaultPrefetchDepth(1);    // disable prefetch for this connection
+```
+
+Prefetch engages only for result sets whose columns are **fixed-width numeric, temporal, or `GUID`**
+types (integers, floating point, `DATE`, `TIMESTAMP`/`DATETIME`, and native `GUID`/`uniqueidentifier`/
+`uuid`) on drivers that support native row-array fetching (Microsoft SQL Server, PostgreSQL, SQLite).
+Result sets that contain character/text, `NUMERIC`/`DECIMAL`, `TIME`, binary or LOB columns transparently
+keep the per-row path: faithful block reconstruction of those is not achievable uniformly across backends
+(e.g. Microsoft SQL Server returns narrow text in the client codepage rather than UTF-8, and SQLite's
+dynamic typing reports text/`NUMERIC` columns with an unreliable, unenforced size), so the dedicated
+single-row binders handle them. Memory is bounded to a few MB per active cursor (the depth is auto-clamped
+to that budget), and prefetch reads ahead up to one block, so a loop that stops early over-reads at most
+one block.
+
 ## Prepared Statements
 
 You can also use prepared statements to execute queries, for example

src/Lightweight/DataMapper/DataMapper.hpp

@@ -744,6 +744,220 @@ namespace detail
         BindAllOutputColumnsWithOffset(reader, record, 1);
     }
 
+    /// @brief Requested rows per SQLFetchScroll round-trip for the native row-wise fetch fast path. The
+    /// statement clamps this to a memory budget, so it is an upper bound, not a guarantee.
+    constexpr std::size_t kDefaultRowArrayFetchDepth = 1024;
+
+    /// @brief Mutable-reference output accessor for member @p I that is a Field/BelongsTo: yields the
+    /// field's mutable value so the row-wise fetch path binds the result column in place. The read-side
+    /// counterpart of @ref FieldValueAccessor.
+    template <std::size_t I>
+    struct MutableFieldValueAccessor
+    {
+        template <typename Record>
+        decltype(auto) operator()(Record& record) const
+        {
+            return GetRecordMemberAt<I>(record).MutableValue();
+        }
+    };
+
+    /// @brief The mutable value type bound for member @p FieldType on the row-wise fetch path (the type
+    /// the result column materializes into).
+    template <typename FieldType>
+    using RowWiseColumnValueType = std::remove_cvref_t<decltype(std::declval<FieldType&>().MutableValue())>;
+
+    /// @return Whether @p FieldType maps to a result column on the bound-output path (Field, BelongsTo, or
+    /// a directly-bindable member) — mirrors the classification in @ref BindAllOutputColumnsWithOffset.
+    template <typename FieldType>
+    constexpr bool RowWiseIsColumn()
+    {
+        return IsField<FieldType> || IsBelongsTo<FieldType> || SqlOutputColumnBinder<FieldType>;
+    }
+
+    /// @return Whether @p FieldType is acceptable on the row-wise fetch path: either it is not a result
+    /// column (a relation member, which is not bound) or it is a column whose value type is
+    /// @ref SqlRowWiseFetchableColumn. Directly-bindable non-Field members are conservatively rejected
+    /// (their value would need a separate accessor shape) so such records fall back to the per-row path.
+    template <typename FieldType>
+    constexpr bool RowWiseColumnAcceptable()
+    {
+        if constexpr (IsField<FieldType> || IsBelongsTo<FieldType>)
+            return SqlRowWiseFetchableColumn<RowWiseColumnValueType<FieldType>>;
+        else if constexpr (SqlOutputColumnBinder<FieldType>)
+            return false;
+        else
+            return true; // relation / non-column member: not bound, imposes no constraint
+    }
+
+    template <typename Record, std::size_t... Is>
+    constexpr bool CanRowWiseFetchRecordImpl(std::index_sequence<Is...>)
+    {
+        // The row-strided indicator slots are addressed at i * sizeof(Record); they must stay SQLLEN
+        // aligned, so sizeof(Record) must be a multiple of alignof(SQLLEN) (mirrors the write-side
+        // indicatorAlignmentSatisfied precondition).
+        return (sizeof(Record) % alignof(SQLLEN) == 0) && (RowWiseColumnAcceptable<RecordMemberTypeOf<Is, Record>>() && ...)
+               && (RowWiseIsColumn<RecordMemberTypeOf<Is, Record>>() || ...);
+    }
+
+    /// @brief Whether @p Record can be materialized via the native row-wise array-fetch fast path: every
+    /// result column is a Field/BelongsTo of a @ref SqlRowWiseFetchableColumn type, there is at least one
+    /// column, and the record size keeps the row-strided indicators aligned. Records that fail this fall
+    /// back to the per-row @c SQLFetch path, with identical results.
+    template <typename Record>
+    constexpr bool CanRowWiseFetchRecord()
+    {
+        return CanRowWiseFetchRecordImpl<Record>(std::make_index_sequence<RecordMemberCount<Record>> {});
+    }
+
+    /// Returns a one-element accessor tuple for member @p I when it is a bound result column, else an empty
+    /// tuple — flattened via tuple_cat so the accessor pack matches the bound column set and order exactly.
+    template <std::size_t I, typename Record>
+    auto MakeOutputColumnAccessor()
+    {
+        using FieldType = RecordMemberTypeOf<I, Record>;
+        if constexpr (IsField<FieldType> || IsBelongsTo<FieldType>)
+            return std::tuple<MutableFieldValueAccessor<I>> {};
+        else
+            return std::tuple<> {};
+    }
+
+    /// @brief Materializes the whole result set into @p records via @ref SqlStatement::FetchAllRowWise,
+    /// building one mutable value accessor per bound result column (same set and order as
+    /// @ref BindAllOutputColumnsWithOffset). Precondition: @ref CanRowWiseFetchRecord<Record>().
+    template <typename Record>
+    void ReadAllRowWise(SqlResultCursor& reader, std::vector<Record>* records)
+    {
+        [&]<std::size_t... Is>(std::index_sequence<Is...>) {
+            std::apply(
+                [&](auto const&... accessors) {
+                    reader.FetchAllRowWise(*records, kDefaultRowArrayFetchDepth, accessors...);
+                },
+                std::tuple_cat(MakeOutputColumnAccessor<Is, Record>()...));
+        }(std::make_index_sequence<RecordMemberCount<Record>> {});
+    }
+
+    /// @return Whether @p FieldType is a result column whose value is a char fixed-capacity string (or a
+    /// @c std::optional of one). Such columns are array-bound narrow (SQL_C_CHAR), which only round-trips
+    /// byte-exact where @ref SqlConnection::RoundTripsNarrowTextByteExact holds.
+    template <typename FieldType>
+    constexpr bool ColumnIsNarrowFixedString()
+    {
+        if constexpr (IsField<FieldType> || IsBelongsTo<FieldType>)
+        {
+            using V = RowWiseColumnValueType<FieldType>;
+            if constexpr (SqlIsStdOptional<V>)
+                return IsSqlFixedString<typename V::value_type>;
+            else
+                return IsSqlFixedString<V>;
+        }
+        else
+            return false;
+    }
+
+    template <typename Record, std::size_t... Is>
+    constexpr bool RecordHasNarrowFixedStringColumnImpl(std::index_sequence<Is...>)
+    {
+        return (ColumnIsNarrowFixedString<RecordMemberTypeOf<Is, Record>>() || ...);
+    }
+
+    /// @brief Whether @p Record has any char fixed-capacity-string result column. Such records take the
+    /// row-wise fetch fast path only on backends that round-trip narrow text byte-exact; elsewhere they
+    /// fall back to the per-row (wide) path. See @ref SqlConnection::RoundTripsNarrowTextByteExact.
+    template <typename Record>
+    constexpr bool RecordHasNarrowFixedStringColumn()
+    {
+        return RecordHasNarrowFixedStringColumnImpl<Record>(std::make_index_sequence<RecordMemberCount<Record>> {});
+    }
+
+    /// @brief Whether @p Record may use the row-wise fetch fast path on @p serverType: it is row-wise
+    /// fetchable, the driver supports row-array fetch, and any narrow fixed-string column round-trips
+    /// byte-exact there. Single runtime gate composed from connection capabilities + the compile-time
+    /// record shape, so business logic never branches on the server type directly.
+    template <typename Record>
+    bool CanRowWiseFetchOn(SqlServerType serverType)
+    {
+        if constexpr (!CanRowWiseFetchRecord<Record>())
+            return false;
+        else
+            return SqlConnection::SupportsNativeRowArrayFetch(serverType)
+                   && (!RecordHasNarrowFixedStringColumn<Record>()
+                       || SqlConnection::RoundTripsNarrowTextByteExact(serverType));
+    }
+
+    // --- Two-record tuple (JOIN) fast path ----------------------------------------------------------
+
+    /// @brief Mutable-reference output accessor for member @p I of the @p TupleIndex-th sub-record of a
+    /// @c std::tuple result row; yields that field's mutable value so a JOIN result binds in place.
+    template <std::size_t TupleIndex, std::size_t I>
+    struct MutableTupleFieldAccessor
+    {
+        template <typename TupleType>
+        decltype(auto) operator()(TupleType& row) const
+        {
+            return GetRecordMemberAt<I>(std::get<TupleIndex>(row)).MutableValue();
+        }
+    };
+
+    template <typename First, typename Second, std::size_t... Fs, std::size_t... Ss>
+    constexpr bool CanRowWiseFetchTupleImpl(std::index_sequence<Fs...>, std::index_sequence<Ss...>)
+    {
+        return (sizeof(std::tuple<First, Second>) % alignof(SQLLEN) == 0)
+               && (RowWiseColumnAcceptable<RecordMemberTypeOf<Fs, First>>() && ...)
+               && (RowWiseColumnAcceptable<RecordMemberTypeOf<Ss, Second>>() && ...)
+               && ((RowWiseIsColumn<RecordMemberTypeOf<Fs, First>>() || ...)
+                   || (RowWiseIsColumn<RecordMemberTypeOf<Ss, Second>>() || ...));
+    }
+
+    /// @brief Whether a @c std::tuple<First,Second> JOIN row can be materialized via the row-wise fetch
+    /// fast path: both sub-records' columns are row-bindable and the combined row size keeps the
+    /// row-strided indicators aligned.
+    template <typename First, typename Second>
+    constexpr bool CanRowWiseFetchTuple()
+    {
+        return CanRowWiseFetchTupleImpl<First, Second>(std::make_index_sequence<RecordMemberCount<First>> {},
+                                                       std::make_index_sequence<RecordMemberCount<Second>> {});
+    }
+
+    /// @brief Whether a @c std::tuple<First,Second> JOIN row may use the row-wise fetch fast path on
+    /// @p serverType (row-wise fetchable + driver supports row-array fetch + any narrow fixed-string
+    /// column round-trips byte-exact there). The tuple counterpart of @ref CanRowWiseFetchOn.
+    template <typename First, typename Second>
+    bool CanRowWiseFetchTupleOn(SqlServerType serverType)
+    {
+        if constexpr (!CanRowWiseFetchTuple<First, Second>())
+            return false;
+        else
+            return SqlConnection::SupportsNativeRowArrayFetch(serverType)
+                   && ((!RecordHasNarrowFixedStringColumn<First>() && !RecordHasNarrowFixedStringColumn<Second>())
+                       || SqlConnection::RoundTripsNarrowTextByteExact(serverType));
+    }
+
+    /// Accessor tuple for member @p I of the @p TupleIndex-th sub-record, or empty for non-columns.
+    template <std::size_t TupleIndex, std::size_t I, typename SubRecord>
+    auto MakeTupleColumnAccessor()
+    {
+        using FieldType = RecordMemberTypeOf<I, SubRecord>;
+        if constexpr (IsField<FieldType> || IsBelongsTo<FieldType>)
+            return std::tuple<MutableTupleFieldAccessor<TupleIndex, I>> {};
+        else
+            return std::tuple<> {};
+    }
+
+    /// @brief Materializes a two-record JOIN result set into @p records via row-wise array fetch. The
+    /// accessor pack is First's columns followed by Second's, matching the column order of
+    /// @ref BindAllOutputColumnsWithOffset's offset scheme. Precondition: @ref CanRowWiseFetchTuple.
+    template <typename First, typename Second>
+    void ReadAllRowWiseTuple(SqlResultCursor& reader, std::vector<std::tuple<First, Second>>* records)
+    {
+        [&]<std::size_t... Fs, std::size_t... Ss>(std::index_sequence<Fs...>, std::index_sequence<Ss...>) {
+            std::apply(
+                [&](auto const&... accessors) {
+                    reader.FetchAllRowWise(*records, kDefaultRowArrayFetchDepth, accessors...);
+                },
+                std::tuple_cat(MakeTupleColumnAccessor<0, Fs, First>()..., MakeTupleColumnAccessor<1, Ss, Second>()...));
+        }(std::make_index_sequence<RecordMemberCount<First>> {}, std::make_index_sequence<RecordMemberCount<Second>> {});
+    }
+
     // when we iterate over all columns using element mask
     // indexes of the mask corresponds to the indexe of the field
     // inside the structure, not inside the SQL result set
@@ -1268,6 +1482,19 @@ void SqlAllFieldsQueryBuilder<Record, QueryOptions, Execution>::ReadResults(SqlS
                                                                             SqlResultCursor reader,
                                                                             std::vector<Record>* records)
 {
+    // Fast path: when every result column is a fixed-width row-bindable field and the driver honours
+    // native row-array fetching, materialize the whole result set in row blocks (one SQLFetchScroll per
+    // block) directly into records, instead of one SQLFetch round-trip per row. Results are identical to
+    // the per-row path below; this only collapses ODBC round-trips (the win on high-latency links).
+    if constexpr (detail::CanRowWiseFetchRecord<Record>())
+    {
+        if (detail::CanRowWiseFetchOn<Record>(sqlServerType))
+        {
+            detail::ReadAllRowWise(reader, records);
+            return;
+        }
+    }
+
     while (true)
     {
         Record& record = records->emplace_back();
@@ -1293,6 +1520,17 @@ template <typename FirstRecord, typename SecondRecord, DataMapperOptions QueryOp
 void SqlAllFieldsQueryBuilder<std::tuple<FirstRecord, SecondRecord>, QueryOptions, Execution>::ReadResults(
     SqlServerType sqlServerType, SqlResultCursor reader, std::vector<RecordType>* records)
 {
+    // Fast path: a JOIN row of two row-bindable records is bound row-wise over the tuple and fetched in
+    // blocks (one SQLFetchScroll per block) instead of one SQLFetch per row. Identical results.
+    if constexpr (detail::CanRowWiseFetchTuple<FirstRecord, SecondRecord>())
+    {
+        if (detail::CanRowWiseFetchTupleOn<FirstRecord, SecondRecord>(sqlServerType))
+        {
+            detail::ReadAllRowWiseTuple<FirstRecord, SecondRecord>(reader, records);
+            return;
+        }
+    }
+
     while (true)
     {
         auto& record = records->emplace_back();

src/Lightweight/SqlConnectInfo.hpp

@@ -5,6 +5,7 @@
 #include "Api.hpp"
 
 #include <chrono>
+#include <cstddef>
 #include <format>
 #include <map>
 #include <string>
@@ -13,6 +14,15 @@
 namespace Lightweight
 {
 
+/// @brief Default block-prefetch depth for new connections: the number of rows a classic per-row
+/// fetch loop requests per @c SQLFetchScroll round-trip on the transparent prefetch path.
+///
+/// Suffixed (not @c DefaultPrefetchDepth) so it does not collide with the
+/// @c SqlConnection::DefaultPrefetchDepth() accessor. A connection's depth can be overridden via
+/// @c SqlConnection::SetDefaultPrefetchDepth or @ref SqlConnectionDataSource::defaultPrefetchDepth;
+/// a value <= 1 disables prefetch.
+constexpr std::size_t PrefetchDepthDefault = 1000;
+
 /// Represents an ODBC connection string.
 struct SqlConnectionString
 {
@@ -62,6 +72,13 @@ struct [[nodiscard]] SqlConnectionDataSource
     std::string password;
     /// The connection timeout duration.
     std::chrono::seconds timeout { 5 };
+    /// @brief Default block-prefetch depth applied to statements created on the resulting connection
+    /// (rows requested per @c SQLFetchScroll round-trip on the transparent per-row fetch path).
+    ///
+    /// A value <= 1 disables prefetch (every classic loop keeps issuing one @c SQLFetch per row).
+    /// Defaults to @c PrefetchDepthDefault. Has effect only on backends whose driver supports
+    /// native row-array fetching (see @c SqlConnection::SupportsNativeRowArrayFetch).
+    std::size_t defaultPrefetchDepth = PrefetchDepthDefault;
 
     /// Constructs a SqlConnectionDataSource from the given connection string.
     LIGHTWEIGHT_API static SqlConnectionDataSource FromConnectionString(SqlConnectionString const& value);