Skip to content

autoPaginate.maxItems silently drops rows and returns a cursor that skips them #329

Description

@V3RON

Summary

callToolWithOptionalPagination in packages/agent-sdk/src/pagination.ts trims the collected items down to maxItems, but copies the producer's page envelope through unchanged. The resulting page.nextCursor points past the rows that were just discarded, so those rows become unreachable — no error, no gap marker.

There are two independent loss paths.

1. Initial page (pagination.ts:97-101)

const merged: PagedResponse = {
  ...initial,
  items: getTrimmedItems(initial.items, maxItems),  // trims to maxItems
  page: { ...initial.page },                        // keeps the full page's nextCursor
};

With limit: 20, maxItems: 5, the tool returns 20 rows, the caller receives 5, and merged.page.nextCursor still points past row 19. Rows 5–19 are silently unreachable. The while loop never runs, because its merged.items.length < maxItems guard is already false.

2. Mid-loop (pagination.ts:132-140)

const remaining = maxItems === undefined
  ? next.items.length
  : Math.max(0, maxItems - merged.items.length);

merged.items.push(...next.items.slice(0, remaining));
merged.page = { ...next.page };   // same defect

Steps to reproduce

Against any cursor-paginated agent tool (e.g. console.getMessages) over a 20-row dataset:

  1. Call with autoPaginate: { pagesLimit: 2, maxItems: 5 } and limit: 20.
  2. Observe 5 items returned, plus a page.nextCursor.
  3. Follow that cursor.
  4. The union of both responses is missing rows 5–19.

Mid-loop variant: limit: 5, maxItems: 7, pagesLimit: 2 over the same dataset returns rows 0–6, and the returned cursor resumes at row 10 — rows 7, 8, 9 are lost.

Why it matters now

The cursor used to be an opaque token that nothing acted on automatically. #318 makes the CLI render it into output as a runnable npx rozenite agent ... --cursor <token> command, so a lossy cursor becomes a copy-pasteable instruction that skips data.

Note

The SDK cannot fix this by synthesizing a cursor — cursor encoding is producer-owned and opaque, so there is no way to express "row 5 of this page". The fix is to stop over-fetching: request only the rows still within budget, so a well-behaved producer's own cursor lands exactly on the truncation boundary. A producer that clamps or ignores limit still needs an explicit "resume position unknown" signal rather than a misleading cursor.

This is independent of #320, which covers the producer-side pagination model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions