Export: jump to "from date" instead of walking entire chat history

[Lertov2424232]

Problem

When the user picks a "From date" for chat / topic export, the exporter still issues messages.getHistory starting at offset_id = 1 (the very first message of the chat) and walks forward through the entire history, filtering each message client-side via SkipMessageByDate. On a chat with millions of messages this is tens of thousands of redundant API calls and tens of millions of messages transferred just to discard them. Users have reported export effectively hanging for hours on large chats when only the last few days were requested.

Fix

Use MTP's offset_date parameter on the first slice of each split / topic to position the server directly at singlePeerFrom, then continue the existing forward walk by offset_id once we have a foothold. The flow is otherwise unchanged.

Changes

• requestMessagesSlice() — on the first slice of a split, if singlePeerFrom > 0, request with offset_id = 0, offset_date = singlePeerFrom, add_offset = -limit, limit = kMessagesSliceLimit. Subsequent slices use the existing largestIdPlusOne walk.
• requestChatMessages() — gains an optional offsetDate parameter (default 0), threaded into the MTPmessages_GetHistory call (which previously hardcoded offset_date = 0). The CHANNEL_PRIVATE → onlyMyMessages fallback recurses with the same offsetDate.
• messages.Search (the onlyMyMessages branch) now passes min_date = singlePeerFrom and max_date = singlePeerTill, which were previously hardcoded to 0.
• finishMessagesSlice() / finishTopicMessagesSlice() — early-stop the loop once the newest message in the current batch is at or past singlePeerTill, instead of walking to the end of the chat just to client-side-skip the tail.
• requestTopicMessagesSlice() / requestTopicReplies() — same offset_date treatment for forum topic export.
• TopicProcess gains a firstSliceRequested flag so the topic loop can cleanly distinguish the first slice request from continuation (the existing offsetId == 0 check could not, since offsetId is seeded with topicRootId after the root-message fetch).

Behaviour without a date filter

When singlePeerFrom == 0 && singlePeerTill == 0 the new branches are all gated off and the export performs the same MTP calls as before, byte-for-byte. The new code paths activate only when the user explicitly picks a date range in the export dialog.

Verification

A small Python simulator of the relevant messages.getHistory call shapes was used to verify that:

1. The saved message set is identical to the analytically-expected in-range set in every scenario (no missed messages, no duplicates).
2. The no-filter path is unchanged (same number of calls, same byte count).
3. The narrow-range path is dramatically cheaper.

Scenario	Legacy calls / msgs received	Patched calls / msgs received	Speed-up
5k chat, no filter (regression)	50 / 5 000	50 / 5 000	1×
100k chat, from = last 500 msgs	1 000 / 100 000	6 / 501	167× / 200× traffic
100k chat, narrow 1h from+till slot	1 000 / 100 000	6 / 600	167×
10k chat, from in future	100 / 10 000	1 / 0	100×
5k chat, from at oldest message	50 / 5 000	50 / 5 000	1× (correct fallback)
5k chat, till older than oldest	50 / 5 000	1 / 100	50×

The simulator script is not included in this PR — happy to add it as a developer tool / test if maintainers want it.

Known follow-up (not in this PR)

_chatProcess->info.messagesCountPerSplit[*] for the getHistory path still reflects the total message count of the chat (not just the in-range count), so the export progress bar percentage in the UI is computed against the full chat. The export itself terminates correctly (lastSlice from the server, or the new early-stop on singlePeerTill), but the visible percentage caps below 100% on a narrow range. Fixing this cleanly would require either a server-side count probe with min_date/max_date (via messages.Search) or counting processed messages locally, which is a larger change in the precount pipeline. Left out to keep this PR surgical.

[CLAassistant]

All committers have signed the CLA.

[23rd]

We already have two similar works.
Please look and check if the problems described there and your solution are related?

#30612
#30618

[Lertov2424232]

Thanks @23rd — yes, all three PRs are tackling the same root issue (the exporter walks the whole chat from offset_id = 1 instead of using offset_date to position at the user-picked "from date"), and all three converge on offset_date in messages.getHistory as the fix. They differ mostly in how invasive the change is and in a couple of boundary details. Quick comparison from my side:

#30727 (this PR)

• First slice is fetched with a single call: offset_id = 0, offset_date = singlePeerFrom, add_offset = -kMessagesSliceLimit, limit = kMessagesSliceLimit. After that the existing largestIdPlusOne walk takes over unchanged. No extra probe round-trip.
• offsetDate is an optional parameter (default 0) on requestChatMessages / requestTopicReplies, so the existing call sites are not touched and behavior with no date filter is byte-for-byte identical.
• Topic path gets a firstSliceRequested flag so the "first slice" condition is not overloaded onto offsetId == 0 (which is already used by the root-message bootstrap).
• Early-stop on singlePeerTill in finishMessagesSlice / finishTopicMessagesSlice so we don't keep paging past the upper bound.

#30612

• Two-step at the start of each split: a tiny probe getHistory(offset_date = from - 1, add_offset = -1, limit = 1) to discover the first id at-or-after the date, then resume the normal id-based walk. Functionally equivalent to what this PR does, at the cost of one extra round-trip per split, but very localized to the existing pagination loop.
• Includes a real edge-case fix that this PR was missing: SingleMessageAfter(data, date) flipped from > date to >= date in export_data_types.cpp. Without this, a split whose newest message has date == singlePeerFrom is wrongly skipped by the pre-iteration filter.
• Adds SkipMessageByDate to loadNextTopicMessageFile so topic media loading honors the date filter too. The chat path already does this in loadNextMessageFile; the topic path didn't.

#30618

• Larger scope: extends the date-range UI to full-data exports (not just single-peer), persists singlePeerFrom / singlePeerTill in Settings, adds a client-side [from, till) trim before custom-emoji resolution / media download as a defensive guard, and adjusts the progress bar for date-filtered runs.
• Its messages.search bounds for the onlyMyMessages fallback are min_date = from - 1, max_date = till, which matches the canonical [from, till) semantics of SkipMessageByDate exactly.

On boundary semantics

SkipMessageByDate keeps messages where from <= date < till. messages.search uses strict > for min_date and strict < for max_date, so to match the exporter's semantics the bounds should be:

min_date = from - 1   // date > from-1  =>  date >= from
max_date = till       // date < till

The previous revision of this PR passed min_date = from, max_date = till in the onlyMyMessages branch, which would have missed messages with date == from. I've now pushed a follow-up commit that:

1. Adopts the SingleMessageAfter > -> >= fix from fix: honor date range in chat/topic export, skip pre-from history #30612.
2. Adds SkipMessageByDate to loadNextTopicMessageFile, also from fix: honor date range in chat/topic export, skip pre-from history #30612.
3. Corrects the onlyMyMessages search bounds to min_date = singlePeerFrom - 1 (the #30618-style canonical mapping).
4. Adds a changelog entry.

So this PR should now be a strict superset of the export-API correctness changes in #30612, while staying surgical (no UI / settings churn). Happy to defer to whichever PR you ultimately prefer — if you'd rather take #30612 or #30618 as the base I'm fine to close this and open a small follow-up with whatever the chosen base is missing instead. Just let me know.

[23rd]

Did you (as human person) test your changes in some cases?

[Lertov2424232]

Re: "Did you (as human person) test your changes in some cases?"

Honest answer: I couldn't get an end-to-end Telegram Desktop build done. My local machine doesn't have the ~60-80 GB of free disk required for the Windows build per docs/building-win.md, and a CI run on my fork (Lertov2424232/tdesktop run 26405941101) failed at the Libraries step on the very first try — fork had no warm libraries cache, so the runner tried to build lzma from scratch and hit MSB8020: The build tools for v145 cannot be found, which is the VS 2026 toolset that upstream CI gets from its pre-populated cache.

Rather than ship something un-tested, I wrote a small Python simulator that models the relevant pieces of the export state machine plus the messages.getHistory / messages.search / SkipMessageByDate / SingleMessageAfter semantics, then compares three code states on synthetic chats with explicit boundary cases:

• baseline — dev as-is
• pr_initial — this PR before the 4 follow-up fixes
• pr_final — this PR with the 4 follow-up fixes

Code + raw output: https://gist.github.com/Lertov2424232/39dc5cd04f033914a9369905c1fc9ada

Summary of results

case                            baseline                pr_initial / pr_final
big_chat_from_near_end          501 RPCs / 50000 recv   2 RPCs / 100 recv
boundary_date_eq_from           11 / 1000               6 / 501
boundary_date_eq_till           11 / 1000               6 / 600
single_message_at_from_only     2 / 50                  1 / 1
no_filter_regression            6 / 500                 6 / 500    (identical)
only_my_messages_with_from      4 / 286                 pr_initial misses 1; pr_final ✓
from_and_till_window            51 / 5000               11 / 1100

• All seven chat cases: pr_final matches ground truth exactly.
• big_chat_from_near_end is the case the PR is meant to fix: baseline does 501 RPCs and ships 50,000 messages over the wire for a 100-message result, both PR variants do 2 RPCs and ship 100. This matches the original --from complaint the PR cites.
• only_my_messages_with_from is the case where pr_initial was off by one and pr_final is correct: messages.search uses strict min_date < date, so min_date = from skipped a message whose date equals from. min_date = from - 1 fixes it.

SingleMessageAfter boundary (direct check)

The count-probe pre-filter (SingleMessageAfter) is a different code path from the slice loop, so the simulator checks it directly. With a split whose newest message has date == singlePeerFrom:

baseline      SingleMessageAfter -> False   skipSplit -> True    skips the only matching split
pr_initial    SingleMessageAfter -> False   skipSplit -> True    skips it as well
pr_final      SingleMessageAfter -> True    skipSplit -> False   keeps it

This is why the original > was changed to >= in export_data_types.cpp.

Limitations of this evidence

• It's a model of the protocol surface, not the real client. It does not exercise MTProto, real messages.getHistory server quirks, or the file-download paths.
• I've assumed messages.search bounds are strict (min_date < date < max_date), which is the convention used by pyrogram/telethon and matches PR Enable date ranges for full data exports #30618's choice; if the server actually applies >= on min_date, the from - 1 fix becomes a harmless one-extra-record fetch (the result is still filtered by SkipMessageByDate) — it doesn't regress anything.
• I had no way to confirm loadNextTopicMessageFile behavior empirically; that fix is a logical mirror of what already exists in the non-topic media path (SkipMessageByDate filter on the media download loop).

Ask

Would you be willing to approve CI on this PR so it can run against your warm libraries cache? My fork couldn't get past the cold-cache lzma issue and I'd much rather submit empirical evidence than a simulator. I'm also happy to drop any of the follow-up fixes if you'd prefer them as separate PRs.

[23rd]

I couldn't get an end-to-end Telegram Desktop build done.

🤣

[john-preston]

Someone's OpenClaw?

Did any of PRs do the correct progress reporting on limited-by-date range exports?

[23rd]

🤩