Skip to content

CHATHISTORY TARGETS is fundamentally broken for long lists #604

@skizzerz

Description

@skizzerz

By requiring start and end timestamps and only considering the most recent message sent to each target, it is entirely possible for target lists which require pagination to either duplicate or omit targets. Duplication is probably a non-issue, but omission could cause clients to not request history that exists because it simply never appeared in TARGETS.

Issues

Consider the following "happy path" output, which I'll be using to demonstrate the issues in other examples:

CHATHISTORY TARGETS a 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS b 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS c 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS d 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS #foo 2026-04-28T13:00:00.000Z
CHATHISTORY TARGETS #bar 2026-04-28T13:01:00.000Z
CHATHISTORY TARGETS #baz 2026-04-28T13:02:00.000Z

Note that a, b, c, and d all have the same timestamp, perhaps due to a multi-target PRIVMSG (e.g. PRIVMSG a,b,c,d :hello!). Let us also assume that the current time is 2026-04-28T17:00:00.000Z (which is another can of worms I'll get into further below).

Duplicate timestamp omission

First issue, multiple messages in the same timestamp. Client sends CHATHISTORY TARGETS 2026-04-28T00:00:00.000Z 2026-04-28T17:00:00.000Z 3 and gets the following reply:

CHATHISTORY TARGETS a 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS b 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS c 2026-04-28T12:00:00.000Z

The sensible way to paginate this would be to send 2026-04-28T12:00:00.000Z as the starting timestamp in the next CHATHISTORY TARGETS command. However, doing so will omit the targets entry for d. Note this can still occur with larger limits than 3. Say the limit was 100 instead, and in our "happy path" example there are 97 other messages with varying timestamps between 2026-04-28T00:00:00.000Z and 2026-04-28T12:00:00.000Z. This will result in the same exact output (with those 97 other target lines prepended) and the same exact omission of d.

REDACT omission

Second issue, in a server implementation where messages removed via REDACT are not sent to users via chathistory, this could cause the timestamp of the most recent message to jump to a previous point in the past. Depending on when REDACT was issued, this is a potential race condition with long CHATHISTORY TARGETS (i.e. those that require pagination).

For this example, let's say that the message in #bar before the most recent message was sent 2026-04-28T10:00:00.000Z. This timestamp is within the original command's criteria but because of a race condition, was omitted in the targets listing entirely. The client is now unaware that #bar has history.

The client sends CHATHISTORY TARGETS 2026-04-28T00:00:00.000Z 2026-04-28T17:00:00.000Z 3 and gets the following reply:

CHATHISTORY TARGETS a 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS b 2026-04-28T12:00:00.000Z
CHATHISTORY TARGETS c 2026-04-28T12:00:00.000Z

Between this point and the time the client sends the followup, a chanop sends REDACT #bar ... to remove #bar's most recent message. The client then follows up with CHATHISTORY TARGETS 2026-04-28T12:00:00.000Z 2026-04-28T17:00:00.000Z 3 and gets the following reply:

CHATHISTORY TARGETS #foo 2026-04-28T13:00:00.000Z
CHATHISTORY TARGETS #baz 2026-04-28T13:02:00.000Z

Notice how #bar is missing entirely. If the REDACT were sent earlier, it would instead be part of the first batch of responses, but now it is outside of the timeframes of the pagination.

New message duplication

If a new message is sent that is still within the upper bound of the timeframe, a duplicated CHATHISTORY TARGETS line for that target will be emitted with the updated (new) timestamp. I'm not going to write a formal example for this as it should be relatively obvious to see, and is less of an issue than omission because a client can either ignore the duplicate or choose which of the two timestamps it cares about more.

Timestamps

The CHATHISTORY TARGETS command presupposes that the client somehow knows a good start and end time to use for the command. Given the very likely event that the client's and server's clocks are not synchronized, it is possible for targets to be omitted at the fringes of these start and end times because of that clock skew. While this is a race condition, consider a message that is sent to a client immediately before they join the network. Let's also say the client's clock is 1 second slower than the server's clock. The client sends the time they joined the network as the end range of CHATHISTORY TARGETS. Because of that skew, the client's criteria for TARGETS omits the timestamp the server believes that most recent message was sent and it will be omitted from the list.

Solutions

The output needs to include the msgid of the most recent message as well as its timestamp, to allow for pagination by msgid rather than timestamp. Similarly, the parameters should accept a msgid as well as or instead of a timestamp (according to MSGREFTYPES), to return targets which occurred after the specified msgid. With both of these combined, the duplicate timestamp omission case is solved.

Finally, it should consider all messages that occurred within the specified span rather than just the most recent message. In the very likely event that a target has multiple messages within the specified span, the msgid and timestamp of the oldest message that is still within that span should be used. This will potentially result in a lot of duplicate TARGET entries as the client paginates, which a client can resolve by considering whether the msgid represents a message it is already aware of and ignoring replies with such messages. However, doing it this way solves the REDACT omission case.

There is no solution for duplicated entries, and indeed the above solutions increase the amount of potential duplication. I view this as a non-issue as clients can ignore messages that are not relevant to them.

The solution for the timestamp issue is for the client to either rely on server-time for the end range or specify an arbitrarily large end range (such as 9999-01-01T00:00:00.000Z). Alternatively, we can make the end range parameter optional so that it is an open-ended query for the common case of a client fetching all relevant history. Once the client receives a message on the live connection, it can then fill in the first such message id or timestamp as the end range to clamp the range to purely historical data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions