Periodic standby status update ignores auto: false, silently advancing replication slot position

[holomekc]

Description:

When configuring { auto: false, timeoutSeconds: 10 }, the expectation is that WAL position is only advanced via explicit acknowledge(lsn) calls. However, the checkStandbyStatus timer calls acknowledge(this._lastLsn) directly — bypassing the auto guard — which sends a standby status update with the last received LSN to PostgreSQL, advancing confirmed_flush_lsn.

This causes silent data loss in manual-ack workflows: if a message fails processing and is not explicitly acknowledged, a subsequent timer-driven standby status update will advance the pointer past it.

Root cause:

In checkStandbyStatus:

this.checkStandbyStatusTimer = setInterval(async () => {
    if (this._stop) return;
    if (this._lastLsn && Date.now() - this.lastStandbyStatusUpdatedTime > ...)
        await this.acknowledge(this._lastLsn);  // calls public acknowledge(), not _acknowledge()
}, 1000);

The private _acknowledge() correctly checks auto:

private async _acknowledge(lsn: string) {
    if (!this.config.acknowledge!.auto) return;  // guarded
    ...
}

But checkStandbyStatus calls this.acknowledge() (public) directly, which has no auto check.

Additionally, _lastLsn is updated on every received message unconditionally (including messages the user hasn't processed yet):

this._lastLsn = lsn;  // updated for every copyData, regardless of ack

###History:

We noticed that this exact issue was already identified and fixed in commit 2198528, which added an auto guard to checkStandbyStatus. This was later reverted in commit 103f686 as part of PR #169 to avoid a breaking change in a minor release.

We fully understand the concern around breaking changes — the periodic standby status has likely been relied upon by existing users, even in auto: false configurations. That said, the current behavior can lead to silent data loss in manual-ack workflows, which is arguably more surprising than a documented behavioral change.

Would it make sense to reconsider the original fix and ship it as part of a major release? That way existing users get fair notice via the version bump, and users relying on auto: false for strict manual acknowledgment get the semantics they'd expect.

An even better middle ground might be to track _lastAcknowledgedLsn separately, so the timer can still send keepalives (using the last safe position) without advancing past unprocessed messages. This would preserve connection health for all users while respecting the auto: false contract.

Workaround:

Set timeoutSeconds: 0 to disable the timer entirely. For short-lived consumers (e.g., Lambda), the connection won't be dropped within wal_sender_timeout. For long-lived consumers, implement a manual heartbeat using the heartbeat event and acknowledging with the last successfully processed LSN.

Affected version: 2.4.0

[kibae]

@holomekc Thanks for the detailed report and for tracing the history so thoroughly.

You're right that this is exactly the issue we identified and fixed in 2198528, then reverted in 2.4.0 due to breaking change concerns.

I want to share the broader design tension I've been wrestling with:

Currently acknowledge config has two distinct behaviors bundled together:

• auto: true → sends ack after every data message
• timeoutSeconds > 0 → sends ack on a periodic timer, regardless of auto

These are actually two separate concerns. For true manual ack, you currently need { auto: false, timeoutSeconds: 0 } — but the config shape doesn't make this obvious at all. timeoutSeconds sounds like it only matters when auto: true, which is misleading.

The clean fix would be to separate these into distinct options — something like autoAck and keepaliveInterval — making the intent explicit. But this library has significant downstream dependents, and renaming config keys is a real breaking change.

I did cut a 3.0.0 with the auto guard added to the timer, then rolled it back to 2.4.0 because I wasn't confident about the impact. The question I keep coming back to: is silently advancing confirmed_flush_lsn (current behavior) worse than a breaking config rename?

I'd genuinely like to hear your take — especially whether you'd prefer a clean 3.0 with renamed config, or a minimal fix (just guarding the timer with auto) as a patch.

[kibae]

@holomekc Thanks for the patience. I went with the cleaner path (Option A from my earlier comment) and opened a PR for v3.0.1: #176

What changed:

1. Config rename — the nested acknowledge: { auto, timeoutSeconds } is split into two top-level keys so the two concerns (per-message auto-ack vs. periodic keepalive) stop being conflated:

   - acknowledge: { auto: false, timeoutSeconds: 10 }
   + autoAck: false
   + keepaliveIntervalSeconds: 10

   The old acknowledge key is still accepted in v3.x with a console.warn and value mapping — so existing code keeps working, but the intent is visibly deprecated. Removal is scheduled for v4.0.

2. Internal LSN separation (this is the actual fix for the bug you reported). The service now tracks lastReceivedLsn and lastAckedLsn separately. When autoAck: false:

   • The keepalive timer still fires (so wal_sender_timeout doesn't drop the connection)
   • But the standby status message reports only lastAckedLsn as the flushed/applied position
   • → the replication slot never silently advances past manually acked work

   This is close to your suggestion 2 (_lastAcknowledgedLsn) — I think the same idea you had is what was needed, just paired with the config rename so users can tell at a glance what each knob does.

3. Migration notice at the top of the README and in CHANGELOG.md with a diff snippet, so anyone updating notices before they hit runtime.

A small unit test (src/test/config-normalize.spec.ts) covers the deprecation shim — defaults, new keys, old keys emitting the warning, and precedence when both are supplied.

Would love your read on this — does the shape feel right, and does the keepalive-without-advancing-slot behavior match what you'd expect for autoAck: false? Any edge case I should add a test for before releasing?

[holomekc]

Sorry for the very, very, very late response! Your suggestion sounds pretty good to us. This would give us more control. The keepAlive would not accidentially move the cursor and with on('heartbeat'...) we still have access to the lastReceivedLsn.