Skip to content

Commit 9f311f8

Browse files
xuyushun441-sysos-zhuangclaude
authored
feat(messaging): digest batching for notifications (ADR-0030 P3b-2) (#1582)
Recipients can batch a topic into a daily/weekly digest instead of receiving every notification immediately. Builds on P3b-1's deferral seam: - PreferenceResolver consumes the `digest` field; digestDeferral() defers a batched recipient to the next window (local midnight / Monday 00:00) and tags the target with a stable window. Digest takes precedence over quiet-hours; critical + mandatory topics bypass it. - sys_notification_delivery gains digest_key (recipient|channel|window). Batched rows partition by that key so a window co-locates; normal claim() skips them and the new claimDigest() drains a window whole (memory + sql outbox). - The dispatcher's digest pass collapses each (recipient, channel, window) group into one renderDigest() message under the per-partition lock, then acks every row in the group with that single outcome. Additive — non-digest delivery is unchanged. +13 tests (127 total). tz-from- sys_user, configurable send-hour, and MJML digest emails are deferred follow-ups. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent a46c017 commit 9f311f8

12 files changed

Lines changed: 601 additions & 32 deletions

.changeset/notification-digest.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
'@objectstack/service-messaging': minor
3+
---
4+
5+
feat(messaging): digest batching for notifications (ADR-0030 P3b-2)
6+
7+
Recipients can now batch a topic into a `daily` / `weekly` **digest** instead of
8+
receiving every notification immediately. Builds on P3b-1's deferral seam:
9+
10+
- `PreferenceResolver` consumes the `digest` preference field and `digestDeferral()`
11+
defers a batched recipient to the next window (local midnight / Monday 00:00),
12+
tagging the target with a stable `window`. Digest takes precedence over
13+
quiet-hours; `critical` and mandatory topics bypass it.
14+
- `sys_notification_delivery` gains a `digest_key` (`recipient|channel|window`).
15+
Batched rows partition by that key so a window's rows co-locate, and the normal
16+
outbox `claim()` skips them while the new `claimDigest()` drains a window whole.
17+
- The dispatcher's digest pass collapses each `(recipient, channel, window)` group
18+
into one `renderDigest()` message under the existing per-partition cluster lock,
19+
then acks every row in the group with that single outcome.
20+
21+
Additive: non-digest notifications are unchanged. Timezone-from-`sys_user`,
22+
configurable send-hour, and MJML digest emails are deferred follow-ups.

docs/adr/0030-notification-platform-convergence.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ADR-0030 — Notification Platform Convergence (single ingress, layered pipeline)
22

3-
**Status**: Accepted (2026-06-01) — **P0–P3b1 shipped**; P3b-2 (digest) + cross-repo objectui cut-over remain. See [§ Implementation status & remaining work](#implementation-status--remaining-work).
3+
**Status**: Accepted (2026-06-01) — **P0–P3b2 shipped** (P3b-2 digest collapse landed); cross-repo objectui cut-over remains. See [§ Implementation status & remaining work](#implementation-status--remaining-work).
44
**Supersedes / refines**: [ADR-0012 — Notification Platform](./0012-notification-platform.md) (Draft)
55
**Related**: [ADR-0019 — Approval as a Flow Node](./0019-approval-as-flow-node.md), [ADR-0022 — Connectors vs Messaging Channels](./0022-connectors-vs-messaging-channels.md)
66
**Build spec**: [docs/design/notification-platform-convergence.md](../design/notification-platform-convergence.md)
@@ -82,16 +82,17 @@ The detailed cut-over runbook and per-item notes live in
8282
| **P2 — Subscription + preference** | `sys_notification_preference` (user×topic×channel, admin-global `*` defaults + per-user override, wildcards) + `sys_notification_subscription`; `PreferenceResolver` wired into `emit()` (most-specific-wins, mandatory bypass, fail-open). | #1444 |
8383
| **P3a — email channel + templates** | `createEmailChannel` (delegates transport to the `email` service per ADR-0022); `sys_notification_template` (topic×channel×locale) + declarative `{{ payload.x }}` renderer with `payload.title`/`body` fallback. | #1449 |
8484
| **P3b-1 — quiet-hours** | Deferred dispatch on the outbox (`EnqueueDeliveryInput.notBefore``nextAttemptAt`); `quietHoursDeferral()` (tz/HH:MM, overnight-aware); `critical` bypass. | #1453 |
85+
| **P3b-2 — digest** | `PreferenceResolver` consumes the `digest` field (`daily`/`weekly`) → `digestDeferral()` defers to the next window and tags the target; `digest_key` on `sys_notification_delivery` (partition-keyed so a window's rows co-locate); `INotificationOutbox.claimDigest()` drains batched rows whole while normal `claim()` skips them; the dispatcher's digest pass collapses each `(recipient, channel, window)` group into one `renderDigest()` message under the partition lock. `critical`/mandatory bypass. | this PR |
8586
| Startup | `messaging` is foundational: in `ALWAYS_ON_CAPABILITIES` (CLI) and auto-loaded when `audit` is required (cloud capability-loader). | (in #1434) |
8687

8788
### Remaining work (handed off to a follow-up agent)
8889

89-
**1. P3b-2 — Digest (completes the build spec).** Build on P3b-1's deferral:
90-
enqueue digest items deferred to the next window, then a **collapse** step merges
91-
same-`(user, channel, window)` deliveries into one materialization at window time.
92-
Needs: a `digest_key` on `sys_notification_delivery`; a digest assembler (in/beside
93-
the dispatcher); a digest render template. Consumes P2's `digest` field;
94-
`critical`/mandatory bypass.
90+
**1. ~~P3b-2 — Digest~~ — done (this PR).** `PreferenceResolver` batches digest
91+
recipients to the next window; deliveries carry `digest_key`; the dispatcher
92+
collapses same-`(recipient, channel, window)` rows into one rendered message.
93+
Deferred sub-items not in this cut: timezone fallback to a `sys_user` field
94+
(digest windows currently use `quiet_hours.tz` → UTC), MJML digest emails, and a
95+
configurable daily send-hour (windows flush at local midnight / Monday 00:00).
9596

9697
**2. Cross-repo objectui cut-over (the user-facing delivery — separate `objectui` repo).**
9798
- Repoint the Console bell (`AppHeader`/`InboxPopover`/record views) from
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import type { NotificationDeliveryRecord } from './outbox.js';
4+
5+
/**
6+
* One collapsed digest message assembled from a window's batched deliveries
7+
* (ADR-0030 P3b-2). `items` preserves the individual notifications so a
8+
* structured channel (inbox) can render a list, while `title`/`body` give a
9+
* flat rendering for plain channels (email text).
10+
*/
11+
export interface DigestRenderResult {
12+
title: string;
13+
body: string;
14+
severity: 'info';
15+
count: number;
16+
items: Array<{
17+
notificationId: string;
18+
title: string;
19+
body?: string;
20+
topic?: string;
21+
actionUrl?: string;
22+
}>;
23+
}
24+
25+
/**
26+
* Collapse same-`(recipient, channel, window)` deliveries into a single message.
27+
* The caller guarantees `rows` is non-empty and shares a recipient + channel
28+
* (the digest group). Only non-`critical` notifications are ever batched, so the
29+
* digest severity is always `info`.
30+
*/
31+
export function renderDigest(rows: NotificationDeliveryRecord[]): DigestRenderResult {
32+
const items = rows.map((r) => {
33+
const p = r.payload ?? {};
34+
return {
35+
notificationId: r.notificationId,
36+
title: typeof p.title === 'string' && p.title ? p.title : (r.topic ?? 'Notification'),
37+
body: typeof p.body === 'string' && p.body ? p.body : undefined,
38+
topic: r.topic,
39+
actionUrl: typeof p.actionUrl === 'string' && p.actionUrl ? p.actionUrl : undefined,
40+
};
41+
});
42+
const count = items.length;
43+
const title = count === 1 ? items[0].title : `You have ${count} notifications`;
44+
const body = items.map((it) => `• ${it.title}`).join('\n');
45+
return { title, body, severity: 'info', count, items };
46+
}
Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { describe, it, expect } from 'vitest';
4+
import { MemoryNotificationOutbox } from './memory-outbox.js';
5+
import { NotificationDispatcher } from './dispatcher.js';
6+
import { renderDigest } from './digest-render.js';
7+
import type { NotificationDeliveryRecord } from './outbox.js';
8+
import type { MessagingChannel, Notification } from './channel.js';
9+
10+
function row(over: Partial<NotificationDeliveryRecord>): NotificationDeliveryRecord {
11+
return {
12+
id: 'd', notificationId: 'n', recipientId: 'u1', channel: 'inbox',
13+
payload: {}, partitionKey: 0, status: 'pending', attempts: 0,
14+
createdAt: 0, updatedAt: 0, ...over,
15+
};
16+
}
17+
18+
describe('renderDigest (P3b-2)', () => {
19+
it('collapses a group into one message with a per-item list', () => {
20+
const out = renderDigest([
21+
row({ notificationId: 'a', payload: { title: 'Task A assigned', body: 'do A' }, topic: 'task.assigned' }),
22+
row({ notificationId: 'b', payload: { title: 'Task B assigned' }, topic: 'task.assigned' }),
23+
row({ notificationId: 'c', payload: { title: 'Mentioned in C' }, topic: 'mention' }),
24+
]);
25+
expect(out.count).toBe(3);
26+
expect(out.title).toBe('You have 3 notifications');
27+
expect(out.body).toBe('• Task A assigned\n• Task B assigned\n• Mentioned in C');
28+
expect(out.items.map((i) => i.notificationId)).toEqual(['a', 'b', 'c']);
29+
expect(out.severity).toBe('info');
30+
});
31+
32+
it('uses the single item’s title when the window holds one notification', () => {
33+
const out = renderDigest([row({ payload: { title: 'Just one' } })]);
34+
expect(out.count).toBe(1);
35+
expect(out.title).toBe('Just one');
36+
});
37+
38+
it('falls back to the topic when an item has no title', () => {
39+
const out = renderDigest([row({ payload: {}, topic: 'system.alert' })]);
40+
expect(out.items[0].title).toBe('system.alert');
41+
});
42+
});
43+
44+
describe('MemoryNotificationOutbox — digest claim separation', () => {
45+
it('claim() skips batched rows; claimDigest() returns only them', async () => {
46+
const outbox = new MemoryNotificationOutbox(1, () => 1000);
47+
await outbox.enqueue({ notificationId: 'imm', recipientId: 'u1', channel: 'inbox', payload: {} });
48+
await outbox.enqueue({ notificationId: 'g1', recipientId: 'u1', channel: 'inbox', payload: {}, digestKey: 'u1|inbox|w', notBefore: 500 });
49+
50+
const normal = await outbox.claim({ nodeId: 'n', limit: 10, claimTtlMs: 1000 });
51+
expect(normal.map((r) => r.notificationId)).toEqual(['imm']);
52+
53+
const digest = await outbox.claimDigest({ nodeId: 'n', limit: 10, claimTtlMs: 1000 });
54+
expect(digest.map((r) => r.notificationId)).toEqual(['g1']);
55+
});
56+
57+
it('claimDigest() defers a batched row until its window opens', async () => {
58+
let now = 100;
59+
const outbox = new MemoryNotificationOutbox(1, () => now);
60+
await outbox.enqueue({ notificationId: 'g', recipientId: 'u1', channel: 'inbox', payload: {}, digestKey: 'u1|inbox|w', notBefore: 1000 });
61+
expect(await outbox.claimDigest({ nodeId: 'n', limit: 10, claimTtlMs: 1000 })).toHaveLength(0);
62+
now = 1000;
63+
expect(await outbox.claimDigest({ nodeId: 'n', limit: 10, claimTtlMs: 1000 })).toHaveLength(1);
64+
});
65+
});
66+
67+
/** A channel that records every notification it is asked to send. */
68+
function recordingChannel(id: string): { channel: MessagingChannel; sent: Notification[] } {
69+
const sent: Notification[] = [];
70+
const channel: MessagingChannel = {
71+
id,
72+
async send(_ctx, req) { sent.push(req.notification); return { ok: true }; },
73+
classifyError: () => 'retryable',
74+
};
75+
return { channel, sent };
76+
}
77+
78+
function dispatcher(outbox: MemoryNotificationOutbox, channels: MessagingChannel[], now: () => number) {
79+
return new NotificationDispatcher({
80+
nodeId: 'node-test',
81+
outbox,
82+
channels: { getChannel: (cid: string) => channels.find((c) => c.id === cid) },
83+
channelContext: { logger: { info() {}, warn() {}, error() {} } },
84+
rng: () => 0.5,
85+
now,
86+
partitionCount: 1,
87+
intervalMs: 10_000,
88+
});
89+
}
90+
91+
describe('NotificationDispatcher — digest collapse (P3b-2)', () => {
92+
it('collapses a window into one message at window time and sends normal rows immediately', async () => {
93+
let now = Date.UTC(2026, 0, 1, 9, 0);
94+
const windowAt = Date.UTC(2026, 0, 2, 0, 0);
95+
const outbox = new MemoryNotificationOutbox(1, () => now);
96+
const key = 'u1|inbox|2026-01-01';
97+
for (let i = 0; i < 3; i++) {
98+
await outbox.enqueue({ notificationId: `n${i}`, recipientId: 'u1', channel: 'inbox', payload: { title: `Item ${i}` }, digestKey: key, notBefore: windowAt });
99+
}
100+
await outbox.enqueue({ notificationId: 'imm', recipientId: 'u1', channel: 'inbox', payload: { title: 'Immediate' } });
101+
102+
const rec = recordingChannel('inbox');
103+
const d = dispatcher(outbox, [rec.channel], () => now);
104+
105+
// Before the window: only the immediate row sends; the batch waits.
106+
await d.tick();
107+
expect(rec.sent.map((n) => n.title)).toEqual(['Immediate']);
108+
109+
// At the window: the 3 batched rows collapse into one message.
110+
now = windowAt;
111+
await d.tick();
112+
expect(rec.sent).toHaveLength(2);
113+
const digest = rec.sent[1];
114+
expect(digest.title).toBe('You have 3 notifications');
115+
expect(digest.payload?.digest).toBe(true);
116+
expect(digest.payload?.count).toBe(3);
117+
118+
// All three batched rows are acked success by the single send.
119+
const success = await outbox.list({ status: 'success' });
120+
expect(success.filter((r) => r.digestKey === key)).toHaveLength(3);
121+
});
122+
123+
it('re-defers the whole group when the digest send fails', async () => {
124+
let now = 1000;
125+
const outbox = new MemoryNotificationOutbox(1, () => now);
126+
const key = 'u1|inbox|w';
127+
for (let i = 0; i < 2; i++) {
128+
await outbox.enqueue({ notificationId: `n${i}`, recipientId: 'u1', channel: 'inbox', payload: { title: `Item ${i}` }, digestKey: key, notBefore: 1000 });
129+
}
130+
const failing: MessagingChannel = { id: 'inbox', async send() { return { ok: false, error: 'boom' }; }, classifyError: () => 'retryable' };
131+
const d = dispatcher(outbox, [failing], () => now);
132+
133+
await d.tick();
134+
// Both rows go back to pending with a future next_attempt_at (retry), not success.
135+
const pending = await outbox.list({ status: 'pending' });
136+
expect(pending.filter((r) => r.digestKey === key)).toHaveLength(2);
137+
expect(pending.every((r) => (r.nextAttemptAt ?? 0) > now)).toBe(true);
138+
});
139+
});

packages/services/service-messaging/src/dispatcher.ts

Lines changed: 82 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
import type { MessagingChannel, MessagingChannelContext, Notification, SendResult } from './channel.js';
44
import type { INotificationOutbox, NotificationDeliveryRecord } from './outbox.js';
55
import { classifyDeliveryAttempt } from './backoff.js';
6+
import { renderDigest } from './digest-render.js';
67

78
/** Minimal channel-registry surface the dispatcher needs (MessagingService satisfies it). */
89
export interface ChannelRegistry {
@@ -166,17 +167,81 @@ export class NotificationDispatcher {
166167
partition: { index, count: this.opts.partitionCount },
167168
claimTtlMs: this.opts.claimTtlMs,
168169
});
169-
if (claimed.length === 0) return;
170-
await handle.renew?.(this.opts.lockTtlMs);
171-
for (const row of claimed) {
172-
if (handle.isHeld && !handle.isHeld()) break;
173-
await this.processRow(row);
170+
if (claimed.length > 0) {
171+
await handle.renew?.(this.opts.lockTtlMs);
172+
for (const row of claimed) {
173+
if (handle.isHeld && !handle.isHeld()) break;
174+
await this.processRow(row);
175+
}
176+
}
177+
178+
// P3b-2 digest pass: collapse due batched rows by group. Runs under
179+
// the same partition lock — a window's rows share a partition (keyed
180+
// on digest_key), so exactly one node assembles each digest.
181+
const digestRows = await this.opts.outbox.claimDigest({
182+
nodeId: this.opts.nodeId,
183+
limit: this.opts.batchSize,
184+
partition: { index, count: this.opts.partitionCount },
185+
claimTtlMs: this.opts.claimTtlMs,
186+
});
187+
if (digestRows.length > 0) {
188+
await handle.renew?.(this.opts.lockTtlMs);
189+
for (const group of groupByDigestKey(digestRows)) {
190+
if (handle.isHeld && !handle.isHeld()) break;
191+
await this.processDigestGroup(group);
192+
}
174193
}
175194
} finally {
176195
await handle.release();
177196
}
178197
}
179198

199+
/**
200+
* Send one collapsed message for a `(recipient, channel, window)` group and
201+
* ack every row in it with that one outcome. On failure the whole group
202+
* re-defers together (each row keeps its own backoff via its `attempts`).
203+
*/
204+
private async processDigestGroup(rows: NotificationDeliveryRecord[]): Promise<void> {
205+
const channelName = rows[0].channel;
206+
const recipient = rows[0].recipientId;
207+
const channel = this.opts.channels.getChannel(channelName);
208+
if (!channel) {
209+
for (const row of rows) {
210+
await this.opts.outbox.ack(row.id, { success: false, error: `channel '${channelName}' not registered`, dead: true });
211+
this.opts.onAttempt?.(row, false);
212+
}
213+
return;
214+
}
215+
216+
const digest = renderDigest(rows);
217+
const notification: Notification = {
218+
notificationId: rows[0].notificationId, // representative event id
219+
organizationId: rows[0].organizationId,
220+
topic: rows[0].topic,
221+
title: digest.title,
222+
body: digest.body,
223+
severity: 'info',
224+
recipients: [recipient],
225+
channels: [channelName],
226+
payload: { digest: true, count: digest.count, items: digest.items },
227+
};
228+
229+
let result: SendResult;
230+
try {
231+
result = await channel.send(this.opts.channelContext, { notification, channel: channelName, recipient });
232+
} catch (err) {
233+
result = { ok: false, error: (err as Error)?.message ?? String(err) };
234+
}
235+
236+
const errorClass = !result.ok && channel.classifyError ? channel.classifyError(result.error) : undefined;
237+
const now = this.opts.now?.() ?? Date.now();
238+
for (const row of rows) {
239+
const ack = classifyDeliveryAttempt(result, errorClass, row.attempts, now, this.opts.rng);
240+
await this.opts.outbox.ack(row.id, ack);
241+
this.opts.onAttempt?.(row, result.ok);
242+
}
243+
}
244+
180245
private async processRow(row: NotificationDeliveryRecord): Promise<void> {
181246
const channel = this.opts.channels.getChannel(row.channel);
182247
if (!channel) {
@@ -223,6 +288,18 @@ export class NotificationDispatcher {
223288
}
224289
}
225290

291+
/** Group claimed digest rows by their `digestKey` (insertion order preserved). */
292+
function groupByDigestKey(rows: NotificationDeliveryRecord[]): NotificationDeliveryRecord[][] {
293+
const groups = new Map<string, NotificationDeliveryRecord[]>();
294+
for (const r of rows) {
295+
const key = r.digestKey ?? r.id; // defensive — claimDigest only returns keyed rows
296+
let g = groups.get(key);
297+
if (!g) { g = []; groups.set(key, g); }
298+
g.push(r);
299+
}
300+
return [...groups.values()];
301+
}
302+
226303
/** Spread the starting partition per node so contention rotates fairly. */
227304
function stableNodeOffset(nodeId: string, partitionCount: number): number {
228305
let h = 0;

0 commit comments

Comments
 (0)