Skip to content

Commit 4d5a892

Browse files
os-zhuangos-zhuang
authored
feat(objectql): filtered roll-up summary fields (#1868) (#3227)
Add optional `summaryOperations.filter` (a query `where` FilterCondition) so a roll-up `summary` field aggregates only the child rows matching a predicate — letting one child object feed several distinct parent totals. Threads the filter through the engine's SummaryDescriptor/buildSummaryIndex/recomputeSummaries (ANDed with the parent-FK match), plus schema + tests, regenerated reference docs/skill indexes, docs, a showcase expense-report example, and a changeset. Purely additive; omitting filter aggregates every child as before. Closes #1868.
1 parent bd7df45 commit 4d5a892

15 files changed

Lines changed: 472 additions & 10 deletions

File tree

‎.changeset/summary-rollup-filter.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/objectql': minor
4+
---
5+
6+
feat(objectql): roll-up `summary` fields can filter which child rows they aggregate (#1868)
7+
8+
`summaryOperations` gains an optional `filter` — a query `where` FilterCondition
9+
evaluated against each child row, so a summary aggregates only the matching
10+
children instead of the whole collection. This is what lets a single child object
11+
feed several distinct parent totals, which the cross-object rollup templates need:
12+
13+
```typescript
14+
// One `engagement` child → distinct filtered totals.
15+
total_signups: {
16+
type: 'summary',
17+
summaryOperations: { object: 'engagement', field: 'id', function: 'count', filter: { type: 'signup' } },
18+
}
19+
// Sum only received receipt lines (3-way match).
20+
received_amount: {
21+
type: 'summary',
22+
summaryOperations: { object: 'procurement_receipt', field: 'amount', function: 'sum', filter: { status: 'received' } },
23+
}
24+
```
25+
26+
The engine ANDs the predicate with the parent-FK match when it recomputes, and
27+
because the whole filtered aggregate is re-run on every child write, a child that
28+
moves in or out of the predicate (e.g. a status change) keeps the parent current
29+
with no extra wiring. Operator and compound forms work too
30+
(`filter: { type: { $in: ['signup', 'trial'] }, amount: { $gte: 100 } }`).
31+
32+
Purely additive: omitting `filter` aggregates every child exactly as before.

‎content/docs/data-modeling/field-types.mdx‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -447,6 +447,7 @@ Roll-up summary from child records.
447447
| `summaryOperations.field` | `string` | **required** | Child field to aggregate; ignored for `count` |
448448
| `summaryOperations.function` | `enum` | **required** | `count`, `sum`, `avg`, `min`, or `max` |
449449
| `summaryOperations.relationshipField` | `string` | auto | Child FK back to the parent when it cannot be inferred |
450+
| `summaryOperations.filter` | `FilterCondition` | — | Only child rows matching this `where` predicate are aggregated |
450451

451452
```typescript
452453
{
@@ -462,6 +463,32 @@ Roll-up summary from child records.
462463
}
463464
```
464465
466+
Add a `filter` to aggregate only a subset of children — this is how one child
467+
object feeds several different totals. The engine ANDs the predicate with the
468+
parent-FK match and recomputes on the child's next write, so a row moving in or
469+
out of the filter (a status change) keeps the parent current:
470+
471+
```typescript
472+
// Sum only received receipt lines (3-way match), not every line.
473+
{
474+
name: 'received_amount',
475+
type: 'summary',
476+
summaryOperations: {
477+
object: 'procurement_receipt',
478+
field: 'amount',
479+
function: 'sum',
480+
filter: { status: 'received' },
481+
},
482+
}
483+
484+
// One `engagement` child → distinct totals, differentiated only by the filter.
485+
{
486+
name: 'total_signups',
487+
type: 'summary',
488+
summaryOperations: { object: 'engagement', field: 'id', function: 'count', filter: { type: 'signup' } },
489+
}
490+
```
491+
465492
### `autonumber`
466493
Auto-incrementing number with format template.
467494

‎content/docs/references/data/field.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ const result = Address.parse(data);
132132
| **allowCreate** | `boolean` | optional | Allow inline quick-create from the record picker: when no match exists the user can create a record from the typed text (optimistic dataSource.create with the display field). Best for simple objects whose only required field is the display field. |
133133
| **expression** | `string \| { dialect: Enum<'cel' \| 'js' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | optional | Formula expression (CEL). e.g. F`record.amount * 0.1` |
134134
| **returnType** | `Enum<'number' \| 'text' \| 'boolean' \| 'date'>` | optional | Inferred value type of a formula field (number/text/boolean/date) |
135-
| **summaryOperations** | `{ object: string; field: string; function: Enum<'count' \| 'sum' \| 'min' \| 'max' \| 'avg'>; relationshipField?: string }` | optional | Roll-up summary definition. The engine recomputes the value when child records are inserted/updated/deleted. |
135+
| **summaryOperations** | `{ object: string; field: string; function: Enum<'count' \| 'sum' \| 'min' \| 'max' \| 'avg'>; relationshipField?: string; … }` | optional | Roll-up summary definition. The engine recomputes the value when child records are inserted/updated/deleted. |
136136
| **language** | `string` | optional | Programming language for syntax highlighting (e.g., javascript, python, sql) |
137137
| **step** | `number` | optional | Step increment for slider (default: 1) |
138138
| **currencyConfig** | `{ precision?: integer; currencyMode?: Enum<'dynamic' \| 'fixed'>; defaultCurrency?: string }` | optional | Configuration for currency field type |

‎examples/app-showcase/src/data/objects/expense-report.object.ts‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { ObjectSchema, Field } from '@objectstack/spec/data';
4+
5+
/**
6+
* Expense Report + Expense Line — the canonical demonstration of **filtered**
7+
* roll-up `summary` fields (framework#1868).
8+
*
9+
* A plain `summary` rolls up EVERY child row (see `showcase_invoice.total`).
10+
* The power added by `summaryOperations.filter` is that ONE child object can
11+
* feed SEVERAL different parent totals, each aggregating only the child rows
12+
* that match a predicate — something that was impossible to express before and
13+
* had to be hand-maintained (and drifted the moment a real child row was added).
14+
*
15+
* Here a single `showcase_expense_line` child feeds six rollups on the report:
16+
*
17+
* • total_amount SUM(amount) — every line
18+
* • approved_amount SUM(amount) WHERE status=approved — filtered sum (equality)
19+
* • reimbursable_amount SUM(amount) WHERE billable=true — filtered sum (boolean)
20+
* • line_count COUNT — every line
21+
* • rejected_count COUNT WHERE status=rejected — filtered count (equality)
22+
* • over_limit_count COUNT WHERE amount>=500 — filtered count (operator)
23+
*
24+
* The engine keeps all six current server-side as lines are inserted / updated
25+
* / deleted. Because each recompute re-runs the whole filtered aggregate, a
26+
* line moving in or out of a predicate — e.g. a manager flipping one line's
27+
* `status` from `submitted` to `approved` in the inline grid — updates the
28+
* relevant totals on that write with no extra wiring. That is the interactive
29+
* story to drive in the running app: edit line statuses and watch
30+
* approved/rejected/reimbursable diverge from the unfiltered total.
31+
*/
32+
export const ExpenseReport = ObjectSchema.create({
33+
name: 'showcase_expense_report',
34+
// [ADR-0090 D1] grandfather stamp: world-writable demo object so any seeded
35+
// persona (and the browser e2e) can create/edit reports without a bespoke
36+
// permission set. Belonging to no permission set, it is intentionally absent
37+
// from the access-matrix snapshot (cf. showcase_cascade).
38+
sharingModel: 'public_read_write',
39+
label: 'Expense Report',
40+
pluralLabel: 'Expense Reports',
41+
icon: 'wallet',
42+
description: 'An expense report whose parent totals are filtered roll-ups of its line items.',
43+
44+
fields: {
45+
name: Field.text({ label: 'Report Title', required: true, searchable: true, maxLength: 120 }),
46+
employee: Field.text({ label: 'Employee', searchable: true, maxLength: 120 }),
47+
status: Field.select({
48+
label: 'Status',
49+
required: true,
50+
options: [
51+
{ label: 'Draft', value: 'draft', default: true, color: '#94A3B8' },
52+
{ label: 'Submitted', value: 'submitted', color: '#3B82F6' },
53+
{ label: 'Approved', value: 'approved', color: '#10B981' },
54+
{ label: 'Reimbursed', value: 'reimbursed', color: '#6366F1' },
55+
],
56+
}),
57+
submitted_on: Field.date({ label: 'Submitted On' }),
58+
59+
// ── Filtered roll-ups (the point of this object) ─────────────────────────
60+
// Child FK is auto-detected (showcase_expense_line.expense_report), so none
61+
// of these need `relationshipField`.
62+
63+
/** Baseline: unfiltered — the grand total of every line. */
64+
total_amount: Field.summary({
65+
label: 'Total',
66+
summaryOperations: { object: 'showcase_expense_line', field: 'amount', function: 'sum' },
67+
}),
68+
/** Filtered SUM (equality) — only lines a manager has approved. */
69+
approved_amount: Field.summary({
70+
label: 'Approved',
71+
summaryOperations: {
72+
object: 'showcase_expense_line',
73+
field: 'amount',
74+
function: 'sum',
75+
filter: { status: 'approved' },
76+
},
77+
}),
78+
/** Filtered SUM (boolean) — only lines billable back to a client. */
79+
reimbursable_amount: Field.summary({
80+
label: 'Reimbursable',
81+
summaryOperations: {
82+
object: 'showcase_expense_line',
83+
field: 'amount',
84+
function: 'sum',
85+
filter: { billable: true },
86+
},
87+
}),
88+
/** Baseline: unfiltered line count. */
89+
line_count: Field.summary({
90+
label: 'Lines',
91+
summaryOperations: { object: 'showcase_expense_line', field: 'amount', function: 'count' },
92+
}),
93+
/** Filtered COUNT (equality) — lines a manager rejected. */
94+
rejected_count: Field.summary({
95+
label: 'Rejected',
96+
summaryOperations: {
97+
object: 'showcase_expense_line',
98+
field: 'amount',
99+
function: 'count',
100+
filter: { status: 'rejected' },
101+
},
102+
}),
103+
/** Filtered COUNT (operator) — lines at or above the $500 receipt-required
104+
* threshold. Shows the FilterCondition operator form, not just equality. */
105+
over_limit_count: Field.summary({
106+
label: 'Over $500',
107+
summaryOperations: {
108+
object: 'showcase_expense_line',
109+
field: 'amount',
110+
function: 'count',
111+
filter: { amount: { $gte: 500 } },
112+
},
113+
}),
114+
},
115+
});
116+
117+
/** Expense line item — owned by its report, entered inline in the grid. */
118+
export const ExpenseLine = ObjectSchema.create({
119+
name: 'showcase_expense_line',
120+
label: 'Expense Line',
121+
pluralLabel: 'Expense Lines',
122+
icon: 'receipt',
123+
description: 'A single expense on a report; the parent report rolls these up with filters.',
124+
125+
// ADR-0055: a line's access is CONTROLLED BY ITS PARENT report — the same
126+
// master-detail pattern as showcase_invoice_line. No RLS authored here; the
127+
// security layer derives it from the required master_detail relationship.
128+
sharingModel: 'controlled_by_parent',
129+
130+
fields: {
131+
expense_report: Field.masterDetail('showcase_expense_report', {
132+
label: 'Report',
133+
required: true,
134+
deleteBehavior: 'cascade',
135+
// Thin, high-volume line items → the editable grid form factor. The
136+
// report's filtered summaries recompute as rows are saved in this grid.
137+
inlineEdit: 'grid',
138+
inlineTitle: 'Expense Lines',
139+
}),
140+
merchant: Field.text({ label: 'Merchant', required: true, maxLength: 120 }),
141+
category: Field.select({
142+
label: 'Category',
143+
options: [
144+
{ label: 'Travel', value: 'travel' },
145+
{ label: 'Meals', value: 'meals' },
146+
{ label: 'Lodging', value: 'lodging' },
147+
{ label: 'Supplies', value: 'supplies' },
148+
{ label: 'Software', value: 'software' },
149+
{ label: 'Other', value: 'other', default: true },
150+
],
151+
}),
152+
amount: Field.currency({ label: 'Amount', required: true, scale: 2, min: 0 }),
153+
// Whether the expense is billable back to a client — the `billable: true`
154+
// filter on the report's `reimbursable_amount` rollup reads this.
155+
billable: Field.boolean({ label: 'Billable to client', defaultValue: false }),
156+
// Per-line approval status — a manager approves/rejects individual lines,
157+
// which is what the report's `approved_amount` / `rejected_count` filter on.
158+
status: Field.select({
159+
label: 'Line Status',
160+
required: true,
161+
options: [
162+
{ label: 'Submitted', value: 'submitted', default: true, color: '#3B82F6' },
163+
{ label: 'Approved', value: 'approved', color: '#10B981' },
164+
{ label: 'Rejected', value: 'rejected', color: '#EF4444' },
165+
],
166+
}),
167+
incurred_on: Field.date({ label: 'Incurred On' }),
168+
},
169+
});

‎examples/app-showcase/src/data/objects/index.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ export { Category } from './category.object.js';
77
export { BusinessUnit } from './business-unit.object.js';
88
export { Team, ProjectMembership } from './team.object.js';
99
export { Product, Invoice, InvoiceLine } from './invoice.object.js';
10+
export { ExpenseReport, ExpenseLine } from './expense-report.object.js';
1011
export { FieldZoo } from './field-zoo.object.js';
1112
export { CascadingSelect } from './cascading-select.object.js';
1213
export { Preference } from './preference.object.js';

‎examples/app-showcase/src/data/seed/index.ts‎

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import { Category } from '../objects/category.object.js';
1010
import { BusinessUnit } from '../objects/business-unit.object.js';
1111
import { Team, ProjectMembership } from '../objects/team.object.js';
1212
import { Product, Invoice, InvoiceLine } from '../objects/invoice.object.js';
13+
import { ExpenseReport, ExpenseLine } from '../objects/expense-report.object.js';
1314
import { Contact } from '../objects/contact.object.js';
1415
import { Inquiry } from '../objects/inquiry.object.js';
1516
import { FieldZoo } from '../objects/field-zoo.object.js';
@@ -300,6 +301,41 @@ const invoiceLines = defineSeed(InvoiceLine, {
300301
],
301302
});
302303

304+
// Expense reports + lines — the filtered-rollup demo (framework#1868). Line
305+
// amounts, `billable`, and per-line `status` are chosen so the report's SIX
306+
// summaries show visibly DIFFERENT values on first boot: the unfiltered total
307+
// vs the approved-only sum vs the billable-only sum, etc. Drive the interactive
308+
// story in the app by flipping a line's status in the inline grid and watching
309+
// approved_amount / rejected_count move. Merchants are unique → externalId.
310+
const expenseReports = defineSeed(ExpenseReport, {
311+
mode: 'upsert',
312+
externalId: 'name',
313+
records: [
314+
{ name: 'EXP-2001', employee: 'Ada Lovelace', status: 'submitted', submitted_on: cel`daysAgo(5)` },
315+
{ name: 'EXP-2002', employee: 'Linus Torvalds', status: 'approved', submitted_on: cel`daysAgo(12)` },
316+
{ name: 'EXP-2003', employee: 'Grace Hopper', status: 'draft' },
317+
],
318+
});
319+
320+
const expenseLines = defineSeed(ExpenseLine, {
321+
mode: 'upsert',
322+
externalId: 'merchant',
323+
records: [
324+
// EXP-2001 → total 1500.50 · approved 960 · reimbursable 512 · rejected 0 · over$500 2
325+
{ merchant: 'United Airlines', expense_report: 'EXP-2001', category: 'travel', amount: 620, billable: false, status: 'approved', incurred_on: cel`daysAgo(9)` },
326+
{ merchant: 'Marriott Downtown', expense_report: 'EXP-2001', category: 'lodging', amount: 340, billable: false, status: 'approved', incurred_on: cel`daysAgo(8)` },
327+
{ merchant: 'Chipotle', expense_report: 'EXP-2001', category: 'meals', amount: 28.5, billable: false, status: 'submitted', incurred_on: cel`daysAgo(8)` },
328+
{ merchant: 'AWS', expense_report: 'EXP-2001', category: 'software', amount: 512, billable: true, status: 'submitted', incurred_on: cel`daysAgo(7)` },
329+
// EXP-2002 → total 917 · approved 825 · reimbursable 825 · rejected 1 · over$500 1
330+
{ merchant: 'Delta Air Lines', expense_report: 'EXP-2002', category: 'travel', amount: 780, billable: true, status: 'approved', incurred_on: cel`daysAgo(15)` },
331+
{ merchant: 'Uber', expense_report: 'EXP-2002', category: 'travel', amount: 45, billable: true, status: 'approved', incurred_on: cel`daysAgo(14)` },
332+
{ merchant: 'Staples', expense_report: 'EXP-2002', category: 'supplies', amount: 92, billable: false, status: 'rejected', incurred_on: cel`daysAgo(14)` },
333+
// EXP-2003 (draft) → total 225.75 · approved 0 · reimbursable 0 · rejected 0 · over$500 0
334+
{ merchant: 'Hilton Garden Inn', expense_report: 'EXP-2003', category: 'lodging', amount: 210, billable: false, status: 'submitted', incurred_on: cel`daysAgo(3)` },
335+
{ merchant: 'Starbucks', expense_report: 'EXP-2003', category: 'meals', amount: 15.75, billable: false, status: 'submitted', incurred_on: cel`daysAgo(2)` },
336+
],
337+
});
338+
303339
// Inquiries so the staff triage list (inquiry views + Contact Form page)
304340
// renders on first boot — the "every view renders real data" principle. One
305341
// per status; the `closed` row doubles as live prey for InquiryPurgeFlow.
@@ -337,4 +373,4 @@ const announcements = defineSeed(Announcement, {
337373
],
338374
});
339375

340-
export const ShowcaseSeedData = [accounts, contacts, inquiries, products, projects, tasks, categories, businessUnits, orgUnits, teams, memberships, fieldZoo, invoices, invoiceLines, preferences, announcements];
376+
export const ShowcaseSeedData = [accounts, contacts, inquiries, products, projects, tasks, categories, businessUnits, orgUnits, teams, memberships, fieldZoo, invoices, invoiceLines, expenseReports, expenseLines, preferences, announcements];

‎examples/app-showcase/src/ui/apps/index.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,9 @@ export const ShowcaseApp = App.create({
5252
{ id: 'nav_accounts', type: 'object', objectName: 'showcase_account', label: 'Accounts', icon: 'building' },
5353
{ id: 'nav_contacts', type: 'object', objectName: 'showcase_contact', label: 'Contacts', icon: 'user' },
5454
{ id: 'nav_invoices', type: 'object', objectName: 'showcase_invoice', label: 'Invoices', icon: 'receipt' },
55+
// Filtered roll-up summary demo (framework#1868): one expense-line child
56+
// feeds six parent totals, each aggregating only the lines a filter matches.
57+
{ id: 'nav_expense_reports', type: 'object', objectName: 'showcase_expense_report', label: 'Expense Reports', icon: 'wallet' },
5558
{ id: 'nav_products', type: 'object', objectName: 'showcase_product', label: 'Products', icon: 'package' },
5659
{ id: 'nav_teams', type: 'object', objectName: 'showcase_team', label: 'Teams', icon: 'users' },
5760
{ id: 'nav_categories', type: 'object', objectName: 'showcase_category', label: 'Categories', icon: 'list-tree' },

‎packages/objectql/src/engine.ts‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,12 @@ interface SummaryDescriptor {
283283
fn: 'count' | 'sum' | 'min' | 'max' | 'avg';
284284
/** Child field aggregated (unused for count). */
285285
sourceField: string;
286+
/**
287+
* Optional predicate (a query `where` FilterCondition) restricting which child
288+
* rows are aggregated. ANDed with the parent-FK match when the aggregate runs.
289+
* Undefined ⇒ aggregate every child of the parent.
290+
*/
291+
filter?: Record<string, unknown>;
286292
}
287293

288294
export class ObjectQL implements IDataEngine {
@@ -1841,8 +1847,14 @@ export class ObjectQL implements IDataEngine {
18411847
}
18421848
}
18431849
if (!fkField) continue; // can't resolve the relationship — skip
1850+
// Optional per-summary predicate: only child rows matching it are
1851+
// aggregated (e.g. sum receipts where { status: 'received' }). ANDed with
1852+
// the parent-FK match at recompute time. Ignore a non-object filter.
1853+
const filter = so.filter && typeof so.filter === 'object' && !Array.isArray(so.filter)
1854+
? so.filter as Record<string, unknown>
1855+
: undefined;
18441856
const list = index.get(childObject) ?? [];
1845-
list.push({ parentObject: parent.name, summaryField, fkField, fn, sourceField: so.field });
1857+
list.push({ parentObject: parent.name, summaryField, fkField, fn, sourceField: so.field, filter });
18461858
index.set(childObject, list);
18471859
}
18481860
}
@@ -1882,8 +1894,12 @@ export class ObjectQL implements IDataEngine {
18821894
// aggregate/update) with backoff — a network blip here used to leave
18831895
// the parent summary silently stale (framework#3147).
18841896
await withTransientRetry(async () => {
1897+
// AND the parent-FK match with the optional per-summary filter so
1898+
// only matching child rows are aggregated (e.g. received receipts).
1899+
const fkMatch = { [desc.fkField]: parentId };
1900+
const where = desc.filter ? { $and: [fkMatch, desc.filter] } : fkMatch;
18851901
const rows = await this.aggregate(childObject, {
1886-
where: { [desc.fkField]: parentId },
1902+
where,
18871903
aggregations: [{
18881904
function: desc.fn,
18891905
...(desc.fn === 'count' ? {} : { field: desc.sourceField }),

0 commit comments

Comments
 (0)