Skip to content

Commit dea4d33

Browse files
os-zhuangclaude
andcommitted
docs(adr): ADR-0071 — add fan-out/to-one boundary + industry alignment
Bake in the expert refinements: restrict multi-hop to to-one relationships (lookup/master_detail) so chains never fan out and SUM stays correct with no symmetric-aggregate machinery; to-many rollups explicitly out of scope. Add a Prior art section (Salesforce report types, Looker Explores, Power BI/Tableau relationships, Cube transitive joins; the Looker/Tableau fan-out lesson). Note dotted paths are self-disambiguating; depth cap is now a perf guard not a correctness one; matrix column cap is a display ceiling. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent a605666 commit dea4d33

1 file changed

Lines changed: 59 additions & 16 deletions

File tree

docs/adr/0071-dataset-semantic-layer-depth.md

Lines changed: 59 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -29,29 +29,61 @@ Two capability gaps remain, both flagged as follow-ups in ADR-0021:
2929
`columns` dimension into a 2-D grid (ADR-0021 D2 follow-up). A true matrix
3030
(rows × distinct-column-values × measure cells) isn't rendered.
3131

32+
## Prior art / industry alignment
33+
34+
- **Curated join graph, not arbitrary SQL.** Salesforce *report types* pre-wire
35+
a primary object + up to 3 related objects; Looker *Explores* declare the
36+
joins a developer blesses; Power BI / Tableau model **relationships** and the
37+
engine resolves the path; Cube (this platform's runtime) resolves **transitive
38+
joins** from pairwise relationship declarations. All keep joins governed and
39+
derived from the object graph — exactly ADR-0021's D-C stance. This ADR stays
40+
in that lane (no `ON` clauses).
41+
- **Spanning depth.** Salesforce report types reach 4 objects (≈3 to-one hops);
42+
SOQL allows 5 levels of parent traversal. A small, fixed depth covers the vast
43+
majority of real reports.
44+
- **The fan-out lesson.** Looker (*symmetric aggregates*) and Tableau (*relationships
45+
vs. joins*) both exist because joining across a **to-many** edge and then
46+
aggregating **double-counts**. The robust answer for a governed, AI-authored,
47+
RLS-scoped layer is to **not** traverse to-many in the join chain at all (see
48+
Decision A) rather than to build symmetric-aggregate machinery now.
49+
- **Matrix.** Salesforce matrix reports allow ≤2 row groupings + ≤2 column
50+
groupings; Power BI's matrix visual pivots with overflow handling. P1 targets
51+
the single-column-grouping MVP with a display-side overflow cap.
52+
3253
## Decision
3354

3455
### A. Multi-hop joins (framework: spec + compiler + strategy + RLS)
3556

57+
- **To-one only (the correctness boundary).** Multi-hop traversal is restricted
58+
to **to-one** relationships — `lookup` / `master_detail` (child→parent), which
59+
is exactly what `include` holds today. To-one chains **never fan out**, so
60+
existing `SUM`/`COUNT`/etc. stay correct with **zero** symmetric-aggregate
61+
machinery. Traversing a **to-many** (child) relationship inside a dataset's
62+
join chain is explicitly **out of scope** (it needs symmetric aggregates /
63+
sub-query rollups — a separate feature). Consequently the depth cap below is a
64+
**performance/complexity** guard, not a correctness one.
3665
- **Spec.** `Dataset.include` accepts relationship **paths** — dotted chains of
37-
lookup/master_detail field names from the base object (`'account'`,
66+
to-one relationship field names from the base object (`'account'`,
3867
`'account.owner'`). A dimension/measure `field` may reference any field
39-
reachable along a **declared** path (`account.owner.region`). A depth cap
40-
(default **3 hops**) bounds join count/perf; undeclared paths are still
41-
rejected (D-C unchanged — only declared relationships are joinable).
68+
reachable along a **declared** path (`account.owner.region`). The dotted path
69+
**is** the join alias, so paths are **self-disambiguating** (no named-join
70+
disambiguation like Power BI active/inactive or Looker aliased joins). A depth
71+
cap (default **3 hops** → 4 objects, Salesforce-report-type parity) bounds join
72+
count/perf; undeclared paths are still rejected (D-C unchanged).
4273
- **Compiler** (`dataset-compiler`). Expand each `include` path into the ordered
4374
join chain; `cube.joins` is keyed by the full path alias (`account.owner`) and
4475
carries `{ parentAlias, fkField, targetObject }` so the chain is reconstructable.
4576
- **Strategy** (`native-sql-strategy.qualifyAndRegisterJoin`). For `a.b.c`,
4677
register the chain (`base → a` on alias `a`; `a → b` on alias `a.b`) and
47-
qualify the column as `"a.b"."c"`. Emit `LEFT JOIN`s in dependency order. Base
48-
columns stay qualified with the base table (the fix from objectui#…/framework
49-
joined-column work) so shared column names remain unambiguous across all hops.
78+
qualify the column as `"a.b"."c"`. Emit `LEFT JOIN`s (outer — base rows
79+
without a related record still appear, the report-friendly default) in
80+
dependency order. Base columns stay qualified with the base table so shared
81+
column names remain unambiguous across all hops.
5082
- **RLS (D-C, fail-closed).** Apply the tenant read-scope to **every** object in
5183
the chain, not just the first hop. The strategy already scopes per joined
5284
alias; generalize the loop to each hop's target object.
5385
- **UI** (objectui). `useDatasetFieldCatalog` lazily expands a relationship's own
54-
relationships (one more level on demand) so the field picker offers
86+
**to-one** relationships (one more level on demand) so the field picker offers
5587
`account.owner.region`; the include editor shows/edits paths; the existing
5688
`missingRelationship` author-time validation generalizes to paths.
5789

@@ -62,16 +94,19 @@ Two capability gaps remain, both flagged as follow-ups in ADR-0021:
6294
- **Renderer** (`DatasetReportRenderer`). When `type === 'matrix'` and `columns`
6395
is non-empty, **pivot** the flat result into a 2-D grid: distinct `columns`
6496
values become column headers, `rows` go down the side, measure(s) fill the
65-
cells. Cap distinct column values (default **50**) with a "+N more — refine
66-
with a filter" notice to avoid column explosion.
97+
cells. Cap distinct column values at a **display** ceiling (default **50**,
98+
with a "+N more — refine with a filter" notice) — a render-side guard, not a
99+
query limit. MVP = a single column grouping; ≤2 row/col groupings (Salesforce
100+
parity) is a fast-follow.
67101
- **Spec unchanged** (`rows` / `columns` already exist on `Report`).
68102

69103
## Consequences
70104

71-
- New capability: analytics two-plus relationships deep, and true matrix reports.
105+
- New capability: analytics two-plus to-one hops deep, and true matrix reports.
72106
- Cost/risk: multi-hop joins multiply `LEFT JOIN`s (perf) and widen the RLS
73107
surface (each hop scoped) — bounded by the depth cap + declared-path allowlist.
74-
Matrix column cardinality is bounded by the column cap.
108+
No fan-out risk by construction (to-one only). Matrix column cardinality is
109+
bounded by the display cap.
75110
- **Backward compatible.** Single-hop `include` and flat rows keep working
76111
unchanged; the depth cap and "flat unless `matrix` + `columns`" defaults
77112
preserve current behaviour.
@@ -81,18 +116,26 @@ Two capability gaps remain, both flagged as follow-ups in ADR-0021:
81116
- **P1 — matrix pivot (objectui-only, lowest risk).** Render the matrix grid from
82117
the existing `rows`/`columns`. No spec/query change. Ships independently.
83118
- **P2 — multi-hop (framework).** Spec path support + compiler chain + strategy
84-
chained joins + per-hop RLS, behind the depth cap. Gated by the ADR-0021
85-
reconciliation harness (old-vs-new numbers).
119+
chained joins + per-hop RLS, behind the depth cap, **to-one only**. Gated by
120+
the ADR-0021 reconciliation harness (old-vs-new numbers).
86121
- **P3 — objectui catalog/UI.** Multi-hop field picker + include-path editor +
87122
path-aware author-time validation.
88123

89124
## Alternatives considered
90125

91126
- **Arbitrary join predicates / raw SQL** — rejected (ADR-0021 D-C: joins are
92127
derived from the object graph, no `ON` clauses; keeps datasets reviewable and
93-
RLS-safe).
128+
RLS-safe). Matches Salesforce/Cube/Power BI, not the developer-only Looker
129+
`sql_on`.
130+
- **To-many traversal with symmetric aggregates** — deferred. Correct
131+
cross-to-many aggregation (Looker-style symmetric aggregates) is a separate,
132+
larger feature; the to-one boundary delivers the common case safely first.
133+
- **Edge-graph + automatic path resolution** (declare relationship edges, let the
134+
engine find the path — Cube/Power BI style) — rejected for v1 in favour of
135+
explicit dotted paths: smaller delta, self-disambiguating, and per-path
136+
RLS-auditable.
94137
- **Materialized / precomputed pivots** — out of scope; query-time pivot is
95-
sufficient at expected cardinalities (bounded by the column cap).
138+
sufficient at expected cardinalities (bounded by the display cap).
96139

97140
## Related
98141

0 commit comments

Comments
 (0)