Skip to content

Commit 4c25370

Browse files
kevin-dpautofix-ci[bot]claude
authored
Support parent-referencing WHERE filters in includes child queries (#1307)
* Unit tests for filtering on parent fields in child query * ci: apply automated fixes * Support parent-referencing WHERE filters in includes child queries Allow child queries to have additional WHERE clauses that reference parent fields (e.g., eq(i.createdBy, p.createdBy)) beyond the single correlation eq(). Parent-referencing WHEREs are detected in the builder, parent fields are projected into the key stream, and filters are re-injected into the child query where parent context is available. When no parent-referencing filters exist, behavior is unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: apply automated fixes * Extract correlation condition from inside and() WHERE clauses When users write a single .where() with and(eq(i.projectId, p.id), ...), the correlation eq() is now found and extracted from inside the and(). The remaining args stay as WHERE clauses. This means users don't need to know that the correlation must be a separate .where() call. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: apply automated fixes * Some more tests * changeset * Add failing test for shared correlation key with distinct parent filter values Two parents share the same correlation key (groupId) but have different values for a parent-referenced filter field (createdBy). The test verifies that each parent receives its own filtered child set rather than a shared union. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: apply automated fixes * Key child collections by composite routing key to fix shared correlation key collision When multiple parents share the same correlation key but have different parent-referenced filter values, child collections were incorrectly shared. Fix by keying child collections by (correlationKey, parentFilterValues) composite, and using composite child keys in the D2 stream to prevent collisions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add failing test for shared correlation key with orderBy + limit Reproduces the bug where grouped ordering for limit uses the raw correlation key instead of the composite routing key, causing parents that share a correlation key but differ on parent-referenced filters to have their children merged before the limit is applied. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: apply automated fixes * Use composite routing key for grouped ordering with limit/offset The includesGroupKeyFn for orderBy + limit/offset was grouping by raw correlationKey, causing parents sharing a correlation key but differing on parent-referenced filters to have their children merged before the limit was applied. Use the same composite key as the routing layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add failing test for nested includes with parent-referencing filters at both levels When both the child and grandchild includes use parent-referencing filters, the grandchild collection comes back empty because the nested routing index uses a different key than the nested buffer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Use composite routing key in nested routing index to match nested buffer keys The nested routing index was keyed by raw correlationKey while nested buffers use computeRoutingKey(correlationKey, parentContext). This mismatch caused drainNestedBuffers lookups to fail, leaving grandchild collections empty when parent-referencing filters exist at both levels. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add test for three levels of nested includes with parent-referencing filters Verifies that composite routing keys work at arbitrary nesting depth, not just the first two levels. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: apply automated fixes * Add test for deleting one parent preserving sibling parent's child collection Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix shared correlation key: deduplicate parentKeyStream and defer child cleanup Two fixes for when multiple parents share the same correlation key: 1. Add reduce operator on parentKeyStream to clamp multiplicities to 1, preventing the inner join from producing duplicate child entries that cause incorrect deletions when one parent is removed. 2. In Phase 5, only delete child registry entry when the last parent referencing it is removed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add test for spread select on child not leaking internal properties Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Strip internal __correlationKey and __parentContext from child results These routing properties leak into user-visible results when the child query uses a spread select (e.g. { ...i }). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: fix prettier formatting in compiler Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 99fb817 commit 4c25370

6 files changed

Lines changed: 1190 additions & 102 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@tanstack/db': patch
3+
---
4+
5+
feat: support parent-referencing WHERE filters in includes child queries

packages/db/src/query/builder/index.ts

Lines changed: 136 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ import type {
3333
OrderBy,
3434
OrderByDirection,
3535
QueryIR,
36+
Where,
3637
} from '../ir.js'
3738
import type {
3839
CompareOptions,
@@ -894,6 +895,39 @@ function buildNestedSelect(obj: any, parentAliases: Array<string> = []): any {
894895
return out
895896
}
896897

898+
/**
899+
* Recursively collects all PropRef nodes from an expression tree.
900+
*/
901+
function collectRefsFromExpression(expr: BasicExpression): Array<PropRef> {
902+
const refs: Array<PropRef> = []
903+
switch (expr.type) {
904+
case `ref`:
905+
refs.push(expr)
906+
break
907+
case `func`:
908+
for (const arg of (expr as any).args ?? []) {
909+
refs.push(...collectRefsFromExpression(arg))
910+
}
911+
break
912+
default:
913+
break
914+
}
915+
return refs
916+
}
917+
918+
/**
919+
* Checks whether a WHERE clause references any parent alias.
920+
*/
921+
function referencesParent(where: Where, parentAliases: Array<string>): boolean {
922+
const expr =
923+
typeof where === `object` && `expression` in where
924+
? where.expression
925+
: where
926+
return collectRefsFromExpression(expr).some(
927+
(ref) => ref.path[0] != null && parentAliases.includes(ref.path[0]),
928+
)
929+
}
930+
897931
/**
898932
* Builds an IncludesSubquery IR node from a child query builder.
899933
* Extracts the correlation condition from the child's WHERE clauses by finding
@@ -915,10 +949,12 @@ function buildIncludesSubquery(
915949
}
916950
}
917951

918-
// Walk child's WHERE clauses to find the correlation condition
952+
// Walk child's WHERE clauses to find the correlation condition.
953+
// The correlation eq() may be a standalone WHERE or nested inside a top-level and().
919954
let parentRef: PropRef | undefined
920955
let childRef: PropRef | undefined
921956
let correlationWhereIndex = -1
957+
let correlationAndArgIndex = -1 // >= 0 when found inside an and()
922958

923959
if (childQuery.where) {
924960
for (let i = 0; i < childQuery.where.length; i++) {
@@ -928,16 +964,15 @@ function buildIncludesSubquery(
928964
? where.expression
929965
: where
930966

931-
// Look for eq(a, b) where one side references parent and other references child
967+
// Try standalone eq()
932968
if (
933969
expr.type === `func` &&
934970
expr.name === `eq` &&
935971
expr.args.length === 2
936972
) {
937-
const [argA, argB] = expr.args
938973
const result = extractCorrelation(
939-
argA!,
940-
argB!,
974+
expr.args[0]!,
975+
expr.args[1]!,
941976
parentAliases,
942977
childAliases,
943978
)
@@ -948,6 +983,37 @@ function buildIncludesSubquery(
948983
break
949984
}
950985
}
986+
987+
// Try inside top-level and()
988+
if (
989+
expr.type === `func` &&
990+
expr.name === `and` &&
991+
expr.args.length >= 2
992+
) {
993+
for (let j = 0; j < expr.args.length; j++) {
994+
const arg = expr.args[j]!
995+
if (
996+
arg.type === `func` &&
997+
arg.name === `eq` &&
998+
arg.args.length === 2
999+
) {
1000+
const result = extractCorrelation(
1001+
arg.args[0]!,
1002+
arg.args[1]!,
1003+
parentAliases,
1004+
childAliases,
1005+
)
1006+
if (result) {
1007+
parentRef = result.parentRef
1008+
childRef = result.childRef
1009+
correlationWhereIndex = i
1010+
correlationAndArgIndex = j
1011+
break
1012+
}
1013+
}
1014+
}
1015+
if (parentRef) break
1016+
}
9511017
}
9521018
}
9531019

@@ -959,19 +1025,81 @@ function buildIncludesSubquery(
9591025
)
9601026
}
9611027

962-
// Remove the correlation WHERE from the child query
1028+
// Remove the correlation eq() from the child query's WHERE clauses.
1029+
// If it was inside an and(), remove just that arg (collapsing the and() if needed).
9631030
const modifiedWhere = [...childQuery.where!]
964-
modifiedWhere.splice(correlationWhereIndex, 1)
1031+
if (correlationAndArgIndex >= 0) {
1032+
const where = modifiedWhere[correlationWhereIndex]!
1033+
const expr =
1034+
typeof where === `object` && `expression` in where
1035+
? where.expression
1036+
: where
1037+
const remainingArgs = (expr as any).args.filter(
1038+
(_: any, idx: number) => idx !== correlationAndArgIndex,
1039+
)
1040+
if (remainingArgs.length === 1) {
1041+
// Collapse and() with single remaining arg to just that expression
1042+
const isResidual =
1043+
typeof where === `object` && `expression` in where && where.residual
1044+
modifiedWhere[correlationWhereIndex] = isResidual
1045+
? { expression: remainingArgs[0], residual: true }
1046+
: remainingArgs[0]
1047+
} else {
1048+
// Rebuild and() without the extracted arg
1049+
const newAnd = new FuncExpr(`and`, remainingArgs)
1050+
const isResidual =
1051+
typeof where === `object` && `expression` in where && where.residual
1052+
modifiedWhere[correlationWhereIndex] = isResidual
1053+
? { expression: newAnd, residual: true }
1054+
: newAnd
1055+
}
1056+
} else {
1057+
modifiedWhere.splice(correlationWhereIndex, 1)
1058+
}
1059+
1060+
// Separate remaining WHEREs into pure-child vs parent-referencing
1061+
const pureChildWhere: Array<Where> = []
1062+
const parentFilters: Array<Where> = []
1063+
for (const w of modifiedWhere) {
1064+
if (referencesParent(w, parentAliases)) {
1065+
parentFilters.push(w)
1066+
} else {
1067+
pureChildWhere.push(w)
1068+
}
1069+
}
1070+
1071+
// Collect distinct parent PropRefs from parent-referencing filters
1072+
let parentProjection: Array<PropRef> | undefined
1073+
if (parentFilters.length > 0) {
1074+
const seen = new Set<string>()
1075+
parentProjection = []
1076+
for (const w of parentFilters) {
1077+
const expr = typeof w === `object` && `expression` in w ? w.expression : w
1078+
for (const ref of collectRefsFromExpression(expr)) {
1079+
if (
1080+
ref.path[0] != null &&
1081+
parentAliases.includes(ref.path[0]) &&
1082+
!seen.has(ref.path.join(`.`))
1083+
) {
1084+
seen.add(ref.path.join(`.`))
1085+
parentProjection.push(ref)
1086+
}
1087+
}
1088+
}
1089+
}
1090+
9651091
const modifiedQuery: QueryIR = {
9661092
...childQuery,
967-
where: modifiedWhere.length > 0 ? modifiedWhere : undefined,
1093+
where: pureChildWhere.length > 0 ? pureChildWhere : undefined,
9681094
}
9691095

9701096
return new IncludesSubquery(
9711097
modifiedQuery,
9721098
parentRef,
9731099
childRef,
9741100
fieldName,
1101+
parentFilters.length > 0 ? parentFilters : undefined,
1102+
parentProjection,
9751103
materializeAsArray,
9761104
)
9771105
}

0 commit comments

Comments
 (0)