Skip to content

Commit 66b5f7c

Browse files
committed
Udpate the query-driven-sync article
1 parent cbfc3e0 commit 66b5f7c

1 file changed

Lines changed: 36 additions & 16 deletions

File tree

src/data/deep-dives/query-driven-sync.mdx

Lines changed: 36 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,11 @@ import {
88

99
## Query-driven sync
1010

11-
Sometimes you don't want to load all your data into your collection, only the subset that is required by the live queries you're running.
11+
Sometimes you don't want to load all your data into your collection (e.g. when you have a lot of data), only the subset required by the live queries you're running.
1212

13-
For example:
13+
In this demo app, some projects have many more todo items than others. Currently, we load all todo items regardless of which project page the user visits. But what if we could load only items from the currently viewed project?
1414

15-
- instead of loading all the todo items a user has when they visit a project's page,
16-
- we can just load the ones that belong to that project.
17-
18-
This can be achieved by using the `on-demand` sync mode.
15+
Well, this is what the `on-demand` sync mode is for.
1916

2017
> FYI: the docs recommend it for large datasets (>50k rows), which means that using it is not really warranted in our case (we have a bit more than 1k rows). <a target="_blank" href="https://tanstack.com/db/latest/docs/overview#sync-modes">(source)</a>
2118
@@ -54,7 +51,7 @@ export const todoItemsCollection = createCollection<TodoItemRecord>(
5451
5552
Using this `queryFn` for the `"on-demand"` `syncMode` would defeat the very purpose of it, because it would fetch every single todo item in the database for each query.
5653
57-
Instead of that, we want the collection to load all the todo items that are required for the queries we run. Right now, we have only one query for the `todoItemsCollection`:
54+
Instead, we want the collection to load all the todo items that are required for the queries we run. Right now, we have only one query for the `todoItemsCollection`:
5855
5956
```tsx {5}
6057
const {data: allTodoItems} = useLiveQuery(
@@ -70,7 +67,7 @@ const {data: allTodoItems} = useLiveQuery(
7067
)
7168
```
7269
73-
The highlighted line in the code block above is the filter we need to use in the database query on the server. Luckily for us, when this live query runs, TanStack DB automatically passes down all the predicates coming from this live query to the `queryFn`.
70+
The highlighted line in the code block above is the filter we need to use in the database query on the server. Fortunately, when this live query runs, TanStack DB automatically passes all query predicates to the `queryFn`.
7471
7572
```ts {22}
7673
export const todoItemsCollection = createCollection<TodoItemRecord>(
@@ -102,35 +99,58 @@ export const todoItemsCollection = createCollection<TodoItemRecord>(
10299
)
103100
```
104101
105-
To observe this behavior
102+
Now the `queryFn` will only attempt to fetch the todo items whose projectId matches the one in the live query. You can quickly check it yourself:
106103
107104
- open the <OpenAPIRequestsPanelLink>API Request Panel</OpenAPIRequestsPanelLink>,
108105
- and [reload this page]()
109106
110107
You should see that the request to `/api/todo-items` has a search string attached to it, similar to this one: `?projectId=aQLI4Nzvmls31l4aep9LqnU`.
111108
109+
The only thing left to do now is to use the project's id from the search params in the API route's handler:
110+
```ts {6,9-12}
111+
export default {
112+
GET: async ({ request }) => {
113+
const url = new URL(request.url);
114+
115+
// Currently we only support filtering by projectId
116+
const projectId = url.searchParams.get("projectId");
117+
118+
if (projectId) {
119+
const results = await db
120+
.select()
121+
.from(todoItemsTable)
122+
.where(eq(todoItemsTable.projectId, projectId));
123+
return json(results);
124+
} else {
125+
const results = await db.select().from(todoItemsTable);
126+
return json(results);
127+
}
128+
},
129+
}
130+
```
131+
112132
### Query predicates
113133
114-
In this app we stop here, but the predicates (`where` clauses, `orderBy`, `limit`, and `offset`) coming from the queries can cover far more advanced use cases.
134+
In this app, we'll stop here, but the predicates (`where` clauses, `orderBy`, `limit`, and `offset`) coming from the queries can cover far more advanced use cases.
115135
116-
We could use `orderBy` to order elements on the server, `offset` to support pagination and more.
136+
We could use `orderBy` to order elements on the server, `offset` to support pagination, and more.
117137
118138
Check out the <a target="_blank" href="https://tanstack.com/db/latest/docs/collections/query-collection#queryfn-and-predicate-push-down">documentation</a> for more info about them.
119139
120140
### Transitioning from `"eager"` to `"on-demand"`: some pitfalls
121141
122-
When I first refactor the `todoItemsCollection` to use `on-demand` sync, I ran into some issues.
142+
When I first refactored `todoItemsCollection` to use `on-demand` sync, I ran into some issues.
123143
124144
#### `.toArrayWhenReady()` vs `.toArray`
125145
126146
For example, this line of code calling the <a target="_blank" href="https://tanstack.com/db/latest/docs/reference/interfaces/Collection#toarraywhenready">`.toArrayWhenReady()` method</a> caused the collection to fetch every single todo item from the remote source:
127147
128148
```ts
129-
// ❌ Get the current state of the collection from the server,
149+
// ❌ Get the current state of the collection from the server
130150
await todoItemsCollection.toArrayWhenReady()
131151
```
132152
133-
I had to change it to <a target="_blank" href="https://tanstack.com/db/latest/docs/reference/interfaces/Collection#toarray">`toArray`</a>, to use only the data that we already have in the client-side cache:
153+
I had to change it to <a target="_blank" href="https://tanstack.com/db/latest/docs/reference/interfaces/Collection#toarray">`toArray`</a>, to use only existing data in the client-side cache:
134154
135155
```ts
136156
// ✅ Get the current state of the collection from the cache
@@ -146,11 +166,11 @@ Another line that resulted in loading all the collection data instead of the sub
146166
todoItemsCollection.utils.refetch();
147167
```
148168
149-
The moral of the story is that you have to watch out and make sure you don't accidentally sync all the data in the collection when you use `"on-demand"` mode.
169+
The key takeaway is that you have to watch out and ensure you don't accidentally sync all the data in the collection when you use `"on-demand"` mode.
150170
151171
### Progressive mode
152172
153-
There is a third sync mode, that I haven't mentioned before called `"progressive"`. Here's how it relates to the other two:
173+
There is a third sync mode that I haven't previously mentioned called `"progressive"`. Here's how it relates to the other two:
154174
155175
- `"eager"`: load all the data in the collection
156176
- `"on-demand"`: load only the queried data

0 commit comments

Comments
 (0)