You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: guides/developer/caching.mdx
+7-1Lines changed: 7 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -93,4 +93,10 @@ The dashboard header displays the date and time of the chart with the oldest cac
93
93
94
94
<Info>
95
95
There is currently no way to invalidate cached results for individual Saved Charts.
96
-
</Info>
96
+
</Info>
97
+
98
+
## Results caching vs pre-aggregates
99
+
100
+
Results caching and [pre-aggregates](/references/pre-aggregates/overview) are complementary. Caching stores query results after the first warehouse hit; pre-aggregates materialize summary tables in advance so the warehouse is never hit at query time.
101
+
102
+
For a detailed comparison, see [Pre-aggregates vs results caching](/references/pre-aggregates/overview#pre-aggregates-vs-results-caching).
For metrics that can't be pre-aggregated, consider using [caching](/guides/developer/caching) instead.
134
+
135
+
## Pre-aggregates vs results caching
136
+
137
+
Lightdash has two independent systems for speeding up queries: **results caching** and **pre-aggregates**. They work differently and are designed to be used together, not as replacements for each other.
138
+
139
+
### Results caching
140
+
141
+
Results caching stores the exact result of any query that runs through Lightdash, keyed by a hash of the generated SQL. The first time a query runs, Lightdash executes it against your warehouse and caches the result in S3. Subsequent identical queries are served from the cache until it expires (24 hours by default).
142
+
143
+
Any change to the query — a different filter, column, limit, or user attribute — produces a new SQL hash, a new cache entry, and another warehouse query. Results caching covers every query shape, including custom metrics, table calculations, and SQL runner queries.
144
+
145
+
See the [caching guide](/guides/developer/caching) for details.
146
+
147
+
### Pre-aggregates
148
+
149
+
Pre-aggregates are summary tables you define in your dbt YAML. Lightdash materializes them on a schedule (or on compile, or manually) and stores the results in S3. When a user query matches the pre-aggregate's dimensions, metrics, filters, and granularity, Lightdash serves the query from the materialized data using in-memory DuckDB workers. The warehouse is not touched at query time, even on the first query.
150
+
151
+
A single pre-aggregate can serve many different queries. A daily pre-aggregate with five dimensions can answer day, week, month, quarter, and year queries across any subset of those dimensions and with any narrower filter. Results caching, in contrast, needs one cache entry per unique SQL.
|**Configuration**| Automatic once enabled for your instance | Defined in dbt YAML |
158
+
|**Trigger**| First query runs against warehouse, then cached | Materialized on compile, cron, or manual refresh |
159
+
|**Storage**| Query result (row set) | Pre-computed summary table |
160
+
|**Query execution**| Exact cached result is returned | DuckDB workers re-aggregate at query time |
161
+
|**Warehouse hit on first query?**| Yes | No — only materialization hits the warehouse, not query-time serving |
162
+
|**Coverage**| All metric types, all query shapes | Only re-aggregatable metrics (sum, count, min, max, average) |
163
+
|**Scope**| One cache entry per unique SQL | One pre-aggregate can serve many query shapes |
164
+
|**Availability**| Cloud Pro+ or self-hosted with license | Enterprise (Early Access) |
165
+
166
+
### When to use which
167
+
168
+
**Use pre-aggregates when:**
169
+
170
+
- You have high-traffic dashboards with predictable query patterns
171
+
- You want to reduce warehouse cost or improve latency on the first query, not just repeat visits
172
+
- The metrics are re-aggregatable (sum, count, min, max, average)
173
+
- You're willing to design and schedule the materializations
174
+
175
+
**Use results caching when:**
176
+
177
+
- Query patterns are ad-hoc or unpredictable
178
+
- You need count_distinct, median, percentile, custom SQL metrics, table calculations, or custom dimensions/metrics
179
+
- You're using the SQL runner
180
+
- You don't want upfront configuration work
181
+
182
+
<Tip>
183
+
In most cases, both should be enabled. Pre-aggregates handle your heaviest, most predictable workloads. Results caching is the safety net for everything else.
184
+
</Tip>
185
+
186
+
### Using both together
187
+
188
+
When both systems are enabled, they act as two layers of caching. A query that matches a pre-aggregate is served from the materialized data by DuckDB workers. The result of that DuckDB query can then be stored in the results cache, so subsequent identical requests skip even the DuckDB step and return the cached result directly. This means pre-aggregates eliminate the warehouse hit, and results caching eliminates repeated computation on top of that.
0 commit comments