Skip to content

Commit 3a84d1a

Browse files
committed
docs: pre-aggregation-specific data source configuration (follow-up to #10587)
1 parent 64602ae commit 3a84d1a

3 files changed

Lines changed: 61 additions & 5 deletions

File tree

docs-mintlify/admin/connect-to-data/multiple-data-sources.mdx

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,5 +98,10 @@ CUBEJS_DS_ANALYTICS_DB_HOST=remotehost
9898
For the full list of variables that support decoration, see the
9999
[environment variables reference][ref-config-ref-env].
100100

101+
Each source can also build and store its pre-aggregations on a dedicated connection
102+
using the `CUBEJS_DS_<NAME>_PRE_AGGREGATIONS_DB_*` variables. See [pre-aggregation data
103+
source][ref-preagg-data-source] for details.
104+
101105
[ref-config-ref-env]: /reference/configuration/environment-variables
106+
[ref-preagg-data-source]: /docs/pre-aggregations/refreshing-pre-aggregations#pre-aggregation-data-source
102107
[ref-config-multitenancy]: /embedding/multitenancy#multitenancy-multitenancy-vs-multiple-data-sources

docs-mintlify/docs/pre-aggregations/refreshing-pre-aggregations.mdx

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,39 @@ behavior:
2121
- [`CUBEJS_DROP_PRE_AGG_WITHOUT_TOUCH`](/reference/configuration/environment-variables#cubejs_drop_pre_agg_without_touch)
2222
- [`CUBEJS_PRE_AGGREGATIONS_BACKOFF_MAX_TIME`](/reference/configuration/environment-variables#cubejs_pre_aggregations_backoff_max_time)
2323

24+
## Pre-aggregation data source
25+
26+
By default, each data source builds and stores its pre-aggregations using its own
27+
connection. You can instead point a data source's pre-aggregations at a dedicated
28+
connection by adding a `PRE_AGGREGATIONS` segment to its environment variables. When
29+
set, that source's pre-aggregations are built on and read from the dedicated
30+
connection rather than the source's own.
31+
32+
Use the `CUBEJS_PRE_AGGREGATIONS_DB_*` variables for the default data source, and the
33+
`CUBEJS_DS_<NAME>_PRE_AGGREGATIONS_DB_*` variables for a [named data
34+
source][ref-multiple-data-sources]:
35+
36+
```dotenv
37+
# Default data source
38+
CUBEJS_DB_TYPE=postgres
39+
CUBEJS_DB_HOST=localhost
40+
41+
# Dedicated pre-aggregation data source for the default data source
42+
CUBEJS_PRE_AGGREGATIONS_DB_TYPE=postgres
43+
CUBEJS_PRE_AGGREGATIONS_DB_HOST=preagg-host
44+
45+
# A named data source and its dedicated pre-aggregation data source
46+
CUBEJS_DATASOURCES=default,analytics
47+
CUBEJS_DS_ANALYTICS_DB_TYPE=postgres
48+
CUBEJS_DS_ANALYTICS_DB_HOST=remotehost
49+
CUBEJS_DS_ANALYTICS_PRE_AGGREGATIONS_DB_TYPE=postgres
50+
CUBEJS_DS_ANALYTICS_PRE_AGGREGATIONS_DB_HOST=analytics-preagg-host
51+
```
52+
53+
The `PRE_AGGREGATIONS` variant supports the same connection variables as the regular
54+
`CUBEJS_DB_*` / `CUBEJS_DS_<NAME>_DB_*` data source variables (for example `_DB_TYPE`,
55+
`_DB_HOST`, `_DB_PORT`, `_DB_USER`, `_DB_PASS`, and `_DB_SSL`).
56+
2457
## Troubleshooting
2558

2659
### `Refresh scheduler interval error`
@@ -48,4 +81,5 @@ that would automatically do this for you.
4881

4982
[ref-multitenancy]: /embedding/multitenancy
5083
[ref-preaggs]: /docs/pre-aggregations/using-pre-aggregations
51-
[ref-production-multi-cluster]: /admin/deployment/deployment-types#multi-cluster
84+
[ref-production-multi-cluster]: /admin/deployment/deployment-types#multi-cluster
85+
[ref-multiple-data-sources]: /admin/connect-to-data/multiple-data-sources

docs-mintlify/reference/configuration/environment-variables.mdx

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1161,6 +1161,22 @@ This allows reusing pre-aggregations across environments, saving both time and c
11611161

11621162
</Info>
11631163

1164+
## `CUBEJS_PRE_AGGREGATIONS_DB_*`
1165+
1166+
Connection variables for a dedicated data source used to build and store the default
1167+
data source's pre-aggregations, instead of its own connection. They mirror the regular
1168+
`CUBEJS_DB_*` connection variables — for example `CUBEJS_PRE_AGGREGATIONS_DB_TYPE`,
1169+
`CUBEJS_PRE_AGGREGATIONS_DB_HOST`, `CUBEJS_PRE_AGGREGATIONS_DB_PORT`,
1170+
`CUBEJS_PRE_AGGREGATIONS_DB_USER`, `CUBEJS_PRE_AGGREGATIONS_DB_PASS`, and
1171+
`CUBEJS_PRE_AGGREGATIONS_DB_SSL`. For a [named data source][ref-config-multiple-ds-decorating-env],
1172+
use the decorated form `CUBEJS_DS_<NAME>_PRE_AGGREGATIONS_DB_*`.
1173+
1174+
| Possible Values | Default in Development | Default in Production |
1175+
| -------------------------------------------- | ---------------------- | --------------------- |
1176+
| Same as the corresponding `CUBEJS_DB_*` variable | N/A | N/A |
1177+
1178+
See [pre-aggregation data source][ref-preagg-data-source] for details and examples.
1179+
11641180
## `CUBEJS_PRE_AGGREGATIONS_ALLOW_NON_STRICT_DATE_RANGE_MATCH`
11651181

11661182
If `true`, allows non-strict date range matching for pre-aggregations by default.
@@ -1298,7 +1314,7 @@ option](/reference/configuration/config#can_switch_sql_user).
12981314
## `CUBEJS_ALLOW_UNGROUPED_WITHOUT_PRIMARY_KEY`
12991315

13001316
If `true`, disables the primary key inclusion check for
1301-
[ungrouped queries](/reference/queries#ungrouped-query).
1317+
[ungrouped queries](/reference/core-data-apis/queries#ungrouped-query).
13021318

13031319
| Possible Values | Default in Development | Default in Production |
13041320
| --------------- | ---------------------- | --------------------- |
@@ -1337,7 +1353,7 @@ Queries without an explicit `LIMIT` still use [`CUBEJS_DB_QUERY_DEFAULT_LIMIT`](
13371353
## `CUBESQL_SQL_NO_IMPLICIT_ORDER`
13381354

13391355
If `true`, prevents adding implicit [default `ORDER BY`
1340-
clause](/reference/queries#order) for [SQL API][ref-sql-api] queries.
1356+
clause](/reference/core-data-apis/queries#order) for [SQL API][ref-sql-api] queries.
13411357

13421358
| Possible Values | Default in Development | Default in Production |
13431359
| --------------- | ---------------------- | --------------------- |
@@ -1882,6 +1898,7 @@ The port for a Cube deployment to listen to API connections on.
18821898
[ref-config-db]: /admin/connect-to-data/data-sources
18831899
[ref-config-multiple-ds-cloud]: /admin/connect-to-data/multiple-data-sources
18841900
[ref-config-multiple-ds-decorating-env]: /admin/connect-to-data/multiple-data-sources#under-the-hood
1901+
[ref-preagg-data-source]: /docs/pre-aggregations/refreshing-pre-aggregations#pre-aggregation-data-source
18851902
[ref-config-sched-refresh-timer]: /reference/configuration/config#scheduled_refresh_timer
18861903
[ref-rest-scopes]: /reference/rest-api#configuration-api-scopes
18871904
[snowflake-docs-account-id]:
@@ -1897,7 +1914,7 @@ The port for a Cube deployment to listen to API connections on.
18971914
[wiki-tz-database]: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
18981915
[ref-sql-api]: /reference/sql-api
18991916
[ref-sql-api-streaming]: /reference/sql-api#streaming
1900-
[ref-row-limit]: /reference/queries#row-limit
1917+
[ref-row-limit]: /reference/core-data-apis/queries#row-limit
19011918
[mysql-server-tz-support]: https://dev.mysql.com/doc/refman/8.4/en/time-zone-support.html
19021919
[ref-schema-ref-preagg-allownonstrict]: /reference/data-modeling/pre-aggregations#allow_non_strict_date_range_match
19031920
[link-tesseract]: https://cube.dev/blog/introducing-next-generation-data-modeling-engine
@@ -1910,5 +1927,5 @@ The port for a Cube deployment to listen to API connections on.
19101927
[ref-mdx-api]: /reference/mdx-api
19111928
[ref-mdx-api-locale]: /reference/mdx-api#measure-format
19121929
[ref-time-dimensions]: /reference/data-modeling/dimensions#type
1913-
[ref-time-zone]: /reference/queries#time-zone
1930+
[ref-time-zone]: /reference/core-data-apis/queries#time-zone
19141931
[link-tzdb]: https://en.wikipedia.org/wiki/Tz_database

0 commit comments

Comments
 (0)