diff --git a/docs.json b/docs.json index c7085648..0a56d6df 100644 --- a/docs.json +++ b/docs.json @@ -96,26 +96,32 @@ "guides/how-to-promote-content", "guides/verified-content", "guides/keyboard-shortcuts", - "guides/table-calculations", { - "group": "Table calculation functions", + "group": "Table calculations", "pages": [ - "references/table-calculation-functions/row-functions", - "references/table-calculation-functions/pivot-functions", - "references/table-calculation-functions/aggregate-functions" - ] - }, - { - "group": "Table calc SQL templates", - "pages": [ - "guides/table-calculations/sql-templates", - "guides/table-calculations/table-calculation-sql-templates/percent-change-from-previous", - "guides/table-calculations/table-calculation-sql-templates/percent-of-previous-value", - "guides/table-calculations/table-calculation-sql-templates/percent-of-total-column", - "guides/table-calculations/table-calculation-sql-templates/percent-of-group-pivot-total", - "guides/table-calculations/table-calculation-sql-templates/rank-in-column", - "guides/table-calculations/table-calculation-sql-templates/running-total", - "guides/table-calculations/table-calculation-sql-templates/rolling-window" + "guides/table-calculations", + "guides/formula-table-calculations", + { + "group": "Functions", + "pages": [ + "references/table-calculation-functions/row-functions", + "references/table-calculation-functions/pivot-functions", + "references/table-calculation-functions/aggregate-functions" + ] + }, + { + "group": "SQL templates", + "pages": [ + "guides/table-calculations/sql-templates", + "guides/table-calculations/table-calculation-sql-templates/percent-change-from-previous", + "guides/table-calculations/table-calculation-sql-templates/percent-of-previous-value", + "guides/table-calculations/table-calculation-sql-templates/percent-of-total-column", + "guides/table-calculations/table-calculation-sql-templates/percent-of-group-pivot-total", + "guides/table-calculations/table-calculation-sql-templates/rank-in-column", + "guides/table-calculations/table-calculation-sql-templates/running-total", + "guides/table-calculations/table-calculation-sql-templates/rolling-window" + ] + } ] } ] diff --git a/guides/formula-table-calculations.mdx b/guides/formula-table-calculations.mdx new file mode 100644 index 00000000..2e32fa87 --- /dev/null +++ b/guides/formula-table-calculations.mdx @@ -0,0 +1,227 @@ +--- +title: "Formula table calculations" +description: "Build table calculations using a spreadsheet-style formula syntax instead of raw SQL." +sidebarTitle: "Formula (Beta)" +--- + + + **Formula table calculations are in Beta (Early Access).** + + The feature is enabled by default on supported warehouses, with no + flag required, and we're actively expanding the function catalog. + We'd love your feedback while we iterate. See [feature maturity levels](/references/workspace/feature-maturity-levels) + for details. + + +Formula table calculations let you write table calculations in a +spreadsheet-style syntax, the way you would in Google Sheets or Excel, +instead of raw SQL. When you create a new [table calculation](/guides/table-calculations) +on a supported warehouse, **Formula** is the default input mode. You +can switch to the SQL editor any time. + + + Creating a formula table calculation + Creating a formula table calculation + + +## Why use formulas? + +- **Faster to write** for common calculations. No `OVER (…)` or + `CASE WHEN` boilerplate to remember. +- **Familiar** if you've used Google Sheets, Excel, or Airtable. +- **Portable** across warehouses. The same formula compiles to the + correct SQL for whichever warehouse your project is connected to. + +If you need something the formula syntax doesn't cover yet, the SQL +editor is always one click away. + +## Supported warehouses + +| Warehouse | Formula support | +| ----------- | --------------- | +| BigQuery | ✅ | +| ClickHouse | ✅ | +| Databricks | ✅ | +| DuckDB | ✅ | +| PostgreSQL | ✅ | +| Redshift | ✅ | +| Snowflake | ✅ | +| Athena | 🚧 Not yet | +| Trino | 🚧 Not yet | + +On warehouses where formulas aren't supported yet, the Formula toggle +is hidden and the SQL editor works exactly as it always has. + +## Writing your first formula + +Every formula starts with `=`. Reference a field by its column name +(the same name you see in the results table header): + +``` +=orders_total_order_amount * 1.2 +``` + +The result appears as a new green column in your results table: + + + Formula table calculation in the results table + Formula table calculation in the results table + + +You can use: + +- **Numbers**: `42`, `3.14`, `-1.5` +- **Strings**: `"hello"` or `'hello'` +- **Booleans**: `TRUE`, `FALSE` +- **Column references**: any field present in your results table +- **Arithmetic operators**: `+`, `-`, `*`, `/`, `%` (modulo) +- **Comparison operators**: `=`, `<>`, `>`, `<`, `>=`, `<=` +- **Boolean operators**: `AND`, `OR`, `NOT` + +## Function reference + +### Math + +| Function | Description | +| --------------------------- | ------------------------------- | +| `ABS(x)` | Absolute value | +| `ROUND(x, [digits])` | Round to N decimal places | +| `CEIL(x)` / `CEILING(x)` | Round up to nearest integer | +| `FLOOR(x)` | Round down to nearest integer | +| `MIN(x, [y])` | Minimum (scalar or aggregate) | +| `MAX(x, [y])` | Maximum (scalar or aggregate) | + +### Logical + +| Function | Description | +| ------------------------------- | ---------------------- | +| `IF(condition, then, [else])` | Conditional expression | +| `AND`, `OR`, `NOT` | Boolean operators | +| `=`, `<>`, `>`, `<`, `>=`, `<=` | Comparison operators | + +### String + +| Function | Description | +| ------------------------ | ----------------------- | +| `CONCAT(a, b, …)` | Concatenate strings | +| `LEN(s)` / `LENGTH(s)` | String length | +| `TRIM(s)` | Remove whitespace | +| `LOWER(s)` | Convert to lowercase | +| `UPPER(s)` | Convert to uppercase | + +### Date + +| Function | Description | +| ------------ | --------------------- | +| `TODAY()` | Current date | +| `NOW()` | Current timestamp | +| `YEAR(d)` | Extract year | +| `MONTH(d)` | Extract month | +| `DAY(d)` | Extract day | + +### Aggregation + +| Function | Description | +| -------------------------- | --------------------------- | +| `SUM(x)` | Sum values | +| `AVG(x)` / `AVERAGE(x)` | Average values | +| `COUNT([x])` | Count rows (or non-null x) | +| `SUMIF(condition, x)` | Sum where condition is true | +| `AVERAGEIF(condition, x)` | Avg where condition is true | +| `COUNTIF(condition)` | Count where condition true | + +### Window + +| Function | Description | +| -------------------------------- | --------------------------------- | +| `RUNNING_TOTAL(x)` | Running (cumulative) total | +| `ROW_NUMBER()` | Sequential row number | +| `RANK()` | Rank with gaps | +| `DENSE_RANK()` | Rank without gaps | +| `LAG(x, [offset], [default])` | Previous row's value | +| `LEAD(x, [offset], [default])` | Next row's value | +| `FIRST(x)` | First value in window | +| `LAST(x)` | Last value in window | +| `NTILE(n)` | Distribute rows into N buckets | +| `MOVING_SUM(x, n)` | Sum of the last N rows | +| `MOVING_AVG(x, n)` | Average of the last N rows | + +### Null handling + +| Function | Description | +| ---------------------- | -------------------------------- | +| `COALESCE(a, b, …)` | First non-null argument | +| `ISNULL(x)` | `TRUE` if x is null | + +## Examples + +**Gross margin as a percentage** + +``` +=ROUND((orders_revenue - orders_cost) / orders_revenue * 100, 2) +``` + +**Flag high-value orders** + +``` +=IF(orders_total_amount > 1000, "VIP", "Standard") +``` + +**Running total of revenue** + +``` +=RUNNING_TOTAL(orders_revenue) +``` + +**Period-over-period growth %** + +``` +=(orders_revenue - LAG(orders_revenue, 1, 0)) / LAG(orders_revenue, 1, 0) * 100 +``` + +**Bucket customers by spend** + +``` +=IF(customers_lifetime_value > 10000, "Platinum", + IF(customers_lifetime_value > 5000, "Gold", + IF(customers_lifetime_value > 1000, "Silver", "Bronze"))) +``` + +**Percent of total** + +``` +=orders_revenue / SUM(orders_revenue) * 100 +``` + +## FAQ + + + + +No. The input mode is chosen when you create the table calculation +and can't be changed afterwards, because formulas and SQL aren't +always losslessly interconvertible. If you need to move a SQL calc to +a formula (or vice versa), delete the old one and create a new one in +the mode you want. + + + +The Formula toggle is hidden and the SQL editor is the only option. +Existing SQL table calculations keep working on every warehouse. The +beta is strictly additive. + + + +Post in the [Lightdash Community Slack](https://join.slack.com/t/lightdash-community/shared_invite/zt-3pbwqmq5e-vhTc7HHcS787w618ngoPHA) +or [open a GitHub issue](https://github.com/lightdash/lightdash/issues). +See [Contact Us](/contact/contact-info) for more options. + + + + +## Related + +- [Table calculations overview](/guides/table-calculations) +- [Table calculation functions](/references/table-calculation-functions/row-functions) +- [SQL templates for table calculations](/guides/table-calculations/sql-templates) +- [Feature maturity levels](/references/workspace/feature-maturity-levels) diff --git a/guides/table-calculations.mdx b/guides/table-calculations.mdx index 7205066a..71090a57 100644 --- a/guides/table-calculations.mdx +++ b/guides/table-calculations.mdx @@ -1,5 +1,6 @@ --- title: "Table calculations" +sidebarTitle: "Overview" description: "Table calculations make it easy to create metrics on the fly (for example, aggregating the data in your table over some window, or getting a running total of a column)." --- @@ -9,7 +10,16 @@ You add table calculations in the Explore view and they appear as green columns -Table calculations are built using raw SQL. That means you can use table calculations to build mathematical, True/False, text, and date-based calculations (basically, anything you can do in SQL, you can do in table calculations). +Table calculations can be built in two ways: + +- **[Formula](/guides/formula-table-calculations)** (Beta): a spreadsheet-style syntax that reads like Google Sheets. The default mode for new table calculations on supported warehouses. +- **SQL**: raw SQL, available on every warehouse. + + + **[Formula table calculations](/guides/formula-table-calculations) are in Beta (Early Access).** Enabled by default on supported warehouses: BigQuery, ClickHouse, Databricks, DuckDB, PostgreSQL, Redshift, and Snowflake. See [feature maturity levels](/references/workspace/feature-maturity-levels). + + +With either mode you can build mathematical, True/False, text, and date-based calculations on top of your results. ## When to use table calculations diff --git a/images/guides/formula-table-calculations/create-table-calculation-formula-dark.png b/images/guides/formula-table-calculations/create-table-calculation-formula-dark.png new file mode 100644 index 00000000..cf685598 Binary files /dev/null and b/images/guides/formula-table-calculations/create-table-calculation-formula-dark.png differ diff --git a/images/guides/formula-table-calculations/create-table-calculation-formula-light.png b/images/guides/formula-table-calculations/create-table-calculation-formula-light.png new file mode 100644 index 00000000..4780ea15 Binary files /dev/null and b/images/guides/formula-table-calculations/create-table-calculation-formula-light.png differ diff --git a/images/guides/formula-table-calculations/table-calculation-formula-in-table-dark.png b/images/guides/formula-table-calculations/table-calculation-formula-in-table-dark.png new file mode 100644 index 00000000..573f45b3 Binary files /dev/null and b/images/guides/formula-table-calculations/table-calculation-formula-in-table-dark.png differ diff --git a/images/guides/formula-table-calculations/table-calculation-formula-in-table-light.png b/images/guides/formula-table-calculations/table-calculation-formula-in-table-light.png new file mode 100644 index 00000000..508c82d2 Binary files /dev/null and b/images/guides/formula-table-calculations/table-calculation-formula-in-table-light.png differ