You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cloud/features/alerts_notifications.md
+1-3Lines changed: 1 addition & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,7 +32,7 @@ Tobiko Cloud sends an alert based on a *trigger*. There are two types of trigger
32
32
33
33
Events are tied to steps in the SQLMesh `plan` and `run` processes. For example, you could alert whenever a `plan` succeeded or a `run` failed.
34
34
35
-
Measures are based on either [automatically calculated measures](../../guides/observer.md#measures)like run time or [custom measure you define](../../guides/observer.md#custom-measures) via SQL queries.
35
+
Measures are [automatically calculated](../../guides/observer.md#measures)at run time.
36
36
37
37
Choose whether the alert will be triggered by a Measure or Event in the alert's Trigger Type field.
38
38
@@ -75,8 +75,6 @@ Some measures, like run time, are most useful when accumulated over an entire `p
75
75
76
76
Configure a cumulative measure alert by choosing an Artifact type of Plan or Run.
77
77
78
-
Other measure alerts are useful when applied to individual models or steps in a `plan` or `run`. For example, you might know that your `users` model should process around 1000 new users each day. You could [create a measure](../../guides/observer.md#custom-measures) to track the number of new users and an alert that notifies you if that number drops below 500.
79
-
80
78
Configure a non-cumulative measure alert by choosing an Artifact type of Measure.
81
79
82
80

Copy file name to clipboardExpand all lines: docs/guides/observer.md
+1-100Lines changed: 1 addition & 100 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,7 +51,7 @@ The five entities in bold - models, audits, environments, runs, and plans - prov
51
51
52
52
We now describe the specific measures SQLMesh captures about each entity.
53
53
54
-
SQLMesh performs its primary actions during **plans** and **runs**, so most measures are generated when they occur. Both plans and runs are executed in a specific **environment**, so all of their measures are environment-specific.
54
+
SQLMesh performs its primary actions during **plans** and **runs**, so measures are automatically generated when they occur. Both plans and runs are executed in a specific **environment**, so all of their measures are environment-specific.
55
55
56
56
These measures are recorded and stored for each plan or run in a specific environment:
57
57
@@ -62,8 +62,6 @@ These measures are recorded and stored for each plan or run in a specific enviro
62
62
- The model versions evaluated during the plan/run
63
63
- Each model's run time
64
64
65
-
Additionally, you can define [custom measures](#custom-measures) that will be captured for each model.
66
-
67
65
## Installation
68
66
69
67
SQLMesh Observer is part of the `sqlmesh-enterprise` Python library and is installed via `pip`.
@@ -209,100 +207,3 @@ Next, the Loaded Intervals section displays the time intervals that have been lo
209
207
The model information page concludes with a list of most frequent audits the model has failed, the most frequent time intervals that failed, and the largest historical model run times:
SQLMesh Observer allows you to calculate and track custom measures in addition to the ones it [automatically calculates](#data).
216
-
217
-
### Definition
218
-
219
-
Each custom measure is associated with a model and is defined by a SQL query in the model file.
220
-
221
-
The `@measure` macro is used to define custom measures. The body of the `@measure` macro is the query, and each column in the query defines a separate measure.
222
-
223
-
A measure's name is the name of the column that defined it. Measure names must be unique within a model, but a name may be used in multiple models.
224
-
225
-
A model may contain more than one `@measure` macro specification. The `@measure` macros must be specified after the model's primary query. They will be executed during a SQLMesh `plan` or `run` after the primary model query is executed.
226
-
227
-
This example shows a model definition that includes a measure query defining two measures: `row_count` (the total number of rows in the table) and `num_col_avg` (the average value of the model's `numeric_col` column).
228
-
229
-
```sql
230
-
MODEL (
231
-
name custom_measure.example,
232
-
kind FULL
233
-
);
234
-
235
-
SELECT
236
-
numeric_col
237
-
FROM
238
-
custom_measure.upstream;
239
-
240
-
@measure( -- Measure query specified in the `@measure` macro
241
-
SELECT
242
-
COUNT(*) AS row_count, -- Table's row count
243
-
AVG(numeric_col) AS num_col_avg -- Average value of `numeric_col`
244
-
FROMcustom_measure.example-- Select FROM the name of the model
245
-
);
246
-
```
247
-
248
-
Every time the `custom_measure.example` model is executed, Observer will execute the measure query and store the value it returns.
249
-
250
-
By default, the measure's timestamp will be the execution time of the `plan`/`run` that captured the measure. [Incremental by time range](../concepts/models/model_kinds.md#incremental_by_time_range) models may specify [custom timestamps](#custom-time-column).
251
-
252
-
An Observer chart allows you to select which measure to display. The chart displays the value of the selected measure on the y-axis and the execution time of the associated `plan`/`run` on the x-axis, allowing you to monitor whether the value has meaningfully changed since the previous execution.
253
-
254
-
### Incremental by time models
255
-
256
-
#### Custom time column
257
-
258
-
In the previous example, Observer automatically associated each measure value with the execution time of the `plan` or `run` that executed it.
259
-
260
-
For [incremental by time range models](../concepts/models/model_kinds.md#incremental_by_time_range), you can customize how measures are associated with time by including your own time column in the measure query.
261
-
262
-
The time column must be named `ts` and may be of any datetime data type (e.g., date string, `DATE`, `TIMESTAMP`, etc.). Custom times are typically derived from a datetime column in the model data and are most useful when the measure groups by the datetime.
263
-
264
-
For example, this incremental model stores the date of each data point in the `event_datestring` column. We could measure each day's row count and numeric column average with this measure query:
265
-
266
-
```sql
267
-
MODEL (
268
-
name custom_measure.incremental_example
269
-
kind INCREMENTAL_BY_TIME_RANGE (
270
-
time_column event_datestring
271
-
)
272
-
);
273
-
274
-
SELECT
275
-
event_datestring,
276
-
numeric_col
277
-
FROM
278
-
custom_measure.upstream
279
-
WHERE
280
-
event_datestring BETWEEN @start_ds AND @end_ds;
281
-
282
-
@measure(
283
-
SELECT
284
-
event_datestring AS ts, -- Custom measure time column `ts`
285
-
COUNT(*) AS daily_row_count, -- Daily row count
286
-
AVG(numeric_col) AS daily_num_col_avg -- Daily average value of `numeric_col`
287
-
FROMcustom_measure.incremental_example
288
-
WHERE event_datestring BETWEEN @start_ds AND @end_ds -- Filter measure on time
289
-
GROUP BY event_datestring -- Group measure by time
290
-
);
291
-
```
292
-
293
-
The measure query both filters and groups the data based on the model's time column `event_datestring`. The filtering and grouping ensures that only one measure value is ever calculated for a specific day of data.
294
-
295
-
NOTE: the custom time column approach will not work correctly if the model's [`lookback` argument](../concepts/models/overview.md#lookback) is specified because a given day's data will be processed every time it is in the lookback window.
296
-
297
-
#### Execution and custom times
298
-
299
-
A model may contain multiple measure queries, so both execution time and custom time measures may be specified for the same model.
300
-
301
-
These two measure types help answer different questions:
302
-
303
-
1. Execution time: has something meaningfully changed **on this `plan`/`run`** compared to previous plans/runs?
304
-
2. Custom time: has something meaningfully changed **in a specific time point's data** compared to other time points?
305
-
306
-
If multiple time points of data are processed during each model execution, an anomaly at a specific time may not be detectable from an execution time measure alone.
307
-
308
-
Custom time measures enable monitoring at the temporal granularity of the data itself.
0 commit comments