Skip to content

Commit fb38553

Browse files
joostboonjoostboon
authored
Merge pull request #912 from elementary-data/feature/execution-sla-test
**feat: add `execution_sla` test to verify pipeline runs before deadline**
2 parents f5c062d + c08644e commit fb38553

4 files changed

Lines changed: 702 additions & 0 deletions

File tree

‎macros/edr/tests/sla_tests.md‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Execution SLA Test
2+
3+
Verifies that a dbt model was executed successfully before a specified SLA deadline time.
4+
5+
**Note:** This test only works with dbt models (not sources), as it queries `dbt_run_results` which tracks model executions.
6+
7+
## Parameters
8+
9+
| Parameter | Required | Description |
10+
| -------------- | -------- | -------------------------------------------------------- |
11+
| `sla_time` | Yes | Deadline time (e.g., `"07:00"`, `"7am"`, `"2:30pm"`) |
12+
| `timezone` | Yes | IANA timezone name (e.g., `"America/Los_Angeles"`) |
13+
| `day_of_week` | No | Day(s) to check: `"Monday"` or `["Monday", "Wednesday"]` |
14+
| `day_of_month` | No | Day(s) of month to check: `1` or `[1, 15]` |
15+
16+
## Schedule Behavior
17+
18+
- **Default (no filters)**: Check every day
19+
- **day_of_week only**: Check only on specified days
20+
- **day_of_month only**: Check only on specified days of the month
21+
- **Both set**: Check if today matches **either** filter (OR logic)
22+
23+
## Examples
24+
25+
```yaml
26+
models:
27+
# Daily check (default)
28+
- name: daily_revenue
29+
tests:
30+
- elementary.execution_sla:
31+
sla_time: "06:00"
32+
timezone: "America/New_York"
33+
34+
# Weekly check - Mondays and Wednesdays only
35+
- name: weekly_report
36+
tests:
37+
- elementary.execution_sla:
38+
sla_time: "7am"
39+
timezone: "Europe/London"
40+
day_of_week: ["Monday", "Wednesday"]
41+
42+
# Monthly check - 1st and 15th only
43+
- name: monthly_close
44+
tests:
45+
- elementary.execution_sla:
46+
sla_time: "09:00"
47+
timezone: "Asia/Tokyo"
48+
day_of_month: [1, 15]
49+
50+
# Combined - check on Mondays OR on the 1st of the month
51+
- name: special_report
52+
tests:
53+
- elementary.execution_sla:
54+
sla_time: "08:00"
55+
timezone: "Europe/Amsterdam"
56+
day_of_week: ["Monday"]
57+
day_of_month: [1]
58+
```
59+
60+
## Logic
61+
62+
1. If today is not a scheduled check day (based on filters): **PASS** (skip)
63+
2. Query `dbt_run_results` for successful runs of this model today
64+
3. If any run completed before the SLA deadline: **PASS**
65+
4. If SLA deadline hasn't passed yet today: **PASS** (still time)
66+
5. If model ran but all runs were after the deadline: **FAIL** (MISSED_SLA)
67+
6. If model ran but all executions failed: **FAIL** (ALL_FAILED)
68+
7. If model didn't run today: **FAIL** (NOT_RUN)
69+
70+
---
71+
72+
## Time Format
73+
74+
The test accepts flexible time formats:
75+
76+
| Format | Example | Interpreted As |
77+
| --------- | -------------------- | ---------------- |
78+
| 24-hour | `"07:00"`, `"14:30"` | 7:00 AM, 2:30 PM |
79+
| 12-hour | `"7am"`, `"2:30pm"` | 7:00 AM, 2:30 PM |
80+
| Hour only | `"7"`, `"14"` | 7:00 AM, 2:00 PM |
81+
82+
**Recommendation:** Use 24-hour format for clarity.
83+
84+
---
85+
86+
## Timezone
87+
88+
Must be a valid IANA timezone name.
89+
90+
### Common Timezones
91+
92+
| Region | IANA Name |
93+
| ----------- | --------------------- |
94+
| US Pacific | `America/Los_Angeles` |
95+
| US Eastern | `America/New_York` |
96+
| UK | `Europe/London` |
97+
| Netherlands | `Europe/Amsterdam` |
98+
| Germany | `Europe/Berlin` |
99+
| India | `Asia/Kolkata` |
100+
| Japan | `Asia/Tokyo` |
101+
| Australia | `Australia/Sydney` |
102+
103+
Full list: <https://en.wikipedia.org/wiki/List_of_tz_database_time_zones>
104+
105+
---
106+
107+
## How Timezone Handling Works
108+
109+
The test uses Python's `pytz` library at **compile time** to:
110+
111+
1. Determine what "today" means in the specified timezone
112+
2. Convert the SLA deadline to UTC
113+
3. All SQL comparisons happen in UTC
114+
115+
This approach is **database-agnostic** - no database-specific timezone functions needed.
116+
117+
---
118+
119+
## Test Output
120+
121+
When the test fails, the result includes:
122+
123+
| Field | Description |
124+
| -------------------- | ------------------------------------------------ |
125+
| `target_date` | The date being checked |
126+
| `sla_time` | The configured SLA time |
127+
| `timezone` | The configured timezone |
128+
| `sla_deadline_utc` | The SLA deadline in UTC |
129+
| `sla_status` | `MET_SLA`, `MISSED_SLA`, `NOT_RUN`, `ALL_FAILED` |
130+
| `result_description` | Human-readable explanation |
131+
132+
**When the SLA deadline has not been reached yet:** The test passes and **no result row is returned**. This means if you run the test at 5:00 AM for a 7:00 AM SLA, the test will pass because there is still time to meet the deadline. Once the deadline passes, the test evaluates the actual run history and returns a row only if the SLA was missed.
133+
134+
---
135+
136+
## Notes
137+
138+
1. **DST Handling**: IANA timezone names automatically handle daylight saving time.
139+
140+
2. **Elementary Required**: Requires Elementary's `dbt_run_results` table to be populated.
141+
142+
3. **Models Only**: This test only works with dbt models (`resource_type = 'model'`). It does not support sources, as sources are not tracked in `dbt_run_results`.

0 commit comments

Comments
 (0)