feat(week-10): add dbt SQL practice exercises for Week 10 (#35)

Author: lassebenni

data-track/week-10/README.md

@@ -0,0 +1,54 @@
+# HYF Data Track — Week 10 Practice Exercises
+
+Four exercises that consolidate Week 10 (dbt Core: SQL transformations as a tested, documented, version-controlled project). Each exercise targets one skill from the content chapters. Work through them in order: Exercise 1 is a prerequisite for the `fct_trips` mart, and later exercises assume the project state from Exercises 1–3.
+
+These exercises require your Week 10 dbt project (set up in Chapter 2) running against the shared Azure PostgreSQL instance. They are not standalone: you copy the provided SQL files into your project and run dbt commands.
+
+## Layout
+
+| Folder | Skill | dbt command to verify |
+|--------|-------|----------------------|
+| `exercise_1` | Macros and computed columns for `stg_trips` | `dbt run --select stg_trips` |
+| `exercise_2` | Write and run a singular test | `dbt test --select test_type:singular` |
+| `exercise_3` | Debug a broken `ref()` | `dbt compile --select fct_trips` |
+| `exercise_4` | Propagate a column change from staging to mart | `dbt run --select +fct_trips` |
+
+## Folder structure
+
+```text
+week-10/
+  exercise_1/
+    safe_divide.sql     -- copy to macros/safe_divide.sql
+    stg_trips.sql       -- copy to models/staging/stg_trips.sql
+    README.md
+    solutions/
+      safe_divide.sql
+      stg_trips.sql
+  exercise_2/
+    assert_fare_amount_non_negative.sql  -- copy to tests/
+    README.md
+    solutions/
+      assert_fare_amount_non_negative.sql
+  exercise_3/
+    fct_trips_broken.sql  -- copy to models/marts/fct_trips.sql (overwrites)
+    README.md
+    solutions/
+      fct_trips.sql
+  exercise_4/
+    stg_trips.sql         -- copy to models/staging/stg_trips.sql
+    fct_trips.sql         -- copy to models/marts/fct_trips.sql
+    README.md
+    solutions/
+      stg_trips.sql
+      fct_trips.sql
+```
+
+## How to run
+
+These are dbt model, macro, and test files. Copy each file to the path shown in the exercise README, then run the dbt command shown in the table above. You need your Week 10 dbt project connected to the shared Azure PostgreSQL and `PG_PASSWORD` set in your environment.
+
+No `requirements.txt` is needed here: your dbt project already has `dbt-core` and `dbt-postgres` installed from Chapter 2.
+
+## Reference solutions and spoiler discipline
+
+Each `exercise_N/solutions/` folder holds the answer with `-- WHY` comments explaining the non-obvious choices. Time-box yourself to 15–20 minutes on each exercise before peeking. When you do open a solution, read the WHY notes: the reasoning is the teaching content.

data-track/week-10/exercise_1/README.md

@@ -0,0 +1,42 @@
+# Exercise 1: Macros and computed columns for `stg_trips`
+
+## What you do
+
+The [Materializations chapter](https://github.com/HackYourFuture/datatrack) builds `fct_trips` assuming `stg_trips` already exposes `tip_pct`, `fare_per_mile`, and `payment_type_label`. This exercise adds those three columns so `fct_trips` can use them.
+
+There are two files:
+
+| File | Copy to your project at |
+|---|---|
+| `safe_divide.sql` | `macros/safe_divide.sql` |
+| `stg_trips.sql` | `models/staging/stg_trips.sql` |
+
+## How to run
+
+Copy both files into your Week 10 dbt project, complete the TODOs, then:
+
+```bash
+dbt compile --select stg_trips
+```
+
+Open `target/compiled/<project>/models/staging/stg_trips.sql` and confirm every `{{ ... }}` block has been replaced with plain SQL.
+
+```bash
+dbt run --select stg_trips
+```
+
+Spot-check the result against your schema:
+
+```sql
+SELECT payment_type, payment_type_label, tip_pct, fare_per_mile
+FROM dev_<your_name>.stg_trips
+LIMIT 10;
+```
+
+## Success criteria
+
+- Every `payment_type` code (1–6) resolves to a readable label.
+- `tip_pct` and `fare_per_mile` are non-null for rows with non-zero denominators.
+- `dbt compile` exits cleanly with no Jinja errors.
+
+Stuck? The reference implementation is in `solutions/`. Try for 20 minutes first.

data-track/week-10/exercise_1/safe_divide.sql

@@ -0,0 +1,21 @@
+-- Exercise 1a: safe_divide macro
+--
+-- Create this file at macros/safe_divide.sql in your Week 10 dbt project.
+--
+-- A macro is a reusable Jinja function. This one wraps a division so it returns
+-- NULL instead of a database error when the denominator is 0 or NULL.
+--
+-- Signature: safe_divide(numerator, denominator) -> SQL expression
+-- Returns:   numerator / denominator, or NULL when denominator is 0 or NULL.
+--
+-- Used by stg_trips.sql for tip_pct and fare_per_mile.
+
+{% macro safe_divide(numerator, denominator) %}
+    -- TODO: write the macro body. It should:
+    --   1. Wrap denominator in NULLIF(denominator, 0) so a zero denominator becomes NULL.
+    --   2. Divide numerator by that result. PostgreSQL propagates NULL automatically.
+    --   3. Return only the SQL expression — no semicolon, no SELECT.
+    --
+    -- Expected compiled output for safe_divide('tip_amount', 'fare_amount'):
+    --   tip_amount / NULLIF(fare_amount, 0)
+{% endmacro %}

data-track/week-10/exercise_1/solutions/safe_divide.sql

@@ -0,0 +1,10 @@
+-- Exercise 1a solution: safe_divide macro
+
+{% macro safe_divide(numerator, denominator) %}
+    {{ numerator }} / NULLIF({{ denominator }}, 0)
+    -- WHY NULLIF: NULLIF(x, 0) returns NULL when x is 0, and x otherwise.
+    -- PostgreSQL propagates NULL through division automatically, so the whole
+    -- expression becomes NULL instead of raising a division-by-zero error.
+    -- This is the standard dbt pattern for any ratio column that could have a
+    -- zero denominator (tip_pct when fare_amount=0, fare_per_mile when trip_distance=0).
+{% endmacro %}

data-track/week-10/exercise_1/solutions/stg_trips.sql

@@ -0,0 +1,48 @@
+-- Exercise 1b solution: stg_trips with computed columns
+
+SELECT
+    -- identifiers
+    vendorid,
+    lpep_pickup_datetime                                AS pickup_datetime,
+    lpep_dropoff_datetime                               AS dropoff_datetime,
+
+    -- location keys
+    pulocationid                                        AS pickup_location_id,
+    dolocationid                                        AS dropoff_location_id,
+
+    -- trip facts
+    trip_distance,
+    fare_amount,
+    tip_amount,
+    payment_type,
+
+    -- computed: tip as a fraction of fare
+    {{ safe_divide('tip_amount', 'fare_amount') }}      AS tip_pct,
+    -- WHY safe_divide: fare_amount can be 0 (voided or no-charge trips in the TLC data).
+    -- Without NULLIF the database raises a division-by-zero error on those rows.
+
+    -- computed: fare per mile of travel
+    {{ safe_divide('fare_amount', 'trip_distance') }}   AS fare_per_mile,
+    -- WHY safe_divide: trip_distance is 0 on some short trips recorded as
+    -- metered-time-only (airport flat-rate, etc.). Same NULLIF guard needed.
+
+    -- computed: human-readable payment label from TLC integer code
+    CASE payment_type
+        {% for code, label in [
+            (1, 'Credit card'),
+            (2, 'Cash'),
+            (3, 'No charge'),
+            (4, 'Dispute'),
+            (5, 'Unknown'),
+            (6, 'Voided trip')
+        ] %}
+        WHEN {{ code }} THEN '{{ label }}'
+        {% endfor %}
+        ELSE 'Other'
+    END                                                 AS payment_type_label
+    -- WHY Jinja loop: the six codes and labels live in one place in the template;
+    -- adding or removing a code is a one-line change rather than editing six WHEN branches.
+
+FROM {{ source('nyc_taxi', 'raw_trips') }}
+WHERE pickup_location_id IS NOT NULL
+  AND fare_amount >= 0

data-track/week-10/exercise_1/stg_trips.sql

@@ -0,0 +1,46 @@
+-- Exercise 1b: Add computed columns to stg_trips
+--
+-- Replace (or update) models/staging/stg_trips.sql in your Week 10 dbt project.
+--
+-- The Materializations chapter builds fct_trips assuming stg_trips already
+-- exposes tip_pct, fare_per_mile, and payment_type_label. This exercise adds
+-- those three derived columns so fct_trips can use them.
+--
+-- Start from this template — it has the source reference and basic column list
+-- from Chapter 2. Your task is to add the three derived columns.
+
+SELECT
+    -- identifiers
+    vendorid,
+    lpep_pickup_datetime                                AS pickup_datetime,
+    lpep_dropoff_datetime                               AS dropoff_datetime,
+
+    -- location keys (used as join keys in fct_trips — keep as integers)
+    pulocationid                                        AS pickup_location_id,
+    dolocationid                                        AS dropoff_location_id,
+
+    -- trip facts
+    trip_distance,
+    fare_amount,
+    tip_amount,
+    payment_type,
+
+    -- TODO 1: add tip_pct using {{ safe_divide('tip_amount', 'fare_amount') }}
+    --         alias it as tip_pct
+
+    -- TODO 2: add fare_per_mile using {{ safe_divide('fare_amount', 'trip_distance') }}
+    --         alias it as fare_per_mile
+
+    -- TODO 3: add payment_type_label using a {% for %} loop over the TLC code mapping:
+    --           1 -> 'Credit card'
+    --           2 -> 'Cash'
+    --           3 -> 'No charge'
+    --           4 -> 'Dispute'
+    --           5 -> 'Unknown'
+    --           6 -> 'Voided trip'
+    --         Use a CASE WHEN block generated by the loop. See the Jinja chapter for
+    --         the pattern. The final alias should be payment_type_label.
+
+FROM {{ source('nyc_taxi', 'raw_trips') }}
+WHERE pickup_location_id IS NOT NULL
+  AND fare_amount >= 0

data-track/week-10/exercise_2/README.md

@@ -0,0 +1,30 @@
+# Exercise 2: Write a singular test
+
+## What you do
+
+Singular tests are plain `.sql` files that return the bad rows. dbt passes the test when the query returns zero rows. You write one yourself to prove you understand when to use a singular test over a generic YAML test, and experience the `FAIL 182 → PASS` cycle that confirms the test actually catches real data issues.
+
+| File | Copy to your project at |
+|---|---|
+| `assert_fare_amount_non_negative.sql` | `tests/assert_fare_amount_non_negative.sql` |
+
+## How to run
+
+1. Copy the file to `tests/` and complete the TODO.
+2. Temporarily remove the `AND fare_amount >= 0` filter from `stg_trips.sql` and rebuild:
+
+   ```bash
+   dbt run --select stg_trips
+   dbt test --select test_type:singular
+   ```
+
+   The test must report `FAIL 182`. This step proves the test actually catches the problem.
+
+3. Restore `AND fare_amount >= 0` to `stg_trips.sql`, rebuild, and re-run the test. It must now report `PASS`.
+
+## Success criteria
+
+- Test reports `FAIL 182` against unfiltered `stg_trips`, and `PASS` after the filter is restored.
+- You can state in one sentence why a singular test is the right tool here rather than a generic `not_null` YAML test.
+
+Stuck? The reference implementation is in `solutions/`. Try for 15 minutes first.

data-track/week-10/exercise_2/assert_fare_amount_non_negative.sql

@@ -0,0 +1,20 @@
+-- Exercise 2: Write a singular test
+--
+-- Create this file at tests/assert_fare_amount_non_negative.sql in your dbt project.
+--
+-- A singular test is a plain SELECT that returns the *bad* rows.
+-- dbt runs this query and reports FAIL if any rows are returned, PASS if zero rows come back.
+--
+-- Your task: write a SELECT against {{ ref('stg_trips') }} that returns every row
+-- where fare_amount is negative.
+--
+-- Expected result when you run `dbt test --select test_type:singular`:
+--   PASS (because stg_trips already filters fare_amount >= 0)
+--
+-- Before completing the TODO, first temporarily comment out the WHERE fare_amount >= 0
+-- filter in stg_trips.sql and re-run dbt run --select stg_trips + dbt test --select test_type:singular
+-- to confirm the test correctly reports FAIL 182 against the unfiltered data.
+-- Then restore the filter and confirm the test goes back to PASS.
+
+-- TODO: write the singular test SELECT here.
+-- Replace this comment with: SELECT * FROM {{ ref('stg_trips') }} WHERE ...

data-track/week-10/exercise_2/solutions/assert_fare_amount_non_negative.sql

@@ -0,0 +1,14 @@
+-- Exercise 2 solution: singular test for negative fare amounts
+
+SELECT *
+FROM {{ ref('stg_trips') }}
+WHERE fare_amount < 0
+-- WHY singular test (not a generic YAML test): the check involves the column's actual
+-- value, not just presence or allowed set. A generic `not_null` only checks for NULL,
+-- and `accepted_values` requires listing every valid value. A singular test can express
+-- any SQL predicate, including numeric ranges or cross-column rules.
+--
+-- WHY this test is expected to PASS on the filtered stg_trips:
+-- stg_trips already applies WHERE fare_amount >= 0, so no negative rows survive into
+-- the model. The test passes because the staging filter removed the bad data upstream.
+-- If the filter were ever accidentally removed, this test would catch the regression.

data-track/week-10/exercise_3/README.md

@@ -0,0 +1,30 @@
+# Exercise 3: Debug a broken `ref()`
+
+## What you do
+
+When dbt cannot resolve a `{{ ref() }}`, it fails at compile time, not at run time. That is different from a plain SQL `CREATE VIEW`: Postgres would not catch a bad table name until someone queries the view. This exercise drills the compile-to-find-the-typo workflow.
+
+| File | Copy to your project at |
+|---|---|
+| `fct_trips_broken.sql` | `models/marts/fct_trips.sql` (overwrites your working file) |
+
+## How to run
+
+1. Copy the file and run:
+
+   ```bash
+   dbt compile --select fct_trips
+   ```
+
+2. Read the error message. Write down: which node did dbt name, and which file did it point at?
+
+3. Find the typo and fix it directly in `models/marts/fct_trips.sql`.
+
+4. Re-run `dbt compile --select fct_trips` to confirm it compiles cleanly.
+
+## Success criteria
+
+- You spotted the typo from the error message without reading the file top-to-bottom.
+- You can state in one sentence why dbt reports the error at compile time rather than at run time.
+
+Stuck? The corrected file is in `solutions/`. Try for 10 minutes first.