Skip to content

feat(#2075): migrate marts__combined_course_enrollment_detail to dimensional models#2314

Draft
quazi-h wants to merge 2 commits into
mainfrom
copilot/2075-migrate-combined-course-enrollment-detail
Draft

feat(#2075): migrate marts__combined_course_enrollment_detail to dimensional models#2314
quazi-h wants to merge 2 commits into
mainfrom
copilot/2075-migrate-combined-course-enrollment-detail

Conversation

@quazi-h

@quazi-h quazi-h commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

What are the relevant tickets?

Closes #2075 (mart migration — depends on prerequisite PR #2317)

⚠️ This PR must be merged and deployed AFTER #2317 (dim_course_run and tfact_enrollment column additions, deployed with --full-refresh). Deploying this mart before #2317 will result in NULL values for courserun_upgrade_deadline and courserunenrollment_is_edx_enrolled.

Description (What does it do?)

Migrates marts__combined_course_enrollment_detail from intermediate combined models (int__combined__courserun_enrollments, int__combined__course_runs, int__combined__users) to the dimensional layer.

The 6-way per-platform UNION ALL is replaced by a single flat query joining dimensional models:

Notable join strategy decisions:

  • Certificate join: uses user_email (from dim_user) as the join key for all platforms. Bootcamps username is not stored in dim_user, so email is the reliable cross-table key.
  • edX.org MITxOnline certificate cross-lookup: previously used int__micromasters__users CTE; now uses dim_user.user_mitxonline_username directly, eliminating that CTE.
  • edX.org MicroMasters order join: uses dim_user.micromasters_user_id replacing the micromasters_users CTE user_id lookup.
  • Bootcamps order join: changed from user_username-based to user_id-based (dim_user.bootcamps_application_user_id = bootcamps_completed_orders.order_purchaser_user_id), since Bootcamps username is not in dim_user.
  • MITxPro orders: bridged via a supplemental mitxpro_enrollment_orders CTE sourcing ecommerce_order_id from int__mitxpro__courserunenrollments (not yet in tfact_enrollment).
  • Grades: sourced from tfact_grade joined on user_fk + courserun_fk — clean dimensional join, no per-platform grade CTEs needed.

All 38 output columns preserved and verified via systematic diff against the original final SELECT.

How can this be tested?

Prerequisite: #2317 must be deployed first (with --full-refresh on both dimensional models).

-- Q1: Row count parity by platform
SELECT
    platform,
    count(*) AS row_count
FROM ol_warehouse_marts_qa.marts__combined_course_enrollment_detail
GROUP BY 1
ORDER BY 1;
-- Q2: Verify courserun_upgrade_deadline is populated for MITxOnline
SELECT
    platform,
    count(*) AS total_rows,
    count(courserun_upgrade_deadline) AS non_null_upgrade_deadline
FROM ol_warehouse_marts_qa.marts__combined_course_enrollment_detail
WHERE platform = 'MITx Online'
GROUP BY 1;
-- Q3: Verify courserunenrollment_is_edx_enrolled distribution
SELECT
    platform,
    courserunenrollment_is_edx_enrolled,
    count(*) AS row_count
FROM ol_warehouse_marts_qa.marts__combined_course_enrollment_detail
GROUP BY 1, 2
ORDER BY 1, 2;
-- Expected: edxorg and residential = all true; mitxonline/mitxpro = mix of true/false/null;
-- bootcamps/emeritus/global_alumni = all null
-- Q4: Verify grade data populates from tfact_grade
SELECT
    platform,
    count(*) AS total_enrollments,
    count(courserungrade_grade) AS enrollments_with_grade,
    count(courserungrade_is_passing) AS enrollments_with_pass_flag
FROM ol_warehouse_marts_qa.marts__combined_course_enrollment_detail
GROUP BY 1
ORDER BY 1;

Additional Context

Deployment order (no breaking change if followed):

  1. Merge and deploy feat(#2075): add courserun_upgrade_deadline to dim_course_run and enrollment_is_edx_enrolled to tfact_enrollment #2317 — run dim_course_run --full-refresh and tfact_enrollment --full-refresh
  2. Verify Q1/Q2 queries in feat(#2075): add courserun_upgrade_deadline to dim_course_run and enrollment_is_edx_enrolled to tfact_enrollment #2317 show correct values
  3. Merge and deploy this PR — run marts__combined_course_enrollment_detail normally (no full-refresh needed)

Downstream consumers (dashboards, reports) using courserun_upgrade_deadline or courserunenrollment_is_edx_enrolled will see no regression as long as step 1 completes before step 3.

…nsional models

- Add courserun_upgrade_deadline to dim_course_run (MITxOnline only; null for other platforms)
- Add enrollment_is_edx_enrolled to tfact_enrollment (per-platform: true for edxorg/residential,
  from source field for mitxonline/mitxpro, null for program/bootcamps)
- Rewrite marts__combined_course_enrollment_detail to source from dimensional layer:
  - tfact_enrollment + dim_course_run + dim_course + dim_user replace 6-way UNION ALL
  - Grade data from tfact_grade (joined on user_fk + courserun_fk)
  - Certificate join uses user_email (from dim_user) as cross-table key
  - edX.org MITxOnline certificate cross-lookup uses dim_user.user_mitxonline_username
  - Bootcamps order join changed from username-based to user_id-based
  - MITxPro order join bridged via supplemental ecommerce_order_id CTE
  - edX.org MicroMasters order join uses dim_user.micromasters_user_id
  - All 38 output columns preserved and verified via systematic diff

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
These changes are now in a separate prerequisite PR:
#2317

This branch now contains only the mart rewrite.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate marts__combined_course_enrollment_detail to use dimensional layer

1 participant