Skip to content

Add job_run_id and job_run_url fields into DbtInvocationSchema#2195

Open
vyagubov wants to merge 2 commits intoelementary-data:masterfrom
vyagubov:add_invocation_fields
Open

Add job_run_id and job_run_url fields into DbtInvocationSchema#2195
vyagubov wants to merge 2 commits intoelementary-data:masterfrom
vyagubov:add_invocation_fields

Conversation

@vyagubov
Copy link
Copy Markdown
Contributor

@vyagubov vyagubov commented Apr 21, 2026

I needed those values in NODE INFO tab. We have them in UI and in Database, but don't have in the schema.

I have tested the solution locally.

Summary by CodeRabbit

  • New Features
    • Invocation records now include optional job run metadata: job run ID and job run URL.
    • Job-related URL/ID fields are returned when available, improving visibility into dbt job executions without affecting records that lack those fields.

@vyagubov vyagubov requested a deployment to elementary_test_env April 21, 2026 11:09 — with GitHub Actions Waiting
@github-actions
Copy link
Copy Markdown
Contributor

👋 @vyagubov
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in this pull request.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 21, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 441e6b2e-0112-4631-a91a-9eee3d4a1b6d

📥 Commits

Reviewing files that changed from the base of the PR and between adc7fec and e50ab29.

📒 Files selected for processing (2)
  • elementary/monitor/dbt_project/macros/get_models_latest_invocations_data.sql
  • elementary/monitor/dbt_project/macros/get_test_last_invocation.sql
🚧 Files skipped from review as they are similar to previous changes (2)
  • elementary/monitor/dbt_project/macros/get_models_latest_invocations_data.sql
  • elementary/monitor/dbt_project/macros/get_test_last_invocation.sql

📝 Walkthrough

Walkthrough

Replaced a single job_url existence check with three column-existence guards (job_url_exists, job_run_id_exists, job_run_url_exists) in dbt macros, conditionally projecting job_url, job_run_id, and job_run_url; and added job_run_id and job_run_url optional fields to the Python DbtInvocationSchema.

Changes

Cohort / File(s) Summary
SQL Invocation Macros
elementary/monitor/dbt_project/macros/get_models_latest_invocations_data.sql, elementary/monitor/dbt_project/macros/get_test_last_invocation.sql
Replaced single job_url column-existence check with three explicit checks (job_url_exists, job_run_id_exists, job_run_url_exists). Updated SELECT lists to conditionally include invocations.job_url, invocations.job_run_id, and invocations.job_run_url (or NULL as ...) across both the existence and fallback branches. Minor whitespace/select formatting adjustments.
Python Schema Definition
elementary/monitor/fetchers/invocations/schema.py
Added two optional fields to DbtInvocationSchema: job_run_id: Optional[str] and job_run_url: Optional[str] to mirror the expanded invocation projections.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰
I hopped through macros, soft and bright,
Found new fields tucked just right.
job_run_id and job_run_url cheer—
More breadcrumbs for tracking here.
Hop on, data—let’s persevere! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding two new fields (job_run_id and job_run_url) to DbtInvocationSchema, which is the primary focus of the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
elementary/monitor/dbt_project/macros/get_models_latest_invocations_data.sql (1)

2-28: ⚠️ Potential issue | 🟠 Major

Use column-existence guards for optional invocation fields.

This newly created macro selects job_url (line 24), job_run_id, and job_run_url (lines 27-28) unconditionally. The Python schema defines these as optional fields; existing dbt_invocations relations without these newer columns will cause the macro to fail. Add guards using elementary.column_exists_in_relation() to ensure backward compatibility, following the pattern used in get_models.sql, get_exposures.sql, and other macros.

Suggested compatibility guard
 {% macro get_models_latest_invocations_data() %}
   {% set invocations_relation = ref("dbt_invocations", package="elementary") %}
+  {% set has_job_url = elementary.column_exists_in_relation(invocations_relation, 'job_url') %}
+  {% set has_job_run_id = elementary.column_exists_in_relation(invocations_relation, 'job_run_id') %}
+  {% set has_job_run_url = elementary.column_exists_in_relation(invocations_relation, 'job_run_url') %}
 
   {% set query %}
@@
       invocations.command,
       invocations.selected,
       invocations.full_refresh,
-      invocations.job_url,
+      {% if has_job_url %}invocations.job_url{% else %}NULL as job_url{% endif %},
       invocations.job_name,
       invocations.job_id,
-      invocations.job_run_id,
-      invocations.job_run_url,
+      {% if has_job_run_id %}invocations.job_run_id{% else %}NULL as job_run_id{% endif %},
+      {% if has_job_run_url %}invocations.job_run_url{% else %}NULL as job_run_url{% endif %},
       invocations.orchestrator
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/monitor/dbt_project/macros/get_models_latest_invocations_data.sql`
around lines 2 - 28, The macro currently selects optional invocation columns
unconditionally (invocations.job_url, invocations.job_run_id,
invocations.job_run_url) which breaks on older relations; update the select list
to include each optional column only when
elementary.column_exists_in_relation(invocations_relation, '<column_name>') is
true (for invocations_relation variable used at top), mirroring the guards used
in get_models.sql/get_exposures.sql: build conditional snippets for 'job_url',
'job_run_id', and 'job_run_url' and include them in the final SELECT only when
the corresponding column_exists_in_relation check passes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@elementary/monitor/dbt_project/macros/get_test_last_invocation.sql`:
- Around line 22-41: The join selects invocations.job_run_id and
invocations.job_run_url unguarded, which breaks when the dbt_invocations
relation exists but lacks those columns; update the select in the invocations
branch of get_test_last_invocation.sql to use column_exists_in_relation() guards
for 'job_run_id' and 'job_run_url' (e.g., return invocations.job_run_id when
column_exists_in_relation(...) is true else NULL, and likewise for job_run_url)
so missing columns fall back to NULL, and apply the same guarded pattern to
get_models_latest_invocations_data.sql.

---

Outside diff comments:
In
`@elementary/monitor/dbt_project/macros/get_models_latest_invocations_data.sql`:
- Around line 2-28: The macro currently selects optional invocation columns
unconditionally (invocations.job_url, invocations.job_run_id,
invocations.job_run_url) which breaks on older relations; update the select list
to include each optional column only when
elementary.column_exists_in_relation(invocations_relation, '<column_name>') is
true (for invocations_relation variable used at top), mirroring the guards used
in get_models.sql/get_exposures.sql: build conditional snippets for 'job_url',
'job_run_id', and 'job_run_url' and include them in the final SELECT only when
the corresponding column_exists_in_relation check passes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 397d95c0-f3d0-4ae3-a485-d70883e20645

📥 Commits

Reviewing files that changed from the base of the PR and between e5af7e7 and adc7fec.

📒 Files selected for processing (3)
  • elementary/monitor/dbt_project/macros/get_models_latest_invocations_data.sql
  • elementary/monitor/dbt_project/macros/get_test_last_invocation.sql
  • elementary/monitor/fetchers/invocations/schema.py

Comment thread elementary/monitor/dbt_project/macros/get_test_last_invocation.sql
@vyagubov vyagubov requested a deployment to elementary_test_env April 21, 2026 11:27 — with GitHub Actions Waiting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant