Skip to content

feat: add edr artifacts commands for warehouse artifact queries#2193

Open
oravi wants to merge 1 commit intomasterfrom
add_artifact_commands
Open

feat: add edr artifacts commands for warehouse artifact queries#2193
oravi wants to merge 1 commit intomasterfrom
add_artifact_commands

Conversation

@oravi
Copy link
Copy Markdown
Collaborator

@oravi oravi commented Apr 19, 2026

Summary

  • Adds a new edr artifacts subcommand group so agents and scripts can query Elementary's warehouse artifact tables directly. Twelve commands across six entities (test-results, run-results, invocations, models, sources, tests) with list and get-by-id variants, backed by dedicated dbt macros under elementary_cli.
  • Commands emit a stable JSON envelope on stdout (with -o table for humans), errors as {error, code, details} on stderr, exit codes 0/1/2 for success/user-error/system-error, and pagination via --limit with has_more. Adds a --profile flag so edr can point at a profile other than elementary.
  • Bootstrap suppresses the ASCII logo / upgrade banner / info logs and routes dbt logs to stderr when the invocation is edr artifacts ..., keeping stdout pipe-safe.

Test plan

  • edr artifacts {entity} list + single-get verified end-to-end against a live Snowflake warehouse for all six entity pairs (test-results, run-results, invocations, models, sources, tests).
  • Confirmed JSON envelope shape and has_more on list truncation.
  • Confirmed not-found path emits {"error":..., "code":"NOT_FOUND", ...} on stderr with exit code 1.
  • Confirmed -o table renders the documented list columns.
  • Reviewer to run edr artifacts --help and spot-check one entity against their own warehouse.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

New Features

  • Added edr artifacts command group for querying Elementary warehouse artifact tables
  • Supports querying test results, run results, invocations, models, sources, and tests
  • JSON output (default) with optional table format
  • Built-in filtering and pagination support
  • Structured error codes and exit statuses for different error conditions

Adds a new `edr artifacts` subcommand group that lets agents and scripts
query Elementary's warehouse artifact tables directly. Twelve commands
across six entities (test-results, run-results, invocations, models,
sources, tests) with list and get-by-id variants.

Output is a stable JSON envelope on stdout (with `-o table` for humans),
errors are JSON on stderr with `{error, code, details}`, and exit codes
follow 0/1/2 for success/user-error/system-error. List commands paginate
via `--limit` and report `has_more`. Logs are redirected to stderr when
running `edr artifacts ...` so stdout stays pipe-safe.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

👋 @oravi
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in this pull request.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 19, 2026

📝 Walkthrough

Walkthrough

This PR introduces a new edr artifacts CLI subsystem for querying Elementary warehouse artifact tables. It includes structured JSON output by default, optional table formatting, pagination support, configurable filtering, dbt macro-backed operations, error handling, and profile override capabilities. The implementation spans Python CLI commands, configuration utilities, error handling, dbt runners, SQL macros, and main CLI integration.

Changes

Cohort / File(s) Summary
Documentation
elementary/artifacts/README.md
Added comprehensive CLI documentation describing the edr artifacts command, its JSON/table output modes, exit codes, global options, pagination semantics, and all supported subcommands (test results, run results, invocations, models, sources, tests) with usage examples.
CLI Core
elementary/artifacts/cli.py, elementary/artifacts/common.py, elementary/artifacts/output.py
Established CLI infrastructure: main artifacts command group with helper functions for error handling, standardized options decorator (common_options) for subcommands, configuration builder, error code constants with exit code mapping, and output utilities (JSON/table/error formatting).
CLI Entity Commands
elementary/artifacts/entities/invocations.py, elementary/artifacts/entities/models.py, elementary/artifacts/entities/run_results.py, elementary/artifacts/entities/sources.py, elementary/artifacts/entities/test_results.py, elementary/artifacts/entities/tests.py
Implemented six command groups (list + single-fetch variants) for querying artifact types, each supporting filtering, pagination, JSON field hydration, dual output modes, and consistent error handling. High-complexity commands include parameter validation, time-window resolution, and multi-field JSON parsing.
Support Infrastructure
elementary/artifacts/fetching.py, elementary/artifacts/runner.py
Created macro execution layer with error translation (ArtifactFetchError), pagination trimming, JSON field normalization, and dbt runner factory with dynamic profile override injection via method patching.
dbt SQL Macros
elementary/monitor/dbt_project/macros/artifacts/_filter_helpers.sql, elementary/monitor/dbt_project/macros/artifacts/get_dbt_*.sql, elementary/monitor/dbt_project/macros/artifacts/get_elementary_*.sql
Introduced 13 dbt macros: parameterized filter helpers for SQL predicate construction, and backend queries for invocations, models, run results, sources, and tests (both dbt and Elementary variants), each with optional filters, result ordering, and limit support.
CLI Integration
elementary/cli/cli.py, elementary/utils/log.py
Registered artifacts subcommand in main CLI, added artifacts-mode detection to suppress startup logos/messages, forced quiet logging in artifacts mode, and extended log handlers to support stderr output for error reporting.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant CLI as edr artifacts<br/>(Command)
    participant Config as Config<br/>(build_config)
    participant Runner as dbt Runner<br/>(create_artifacts_runner)
    participant Macro as dbt Macro<br/>(run_macro)
    participant Warehouse as Warehouse<br/>(query execution)
    participant Output as Output<br/>(emit_json/table)

    User->>CLI: edr artifacts <subcommand> [filters]
    CLI->>Config: build_config(paths, profile)
    Config-->>CLI: Config instance
    CLI->>Runner: create_artifacts_runner(config, profile)
    Runner-->>CLI: dbt CommandLineDbtRunner<br/>(with profile override if set)
    CLI->>Macro: run_macro(runner, macro_name, filters)
    Macro->>Warehouse: execute SQL query
    Warehouse-->>Macro: result rows
    Macro-->>CLI: JSON or error
    alt Success
        CLI->>CLI: parse_json_field (hydrate data)
        CLI->>CLI: apply_pagination (trim + has_more)
        CLI->>Output: emit_json or emit_table
        Output-->>User: formatted output to stdout
    else ArtifactFetchError
        CLI->>Output: emit_error(code, details)
        Output-->>User: JSON error to stderr
    else Other Exception
        CLI->>Output: emit_error(INTERNAL_ERROR)
        Output-->>User: JSON error to stderr
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

This PR spans 15+ new Python files and 13+ SQL macros with heterogeneous logic: CLI commands with varied filter semantics and pagination patterns, SQL macros with conditional filter composition, Python utilities for error translation and runner patching, and integration points in the main CLI and logging systems. Each entity command exhibits distinct complexity (invocations, test_results, and tests are notably dense), requiring separate reasoning for filters, field hydration, output formatting, and error paths. The total scope (~1600 lines) and architectural breadth demand methodical verification of configuration propagation, macro parameter binding, error handling uniformity, and output structure consistency.

Poem

🐰 Artifacts now flutter and gleam,
Seven new queries fulfill every dream!
With filters and pages and JSON so bright,
The warehouse reveals what's hidden from sight!
Elementary wisdom, at last, in your shell—
Hop forward with confidence; all shall be well!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 32.26% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the primary change: adding a new edr artifacts command group for querying warehouse artifact tables.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch add_artifact_commands

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (8)
elementary/monitor/dbt_project/macros/artifacts/get_dbt_run_result.sql (1)

7-28: Duplicated projection definition with get_dbt_run_results.sql.

The heavy_cols / base_cols blocks are byte-identical to the ones in get_dbt_run_results.sql. If a schema column is added/removed, both macros must be edited in lockstep — easy to miss. Consider extracting into a tiny shared macro (e.g., _dbt_run_results_projection(include_heavy)) in _filter_helpers.sql or a new _projections.sql, and using it from both call sites.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/monitor/dbt_project/macros/artifacts/get_dbt_run_result.sql`
around lines 7 - 28, The projection logic in get_dbt_run_result.sql duplicates
the identical heavy_cols and base_cols definitions from get_dbt_run_results.sql,
which risks drift when schema changes; extract these into a single reusable
macro (e.g., _dbt_run_results_projection(include_heavy)) placed in a shared file
like _filter_helpers.sql or _projections.sql, then replace the
base_cols/heavy_cols and projection construction in both get_dbt_run_result.sql
and get_dbt_run_results.sql to call _dbt_run_results_projection(include_heavy)
so both macros use the same canonical column list.
elementary/monitor/dbt_project/macros/artifacts/get_dbt_models.sql (1)

40-45: Consider a two-column LIKE helper.

The name/alias block is duplicated verbatim in get_dbt_tests.sql (which adds short_name). If you want to trim duplication, a small _like_ci_any_filter(columns, value) helper in _filter_helpers.sql that ORs across a list of columns would collapse both sites. Optional.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/monitor/dbt_project/macros/artifacts/get_dbt_models.sql` around
lines 40 - 45, Create a reusable two-column case-insensitive LIKE helper to
remove the duplicated name/alias logic: add a macro named
_like_ci_any_filter(columns, value) in _filter_helpers.sql that accepts a list
of column names and the value, escapes the value via
elementary_cli._sql_escape(value), and returns an OR-clause like lower(col) LIKE
lower('%...%') for each column (joined with OR). Replace the duplicated block in
get_dbt_models.sql (the name/alias predicate) and the similar block in
get_dbt_tests.sql (which also checks short_name) by calling
_like_ci_any_filter(['name','alias'] , name) or
_like_ci_any_filter(['name','alias','short_name'], name) as appropriate. Ensure
the macro returns a properly parenthesized expression and is only rendered when
value is not none to preserve existing conditional behavior.
elementary/artifacts/entities/models.py (1)

108-117: Consolidate repeated error handling across entity commands.

This except ArtifactFetchError / except Exception block is duplicated verbatim in models.model (L154-163) and across test_results.py, run_results.py (and presumably the other three entities). Consider extracting a small helper or decorator (or reusing cli._handle_fetch_error plus a generic internal-error helper) so every entity command shares one implementation. This also addresses Ruff BLE001 (blind except Exception) in one place instead of six.

♻️ Example helper
# elementary/artifacts/common.py (or output.py)
from functools import wraps

def handle_cli_errors(func):
    `@wraps`(func)
    def wrapper(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except ArtifactFetchError as exc:
            sys.exit(emit_error(str(exc), exc.code, exc.details))
        except Exception as exc:  # noqa: BLE001
            sys.exit(emit_error(
                f"Unexpected error: {exc}",
                ErrorCode.INTERNAL_ERROR,
                {"type": type(exc).__name__},
            ))
    return wrapper

Then decorate each entity command with @handle_cli_errors and drop the inline try/except.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/entities/models.py` around lines 108 - 117, Extract the
duplicated try/except into a single reusable helper (e.g., handle_cli_errors) in
a shared module (like elementary.artifacts.common or output) that catches
ArtifactFetchError and generic Exception and calls emit_error with the same
payloads; alternatively, compose/extend the existing cli._handle_fetch_error by
adding a small internal-error helper that handles the broad Exception case
(using ErrorCode.INTERNAL_ERROR and type(exc).__name__) so callers can reuse it.
Then remove the repeated except blocks in models.model and the commands in
test_results.py, run_results.py, etc., and either decorate those command
functions with `@handle_cli_errors` or call the shared handler wrapper so all
entity commands use one implementation and Ruff BLE001 is isolated in one place.
elementary/artifacts/entities/run_results.py (1)

133-133: Inconsistent macro parameter naming for the same user-facing flag.

The same --include-compiled-code CLI flag is forwarded to the list macro as lightweight=not include_compiled_code (L133) and to the single-get macro as include_compiled_code=include_compiled_code (L182). The inverted sense + different parameter names between get_dbt_run_results and get_dbt_run_result makes future maintenance error-prone (easy to flip the boolean). Consider aligning both SQL macros on one parameter name (e.g., include_compiled_code) and keeping the boolean polarity identical in both call sites.

Also applies to: 182-182

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/entities/run_results.py` at line 133, The list macro
call in get_dbt_run_results uses the parameter name lightweight with inverted
polarity ("lightweight = not include_compiled_code") while get_dbt_run_result
passes include_compiled_code directly; standardize both to the same parameter
name and polarity to avoid confusion—update the call in get_dbt_run_results to
pass "include_compiled_code = include_compiled_code" (or, alternatively, change
get_dbt_run_result to use "lightweight = not include_compiled_code") so both
macros use the identical parameter name (include_compiled_code) and the same
boolean sense; adjust any downstream macro signatures if necessary to accept
include_compiled_code.
elementary/artifacts/cli.py (1)

27-34: Remove unused helpers and fix implicit Optional typing.

The functions _handle_fetch_error and _handle_bad_argument are never called within this module or elsewhere. Meanwhile, all entity modules (models, test_results, run_results, sources, tests, invocations) inline the same sys.exit(emit_error(...)) pattern directly. Either adopt these helpers consistently across entities to eliminate duplication, or remove them. Additionally, the details parameter in _handle_bad_argument should explicitly type None as Optional[Dict[str, Any]] to match the signature of emit_error and avoid Ruff RUF013 violations.

♻️ Proposed refactor
-def _handle_fetch_error(exc: ArtifactFetchError) -> None:
-    code = emit_error(str(exc), exc.code, exc.details)
-    sys.exit(code)
-
-
-def _handle_bad_argument(message: str, details: dict = None) -> None:
-    code = emit_error(message, ErrorCode.BAD_ARGUMENT, details or {})
-    sys.exit(code)
+def _handle_fetch_error(exc: ArtifactFetchError) -> None:
+    sys.exit(emit_error(str(exc), exc.code, exc.details))
+
+
+def _handle_bad_argument(message: str, details: Optional[Dict[str, Any]] = None) -> None:
+    sys.exit(emit_error(message, ErrorCode.BAD_ARGUMENT, details))

If adopting these helpers, add from typing import Any, Dict, Optional and refactor entity modules to use them instead of inlining the pattern.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/cli.py` around lines 27 - 34, The two helpers
_handle_fetch_error and _handle_bad_argument are unused and _handle_bad_argument
has an implicit Optional typing for details; either remove both helpers or
consistently use them across entity modules instead of inlining
sys.exit(emit_error(...)), and if you keep _handle_bad_argument update its
signature to explicitly type details as Optional[Dict[str, Any]] and add the
required imports (Any, Dict, Optional) so the parameter matches emit_error and
avoids RUF013; ensure references to _handle_fetch_error (ArtifactFetchError) and
_handle_bad_argument are wired into callers or deleted to eliminate dead code.
elementary/artifacts/entities/invocations.py (2)

203-221: Redundant data.length field and _emit_single table uses list-only columns.

Two small output concerns:

  1. Line 212: "data": {"length": len(rows)} is redundant with the top-level "count" already emitted on line 209. If this isn't load-bearing for a documented contract it's dead weight in the envelope and risks becoming a stale parallel counter.
  2. _emit_single renders the detail record through LIST_COLUMNS, so edr artifacts invocation <id> -o table hides fields that the JSON path exposes (e.g. job_id, job_run_id, timing breakdowns). The JSON envelope already uses a distinct key ("invocation" vs "invocations"); a separate DETAIL_COLUMNS (or pivoted key/value rendering for a single row) would be more useful and mirror what tests.py does with DETAIL_JSON_FIELDS.
♻️ Suggested cleanup for the redundant field
     emit_json(
         {
             "count": len(rows),
             "has_more": has_more,
             "invocations": rows,
-            "data": {"length": len(rows)},
         }
     )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/entities/invocations.py` around lines 203 - 221, The
_emit_list function emits a redundant nested data.length that duplicates the
top-level count—remove the "data": {"length": ...} entry so _emit_list only
returns count, has_more, and invocations via emit_json; and update _emit_single
to render table output using a richer column set instead of LIST_COLUMNS
(introduce or use DETAIL_COLUMNS or a pivoted key/value rendering) so
emit_table([row], DETAIL_COLUMNS) (or equivalent key/value display) shows all
detail fields like job_id, job_run_id, and timing breakdowns while JSON output
remains emit_json({"invocation": row}).

192-200: Consider validating user-supplied --started-after/--started-before locally.

When the user passes these flags, the strings are forwarded verbatim to the macro without parsing. A typo like 2024-13-01 will travel all the way to the warehouse and surface as a WAREHOUSE_ERROR, which is a noisy failure mode for a client-side mistake. Parsing with datetime.fromisoformat (or click.DateTime) here would produce a clean USER_ERROR with a clear message and also let you normalize to UTC consistently with the defaults.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/entities/invocations.py` around lines 192 - 200, The
_resolve_time_window function currently forwards user-supplied
started_after/started_before strings verbatim; parse each non-None value with
datetime.fromisoformat (or equivalent) inside _resolve_time_window, catch
ValueError and propagate a clear USER_ERROR (or raise a ValueError with a
descriptive message) so typos like "2024-13-01" fail locally, normalize parsed
datetimes to UTC (use astimezone(timezone.utc)) and then return their
.isoformat() strings; keep the same return signature and treat None by
substituting the existing defaults before formatting.
elementary/artifacts/entities/tests.py (1)

209-227: Same redundant data.length field; detail table ignores DETAIL_JSON_FIELDS.

  1. Line 218: "data": {"length": len(rows)} duplicates the top-level "count" on line 215 — same observation as in invocations.py. Recommend dropping unless it's part of a documented stable contract.
  2. _emit_single renders the single test via LIST_COLUMNS, which intentionally omits the richer fields hydrated by DETAIL_JSON_FIELDS (test_params, meta, depends_on_*, etc.). For edr artifacts test <unique_id> -o table that means the table output is strictly less informative than the JSON output, which is surprising given the detail hydration above it. A dedicated DETAIL_COLUMNS (or a two-column key/value rendering for single-record output) would make the hydration actually observable.
♻️ Suggested cleanup for the redundant field
     emit_json(
         {
             "count": len(rows),
             "has_more": has_more,
             "tests": rows,
-            "data": {"length": len(rows)},
         }
     )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/entities/tests.py` around lines 209 - 227, Remove the
redundant "data": {"length": ...} in _emit_list (it duplicates "count") and stop
emitting it unless it's a documented contract; in _emit_single, change the table
rendering to show the detailed fields instead of LIST_COLUMNS so the hydrated
DETAIL_JSON_FIELDS are visible — either use a new DETAIL_COLUMNS constant (or a
two-column key/value renderer) when calling emit_table([row], ...) in
_emit_single, or branch to a dedicated detail-table helper; keep emit_json
behavior unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@elementary/artifacts/entities/test_results.py`:
- Line 2: The import list in elementary.artifacts.entities.test_results
currently includes Optional but it's unused; remove Optional from the from
typing import Any, Dict, List, Optional statement (leaving Any, Dict, List) to
satisfy flake8 F401 and avoid unused imports in this module (check the top-level
import line in test_results.py).

In `@elementary/artifacts/fetching.py`:
- Around line 45-54: The json.loads call in fetch logic can raise TypeError for
non-string elements (e.g. None/dict/bytes), so update the error handling in the
block around json.loads(response[0]) (in elementary.artifacts.fetching.py) to
also catch TypeError (and you can drop IndexError because of the preceding if
not response guard); ensure ArtifactFetchError is raised with the same
ErrorCode.MALFORMED_ARTIFACTS and include the raw response[0] in the payload as
currently done so `[None]` or other non-string values produce the intended
ArtifactFetchError instead of an unhandled TypeError.

In `@elementary/artifacts/runner.py`:
- Around line 33-41: The wrapper _inject_profile_override currently calls the
saved original as original(command_args=new_args, *args, **kwargs) which both
violates the star-arg-after-keyword rule and can raise TypeError if positional
args are passed; change the call in patched to pass new_args positionally (e.g.,
original(new_args, *args, **kwargs)) so command_args is supplied as a positional
argument and remove the keyword form; keep references to patched, original and
runner._run_command when updating. Also consider the optional durability
improvement of avoiding monkey-patching runner._run_command by subclassing
CommandLineDbtRunner or threading the profile through create_dbt_runner instead
of patching the private method.

In `@elementary/monitor/dbt_project/macros/artifacts/_filter_helpers.sql`:
- Around line 23-27: The _like_ci_filter macro uses
elementary_cli._sql_escape(value) but does not escape SQL LIKE wildcards (%) and
(_) so user input like "foo_bar" matches unexpectedly; update _like_ci_filter to
additionally replace '%' and '_' in the escaped value with '\%' and '\_'
(ensuring proper escaping of the backslash in the SQL literal) and append an
"ESCAPE '\'" clause to the LIKE expression so the backslashes are honored; keep
the case-insensitive lower(...) wrappers and perform the wildcard-escaping on
the result of elementary_cli._sql_escape(value) before embedding it into the
pattern.
- Around line 57-67: The numeric filter macros (_gt_number_filter and
_lt_number_filter) currently inline raw {{ value }} into SQL; coerce and
validate the value before emitting it to prevent injection. Update both macros
to convert value to a numeric type (e.g., use Jinja's int or float filter) and
only render the "and {{ column }} > ..." / "< ..." clause when the coercion
succeeds (keep the existing "if value is not none" guard but replace raw
insertion with the coerced numeric), so the emitted SQL always contains a
validated numeric literal; reference the macros _gt_number_filter and
_lt_number_filter when making this change.

In `@elementary/monitor/dbt_project/macros/artifacts/get_dbt_run_results.sql`:
- Around line 57-58: The ORDER BY in get_dbt_run_results.sql uses only
execute_started_at which is not a total order and can cause pagination
instability; update the ORDER BY clause (the line containing "order by
execute_started_at desc") to add a deterministic tiebreaker using a unique
column (for example run_id or the table primary key column) so it becomes a
total order (e.g., ORDER BY execute_started_at DESC, <unique_column> DESC) while
keeping the existing LIMIT handling intact.

---

Nitpick comments:
In `@elementary/artifacts/cli.py`:
- Around line 27-34: The two helpers _handle_fetch_error and
_handle_bad_argument are unused and _handle_bad_argument has an implicit
Optional typing for details; either remove both helpers or consistently use them
across entity modules instead of inlining sys.exit(emit_error(...)), and if you
keep _handle_bad_argument update its signature to explicitly type details as
Optional[Dict[str, Any]] and add the required imports (Any, Dict, Optional) so
the parameter matches emit_error and avoids RUF013; ensure references to
_handle_fetch_error (ArtifactFetchError) and _handle_bad_argument are wired into
callers or deleted to eliminate dead code.

In `@elementary/artifacts/entities/invocations.py`:
- Around line 203-221: The _emit_list function emits a redundant nested
data.length that duplicates the top-level count—remove the "data": {"length":
...} entry so _emit_list only returns count, has_more, and invocations via
emit_json; and update _emit_single to render table output using a richer column
set instead of LIST_COLUMNS (introduce or use DETAIL_COLUMNS or a pivoted
key/value rendering) so emit_table([row], DETAIL_COLUMNS) (or equivalent
key/value display) shows all detail fields like job_id, job_run_id, and timing
breakdowns while JSON output remains emit_json({"invocation": row}).
- Around line 192-200: The _resolve_time_window function currently forwards
user-supplied started_after/started_before strings verbatim; parse each non-None
value with datetime.fromisoformat (or equivalent) inside _resolve_time_window,
catch ValueError and propagate a clear USER_ERROR (or raise a ValueError with a
descriptive message) so typos like "2024-13-01" fail locally, normalize parsed
datetimes to UTC (use astimezone(timezone.utc)) and then return their
.isoformat() strings; keep the same return signature and treat None by
substituting the existing defaults before formatting.

In `@elementary/artifacts/entities/models.py`:
- Around line 108-117: Extract the duplicated try/except into a single reusable
helper (e.g., handle_cli_errors) in a shared module (like
elementary.artifacts.common or output) that catches ArtifactFetchError and
generic Exception and calls emit_error with the same payloads; alternatively,
compose/extend the existing cli._handle_fetch_error by adding a small
internal-error helper that handles the broad Exception case (using
ErrorCode.INTERNAL_ERROR and type(exc).__name__) so callers can reuse it. Then
remove the repeated except blocks in models.model and the commands in
test_results.py, run_results.py, etc., and either decorate those command
functions with `@handle_cli_errors` or call the shared handler wrapper so all
entity commands use one implementation and Ruff BLE001 is isolated in one place.

In `@elementary/artifacts/entities/run_results.py`:
- Line 133: The list macro call in get_dbt_run_results uses the parameter name
lightweight with inverted polarity ("lightweight = not include_compiled_code")
while get_dbt_run_result passes include_compiled_code directly; standardize both
to the same parameter name and polarity to avoid confusion—update the call in
get_dbt_run_results to pass "include_compiled_code = include_compiled_code" (or,
alternatively, change get_dbt_run_result to use "lightweight = not
include_compiled_code") so both macros use the identical parameter name
(include_compiled_code) and the same boolean sense; adjust any downstream macro
signatures if necessary to accept include_compiled_code.

In `@elementary/artifacts/entities/tests.py`:
- Around line 209-227: Remove the redundant "data": {"length": ...} in
_emit_list (it duplicates "count") and stop emitting it unless it's a documented
contract; in _emit_single, change the table rendering to show the detailed
fields instead of LIST_COLUMNS so the hydrated DETAIL_JSON_FIELDS are visible —
either use a new DETAIL_COLUMNS constant (or a two-column key/value renderer)
when calling emit_table([row], ...) in _emit_single, or branch to a dedicated
detail-table helper; keep emit_json behavior unchanged.

In `@elementary/monitor/dbt_project/macros/artifacts/get_dbt_models.sql`:
- Around line 40-45: Create a reusable two-column case-insensitive LIKE helper
to remove the duplicated name/alias logic: add a macro named
_like_ci_any_filter(columns, value) in _filter_helpers.sql that accepts a list
of column names and the value, escapes the value via
elementary_cli._sql_escape(value), and returns an OR-clause like lower(col) LIKE
lower('%...%') for each column (joined with OR). Replace the duplicated block in
get_dbt_models.sql (the name/alias predicate) and the similar block in
get_dbt_tests.sql (which also checks short_name) by calling
_like_ci_any_filter(['name','alias'] , name) or
_like_ci_any_filter(['name','alias','short_name'], name) as appropriate. Ensure
the macro returns a properly parenthesized expression and is only rendered when
value is not none to preserve existing conditional behavior.

In `@elementary/monitor/dbt_project/macros/artifacts/get_dbt_run_result.sql`:
- Around line 7-28: The projection logic in get_dbt_run_result.sql duplicates
the identical heavy_cols and base_cols definitions from get_dbt_run_results.sql,
which risks drift when schema changes; extract these into a single reusable
macro (e.g., _dbt_run_results_projection(include_heavy)) placed in a shared file
like _filter_helpers.sql or _projections.sql, then replace the
base_cols/heavy_cols and projection construction in both get_dbt_run_result.sql
and get_dbt_run_results.sql to call _dbt_run_results_projection(include_heavy)
so both macros use the same canonical column list.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a4a3fbd7-a217-479e-83a6-6c54794d53ed

📥 Commits

Reviewing files that changed from the base of the PR and between e5af7e7 and 3c2b9fa.

📒 Files selected for processing (29)
  • elementary/artifacts/README.md
  • elementary/artifacts/__init__.py
  • elementary/artifacts/cli.py
  • elementary/artifacts/common.py
  • elementary/artifacts/entities/__init__.py
  • elementary/artifacts/entities/invocations.py
  • elementary/artifacts/entities/models.py
  • elementary/artifacts/entities/run_results.py
  • elementary/artifacts/entities/sources.py
  • elementary/artifacts/entities/test_results.py
  • elementary/artifacts/entities/tests.py
  • elementary/artifacts/fetching.py
  • elementary/artifacts/output.py
  • elementary/artifacts/runner.py
  • elementary/cli/cli.py
  • elementary/monitor/dbt_project/macros/artifacts/_filter_helpers.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_invocation.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_invocations.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_model.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_models.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_run_result.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_run_results.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_source.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_sources.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_test.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_dbt_tests.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_elementary_test_result.sql
  • elementary/monitor/dbt_project/macros/artifacts/get_elementary_test_results.sql
  • elementary/utils/log.py

@@ -0,0 +1,214 @@
import sys
from typing import Any, Dict, List, Optional
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Remove unused Optional import.

Flake8 F401: Optional is imported but not used in this module.

-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
from typing import Any, Dict, List, Optional
from typing import Any, Dict, List
🧰 Tools
🪛 Flake8 (7.3.0)

[error] 2-2: 'typing.Optional' imported but unused

(F401)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/entities/test_results.py` at line 2, The import list in
elementary.artifacts.entities.test_results currently includes Optional but it's
unused; remove Optional from the from typing import Any, Dict, List, Optional
statement (leaving Any, Dict, List) to satisfy flake8 F401 and avoid unused
imports in this module (check the top-level import line in test_results.py).

Comment on lines +45 to +54
if not response:
return None
try:
return json.loads(response[0])
except (ValueError, IndexError) as exc:
raise ArtifactFetchError(
f"Malformed response from {macro_name}.",
ErrorCode.MALFORMED_ARTIFACTS,
{"raw": response[0] if response else None},
) from exc
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

json.loads can raise TypeError, which escapes the MALFORMED_ARTIFACTS path.

json.loads(response[0]) only raises ValueError/IndexError when response[0] is a string or the sequence is empty. If the dbt runner ever returns [None] or a non-string element (e.g. a dict, or a bytes payload on some adapters), json.loads raises TypeError, which is not in the caught tuple on line 49. The exception then propagates past ArtifactFetchError and gets reported by the entity commands as a generic INTERNAL_ERROR with an ugly message instead of the intended stable MALFORMED_ARTIFACTS envelope — undermining the "stable error contract" the PR is specifically trying to establish.

Separately, line 46's if not response: return None treats an empty list the same as no response at all, which seems fine, but note that [None] is truthy and will fall through to json.loads(None) — that's the concrete case that hits the TypeError gap above.

🛡️ Proposed fix
     if not response:
         return None
+    payload = response[0]
+    if payload is None:
+        return None
     try:
-        return json.loads(response[0])
-    except (ValueError, IndexError) as exc:
+        return json.loads(payload)
+    except (TypeError, ValueError) as exc:
         raise ArtifactFetchError(
             f"Malformed response from {macro_name}.",
             ErrorCode.MALFORMED_ARTIFACTS,
-            {"raw": response[0] if response else None},
+            {"raw": payload},
         ) from exc

(IndexError is no longer reachable after the if not response guard, so it can be dropped.)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/fetching.py` around lines 45 - 54, The json.loads call
in fetch logic can raise TypeError for non-string elements (e.g.
None/dict/bytes), so update the error handling in the block around
json.loads(response[0]) (in elementary.artifacts.fetching.py) to also catch
TypeError (and you can drop IndexError because of the preceding if not response
guard); ensure ArtifactFetchError is raised with the same
ErrorCode.MALFORMED_ARTIFACTS and include the raw response[0] in the payload as
currently done so `[None]` or other non-string values produce the intended
ArtifactFetchError instead of an unhandled TypeError.

Comment on lines +33 to +41
original = runner._run_command

def patched(self, command_args, *args, **kwargs):
new_args = list(command_args)
if new_args:
new_args = [new_args[0], "--profile", profile] + new_args[1:]
return original(command_args=new_args, *args, **kwargs)

runner._run_command = types.MethodType(patched, runner)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Locate the definition and all call sites of _run_command on the dbt runner.
ast-grep --pattern 'def _run_command($$$): $$$'
rg -nP --type=py -C2 '\b_run_command\s*\('

Repository: elementary-data/elementary

Length of output: 5172


🏁 Script executed:

# Get the full _run_command signature
cat -n elementary/clients/dbt/command_line_dbt_runner.py | sed -n '170,190p'

Repository: elementary-data/elementary

Length of output: 974


🏁 Script executed:

# Examine the _inject_profile_override function in the file under review
cat -n elementary/artifacts/runner.py | head -50

Repository: elementary-data/elementary

Length of output: 1862


🏁 Script executed:

# Check what parameters _run_command actually accepts
rg -A 15 'def _run_command\(' elementary/clients/dbt/command_line_dbt_runner.py

Repository: elementary-data/elementary

Length of output: 674


Fix star-arg after keyword violation and latent TypeError in _run_command wrapper.

Two concerns in _inject_profile_override:

  1. Line 39 uses original(command_args=new_args, *args, **kwargs), which violates Ruff B026 (star-arg after keyword). More critically, if any element ever appears in *args, Python evaluates it positionally for command_args, causing TypeError: got multiple values for keyword argument 'command_args'. While no current caller passes extra positional args, this is a trivial footgun to remove: pass new_args positionally instead.

  2. Wrapping the private _run_command couples this module to an internal method of CommandLineDbtRunner. If the base class renames or changes the signature of this hook, --profile injection silently breaks. A subclass override or threading profile through create_dbt_runner would be more durable. Optional, but worth considering.

🛠️ Proposed fix for (1)
     def patched(self, command_args, *args, **kwargs):
         new_args = list(command_args)
         if new_args:
-            new_args = [new_args[0], "--profile", profile] + new_args[1:]
-        return original(command_args=new_args, *args, **kwargs)
+            new_args = [new_args[0], "--profile", profile, *new_args[1:]]
+        return original(new_args, *args, **kwargs)
🧰 Tools
🪛 Ruff (0.15.10)

[warning] 38-38: Consider [new_args[0], "--profile", profile, *new_args[1:]] instead of concatenation

Replace with [new_args[0], "--profile", profile, *new_args[1:]]

(RUF005)


[warning] 39-39: Star-arg unpacking after a keyword argument is strongly discouraged

(B026)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/artifacts/runner.py` around lines 33 - 41, The wrapper
_inject_profile_override currently calls the saved original as
original(command_args=new_args, *args, **kwargs) which both violates the
star-arg-after-keyword rule and can raise TypeError if positional args are
passed; change the call in patched to pass new_args positionally (e.g.,
original(new_args, *args, **kwargs)) so command_args is supplied as a positional
argument and remove the keyword form; keep references to patched, original and
runner._run_command when updating. Also consider the optional durability
improvement of avoiding monkey-patching runner._run_command by subclassing
CommandLineDbtRunner or threading the profile through create_dbt_runner instead
of patching the private method.

Comment on lines +23 to +27
{%- macro _like_ci_filter(column, value) -%}
{%- if value is not none -%}
and lower({{ column }}) like lower('%{{ elementary_cli._sql_escape(value) }}%')
{%- endif -%}
{%- endmacro -%}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

_like_ci_filter does not escape LIKE wildcards.

% and _ in user input are treated as wildcards, so a search like --name "foo_bar" will unexpectedly match fooXbar. Since the only escaping done is for single quotes, users cannot search for literal %/_. Not a correctness blocker, but worth documenting or escaping (replace('%','\\%') / replace('_','\\_') plus an ESCAPE '\' clause) if literal matching is desired. Minor UX nit given the CLI context.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/monitor/dbt_project/macros/artifacts/_filter_helpers.sql` around
lines 23 - 27, The _like_ci_filter macro uses elementary_cli._sql_escape(value)
but does not escape SQL LIKE wildcards (%) and (_) so user input like "foo_bar"
matches unexpectedly; update _like_ci_filter to additionally replace '%' and '_'
in the escaped value with '\%' and '\_' (ensuring proper escaping of the
backslash in the SQL literal) and append an "ESCAPE '\'" clause to the LIKE
expression so the backslashes are honored; keep the case-insensitive lower(...)
wrappers and perform the wildcard-escaping on the result of
elementary_cli._sql_escape(value) before embedding it into the pattern.

Comment on lines +57 to +67
{%- macro _gt_number_filter(column, value) -%}
{%- if value is not none -%}
and {{ column }} > {{ value }}
{%- endif -%}
{%- endmacro -%}

{%- macro _lt_number_filter(column, value) -%}
{%- if value is not none -%}
and {{ column }} < {{ value }}
{%- endif -%}
{%- endmacro -%}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Numeric filters embed raw value without coercion/validation.

_gt_number_filter / _lt_number_filter inline {{ value }} directly into SQL. Unlike the string filters (which go through _sql_escape), these have no hygiene at all — any non-numeric input (e.g., user passing 1; drop table x) would be emitted verbatim. Even though the CLI is the only caller today, a cheap coercion keeps parity with the rest of this module and protects against future misuse.

🛡️ Proposed fix
 {%- macro _gt_number_filter(column, value) -%}
     {%- if value is not none -%}
-        and {{ column }} > {{ value }}
+        and {{ column }} > {{ value | float }}
     {%- endif -%}
 {%- endmacro -%}

 {%- macro _lt_number_filter(column, value) -%}
     {%- if value is not none -%}
-        and {{ column }} < {{ value }}
+        and {{ column }} < {{ value | float }}
     {%- endif -%}
 {%- endmacro -%}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
{%- macro _gt_number_filter(column, value) -%}
{%- if value is not none -%}
and {{ column }} > {{ value }}
{%- endif -%}
{%- endmacro -%}
{%- macro _lt_number_filter(column, value) -%}
{%- if value is not none -%}
and {{ column }} < {{ value }}
{%- endif -%}
{%- endmacro -%}
{%- macro _gt_number_filter(column, value) -%}
{%- if value is not none -%}
and {{ column }} > {{ value | float }}
{%- endif -%}
{%- endmacro -%}
{%- macro _lt_number_filter(column, value) -%}
{%- if value is not none -%}
and {{ column }} < {{ value | float }}
{%- endif -%}
{%- endmacro -%}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/monitor/dbt_project/macros/artifacts/_filter_helpers.sql` around
lines 57 - 67, The numeric filter macros (_gt_number_filter and
_lt_number_filter) currently inline raw {{ value }} into SQL; coerce and
validate the value before emitting it to prevent injection. Update both macros
to convert value to a numeric type (e.g., use Jinja's int or float filter) and
only render the "and {{ column }} > ..." / "< ..." clause when the coercion
succeeds (keep the existing "if value is not none" guard but replace raw
insertion with the coerced numeric), so the emitted SQL always contains a
validated numeric literal; reference the macros _gt_number_filter and
_lt_number_filter when making this change.

Comment on lines +57 to +58
order by execute_started_at desc
limit {{ limit }}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Pagination ordering may be non-deterministic on ties.

order by execute_started_at desc alone is not a total order — rows sharing the same execute_started_at can shuffle between pages under limit/limit+1 pagination, causing duplicates or skips when the CLI stitches pages. Adding a tiebreaker on a unique column keeps pagination stable.

🛡️ Proposed fix
-        order by execute_started_at desc
+        order by execute_started_at desc, model_execution_id desc
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
order by execute_started_at desc
limit {{ limit }}
order by execute_started_at desc, model_execution_id desc
limit {{ limit }}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@elementary/monitor/dbt_project/macros/artifacts/get_dbt_run_results.sql`
around lines 57 - 58, The ORDER BY in get_dbt_run_results.sql uses only
execute_started_at which is not a total order and can cause pagination
instability; update the ORDER BY clause (the line containing "order by
execute_started_at desc") to add a deterministic tiebreaker using a unique
column (for example run_id or the table primary key column) so it becomes a
total order (e.g., ORDER BY execute_started_at DESC, <unique_column> DESC) while
keeping the existing LIMIT handling intact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant