Add Data Metric Function cleanup support (#28)

DMFs are now cleaned as a separate object type to prevent them from being
incorrectly dropped by the regular function cleanup macro.

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-authored-by: Cortex Code <noreply@snowflake.com>

Author: jonhopper-dataengineers

CHANGELOG.md

@@ -1,6 +1,17 @@
 # Data Engineers Snowflake DataOps Utils Project Changelog
 This file contains the changelog for the Data Engineers Snowflake DataOps Utils project, detailing updates, fixes, and enhancements made to the project over time.
 
+## v1.0.3 - 2026-05-16 - Data Metric Function Cleanup Support
+
+### Added
+- Added `clean_data_metric_functions` macro to reconcile dbt-defined Data Metric Functions (DMFs) against deployed DMFs in Snowflake and drop orphaned ones. DMFs are identified via `information_schema.functions` where `is_data_metric = 'YES'` and matched against dbt graph nodes with `config.materialized = 'data_metric_function'`.
+- Added `data_metric_functions` as a supported `object_type` in the `clean_objects` orchestrator macro (included in the default list).
+
+### Changed
+- Modified `clean_functions` macro to exclude Data Metric Functions (`is_data_metric = 'NO'` filter) so DMFs are not incorrectly dropped by the regular function cleanup. DMFs are now handled exclusively by `clean_data_metric_functions`.
+- Added `{% if execute %}` guards to all clean macros (`clean_functions`, `clean_generic`, `clean_models`, `clean_schemas`, `clean_stale_models`) to prevent `run_query()` and `graph.nodes` access during parsing or docs generation.
+- Bumped version from 1.0.2 to 1.0.3
+
 ## v1.0.2 - 2026-05-12 - Clean Functions Signature Fallback Fix
 
 ### Fixed

README.md

@@ -2,7 +2,7 @@
 
 A macro-only [dbt](https://github.com/dbt-labs/dbt) package for Snowflake DataOps. Provides utilities for object lifecycle management, RBAC grant orchestration, dimensional modelling helpers, tagging, shares, and more.
 
-- **Version**: 1.0.0
+- **Version**: 1.0.3
 - **dbt**: `>=1.3.0, <3.0.0`
 - **Dependencies**: None (zero external package dependencies)
 - **dbt Fusion**: Compatible
@@ -16,7 +16,7 @@ Add the following to your `packages.yml`:
 ```yaml
 packages:
   - git: https://github.com/DataEngineersNZ/dbt-snowflake-datops-utils.git
-    revision: "1.0.0"
+    revision: "1.0.3"
 ```
 
 Then run `dbt deps`.
@@ -62,7 +62,8 @@ The following `vars` can be set in your `dbt_project.yml` or via `--vars` on the
 | `clean_objects(database, clean_targets, object_types)` | Orchestrate all clean macros for specified object types and environments |
 | `clean_schemas(database, dry_run)` | Drop schemas not defined in the dbt project |
 | `clean_models(database, dry_run)` | Drop orphaned tables/views/dynamic tables/external tables/materialized views |
-| `clean_functions(database, dry_run)` | Drop orphaned UDFs and stored procedures |
+| `clean_functions(database, dry_run)` | Drop orphaned UDFs and stored procedures (excludes DMFs) |
+| `clean_data_metric_functions(database, dry_run)` | Drop orphaned Data Metric Functions (DMFs) |
 | `clean_generic(object_type, database, dry_run)` | Drop orphaned tasks/streams/stages/alerts/file formats/network rules/secrets/semantic views/agents |
 | `clean_stale_models(database, schema, days, dry_run)` | Drop models older than N days from a specific schema |
 
@@ -286,7 +287,7 @@ The `clean_objects` macro orchestrates removal of Snowflake objects not defined
 dbt run-operation clean_objects --args '{"clean_targets": ["prod"], "object_types": ["schemas", "tables_and_views", "functions_and_procedures"]}'
 ```
 
-Supported `object_types`: `schemas`, `functions_and_procedures`, `tasks`, `streams`, `stages`, `tables_and_views`, `alerts`, `file_formats`, `semantic_views`, `agents`.
+Supported `object_types`: `schemas`, `functions_and_procedures`, `data_metric_functions`, `tasks`, `streams`, `stages`, `tables_and_views`, `alerts`, `file_formats`, `semantic_views`, `agents`.
 
 All clean macros support `dry_run` mode (default: `true`) to preview drops before executing.
 

dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'dbt_dataengineers_utils'
-version: '1.0.2'
+version: '1.0.3'
 config-version: 2
 
 require-dbt-version: [">=1.9.4", "<3.0.0"]

macros/clean/clean.yml

@@ -56,6 +56,18 @@ macros:
         description: List of object types to clean up
 
 
+  - name: clean_data_metric_functions
+    description: Reconcile dbt-defined data metric functions vs deployed; drop any orphaned DMFs in Snowflake.
+    docs:
+      show: true
+    arguments:
+      - name: database
+        type: string
+        description: target database
+      - name: dry_run
+        type: boolean
+        description: specifies if the macro should run in dry run mode
+
   - name: clean_schemas
     description: Reconcile schema list vs dbt; drop schemas no longer defined.
     docs:

macros/clean/clean_data_metric_functions.sql

@@ -0,0 +1,45 @@
+{% macro clean_data_metric_functions(database=target.database, dry_run=True) %}
+    {% if execute %}
+    {% set snowflake_dmfs_to_drop = [] %}
+    {% set nodes = graph.nodes.values() if graph.nodes else [] %}
+
+    {% set get_snowflake_dmfs %}
+        SELECT
+            function_schema AS schema,
+            function_name AS name,
+            argument_signature AS argument_signature
+        FROM {{ database }}.information_schema.functions
+        WHERE is_data_metric = 'YES'
+        AND function_schema NOT IN ('INFORMATION_SCHEMA', 'META', 'PUBLIC', 'PUBLIC_META', 'SCHEMACHANGE', 'UNIT_TESTS')
+    {% endset %}
+
+    {% set snowflake_dmf_results = run_query(get_snowflake_dmfs) %}
+
+    {% for result in snowflake_dmf_results %}
+        {% set dbt_models = [] %}
+        {% set sql_object_schema = result.values()[0] %}
+        {% set sql_object_name = result.values()[1] %}
+        {% set sql_arguments = result.values()[2] %}
+
+        {# Match against dbt graph nodes with materialized='data_metric_function' #}
+        {# Uses selectattr for materialized check (consistent with clean_generic) #}
+        {# Then checks both node.name and override_name (consistent with has_matching_nodes) #}
+        {% set matching_nodes = nodes
+            | selectattr("schema", "equalto", sql_object_schema | lower)
+            | selectattr("config.materialized", "equalto", "data_metric_function")
+        %}
+        {% for node in matching_nodes %}
+            {% set node_name = node.config.get("meta", {}).get("override_name", node.config.get("override_name", node.name)) %}
+            {% if node_name | lower == sql_object_name | lower %}
+                {% do dbt_models.append(node.schema ~ "." ~ node.name) %}
+            {% endif %}
+        {% endfor %}
+
+        {% if dbt_models | length == 0 %}
+            {% do snowflake_dmfs_to_drop.append(sql_object_schema ~ "." ~ sql_object_name ~ sql_arguments) %}
+        {% endif %}
+    {% endfor %}
+
+    {% do dbt_dataengineers_utils.drop_object("FUNCTION", database, snowflake_dmfs_to_drop, dry_run) %}
+    {% endif %}
+{% endmacro %}

macros/clean/clean_functions.sql

@@ -1,4 +1,5 @@
 {% macro clean_functions(database=target.database, dry_run=True) %}
+    {% if execute %}
     {% set snowflake_functions_to_drop = [] %}
     {% set snowflake_procedures_to_drop = [] %}
     {% set ns = namespace(sql_arguments="") %}
@@ -13,6 +14,7 @@
             argument_signature AS argument_signature
         FROM {{ database }}.information_schema.functions
         WHERE schema NOT IN ('INFORMATION_SCHEMA', 'META', 'PUBLIC', 'PUBLIC_META', 'SCHEMACHANGE', 'UNIT_TESTS')
+        AND is_data_metric = 'NO'
         UNION ALL
         SELECT
             'PROCEDURE' AS object_type,
@@ -71,4 +73,5 @@
     {% do dbt_dataengineers_utils.drop_object("PROCEDURE", database, snowflake_procedures_to_drop, dry_run) %}
     {% do dbt_dataengineers_utils.drop_object("FUNCTION", database, snowflake_functions_to_drop, dry_run) %}
 
+    {% endif %}
 {% endmacro %}

macros/clean/clean_generic.sql

@@ -1,4 +1,5 @@
 {% macro clean_generic(object_type, database=target.database, dry_run=True) %}
+    {% if execute %}
     {% set snowflake_objects_to_drop = [] %}
     {% set nodes = graph.nodes.values() if graph.nodes else [] %}
     {% set schema_index = 3 %}
@@ -48,4 +49,5 @@
     {% endfor %}
 
     {% do dbt_dataengineers_utils.drop_object(object_type, database, snowflake_objects_to_drop, dry_run) %}
+    {% endif %}
 {% endmacro %}

macros/clean/clean_models.sql

@@ -1,4 +1,5 @@
 {% macro clean_models(database=target.database, dry_run=True) %}
+    {% if execute %}
     {% set snowflake_views_to_drop = [] %}
     {% set snowflake_materialized_views_to_drop = [] %}
     {% set snowflake_tables_to_drop = [] %}
@@ -67,4 +68,5 @@
     {% do dbt_dataengineers_utils.drop_object("TABLE", database, snowflake_tables_to_drop, dry_run) %}
     {% do dbt_dataengineers_utils.drop_object("EXTERNAL TABLE", database, snowflake_external_tables_to_drop, dry_run) %}
     {% do dbt_dataengineers_utils.drop_object("DYNAMIC TABLE", database, snowflake_dynamic_tables_to_drop, dry_run) %}
+    {% endif %}
 {% endmacro %}

macros/clean/clean_objects.sql

@@ -1,4 +1,4 @@
-{% macro clean_objects(database=target.database, clean_targets=['local-dev', 'unit-test', 'test', 'prod'], object_types= ['schemas', 'functions_and_procedures', 'tasks', 'streams', 'stages', 'tables_and_views', 'alerts', 'file_formats', 'semantic_views', 'agents']) %}
+{% macro clean_objects(database=target.database, clean_targets=['local-dev', 'unit-test', 'test', 'prod'], object_types= ['schemas', 'functions_and_procedures', 'data_metric_functions', 'tasks', 'streams', 'stages', 'tables_and_views', 'alerts', 'file_formats', 'semantic_views', 'agents']) %}
     {%if execute %}
         {% if flags.WHICH in ('run', 'run-operation') %}
             {% if target.name in clean_targets %}
@@ -12,6 +12,9 @@
             {% if 'functions_and_procedures' in object_types %}
                 {% do dbt_dataengineers_utils.clean_functions(database, dry_run) %}
             {% endif %}
+            {% if 'data_metric_functions' in object_types %}
+                {% do dbt_dataengineers_utils.clean_data_metric_functions(database, dry_run) %}
+            {% endif %}
             {% if 'tasks' in object_types %}
                 {% do dbt_dataengineers_utils.clean_generic("TASK", database, dry_run) %}
             {% endif %}

macros/clean/clean_schemas.sql

@@ -1,4 +1,5 @@
 {% macro clean_schemas(database=target.database, dry_run=True) %}
+    {% if execute %}
     {% set snowflake_schemas_to_drop = [] %}
     {% set nodes = graph.nodes.values() if graph.nodes else [] %}
     {% set sources = graph.sources.values() if graph.sources else [] %}
@@ -28,4 +29,5 @@
     {% endfor %}
 
     {% do dbt_dataengineers_utils.drop_object("SCHEMA", database, snowflake_schemas_to_drop, dry_run) %}
+    {% endif %}
 {% endmacro %}