Skip to content

column-level-lineage.mdx

Latest commit

 

History

History
52 lines (32 loc) · 1.86 KB

column-level-lineage.mdx

File metadata and controls

52 lines (32 loc) · 1.86 KB
title Column-Level Lineage
sidebarTitle Column level lineage

import CloudFeatureTag from '/snippets/cloud/cloud-feature-tag.mdx';

The table nodes in Elementary lineage can be expanded to show the columns. When you select a column, the lineage of that specific column will be highlighted.

Column-level lineage is useful for answering questions such as:

  • Which downstream columns are actually impacted by a data quality issue?

  • Can we deprecate or rename a column?

  • Will changing this column impact a dashboard?

Filter and highlight columns path

To help navigate graphs with large amount of columns per table, use the ... menu to the right of the column:

  • Filter: Will show a graph of only the selected column and its dependencies.

  • Highlight: Will highlight only the selected column and its dependencies.

<iframe width="700" height="400" src="https://res.cloudinary.com/diuctyblm/video/upload/v1731591441/Elementary%20Cloud%20Platform%20Demo%20Videos/Column-level_lineage_buotsd.mp4" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen alt="Elementary Lineage" />

Column-level lineage generation

Elementary parses SQL queries to determine the dependencies between columns. Note that the lineage is only of the columns that directly contribute data to the column.

For example, for the query:

create or replace table db.schema.users as
select
  user_name,
  count(distinct login_time) as total_logins
from db.schema.login_events
where user_type != 'test_user'

The direct dependency of total_logins is login_events.login_time. The column login_events.user_type filter the data of total_logins, but it is an indirect dependency and will not show in lineage.

If you want a different approach in your Elementary Cloud instance - contact us.