Skip to content

Fixes #27041: Clean up profiler time series data on hard delete of Table #27044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RajdeepKushwaha5 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Fixes #27041: Clean up profiler time series data on hard delete of Table #27044

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
65f3757
fix: Clean up profiler time series data on hard delete of Table
RajdeepKushwaha5 Apr 4, 2026
2bc5384
Use prefix-based DELETE for column profile cleanup on hard delete
RajdeepKushwaha5 Apr 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/EntityTimeSeriesDAO.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import org.openmetadata.schema.utils.JsonUtils;
import org.openmetadata.service.jdbi3.locator.ConnectionAwareSqlQuery;
import org.openmetadata.service.jdbi3.locator.ConnectionAwareSqlUpdate;
import org.openmetadata.service.util.FullyQualifiedName;
import org.openmetadata.service.util.RestUtil;
import org.openmetadata.service.util.jdbi.BindFQN;

Expand Down Expand Up @@ -530,6 +531,18 @@ default void delete(String entityFQNHash, String extension) {
delete(getTimeSeriesTableName(), entityFQNHash, extension);
}

@SqlUpdate(
"DELETE FROM <table> WHERE entityFQNHash LIKE :fqnHashPrefix AND extension = :extension")
void deleteByFQNPrefix(
@Define("table") String table,
@Bind("fqnHashPrefix") String fqnHashPrefix,
@Bind("extension") String extension);

default void deleteByFQNPrefix(String entityFQN, String extension) {
String hashPrefix = FullyQualifiedName.buildHash(entityFQN) + ".%";
deleteByFQNPrefix(getTimeSeriesTableName(), hashPrefix, extension);
}

@SqlUpdate(
"DELETE FROM <table> WHERE entityFQNHash = :entityFQNHash AND extension = :extension AND timestamp = :timestamp")
void deleteAtTimestamp(
Expand Down
10 changes: 10 additions & 0 deletions openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TableRepository.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1154,6 +1154,16 @@ public void deleteTableProfile(String fqn, String entityType, Long timestamp) {
daoCollection.profilerDataTimeSeriesDao().deleteAtTimestamp(fqn, extension, timestamp);
}

@Override
protected void entitySpecificCleanup(Table table) {
String fqn = table.getFullyQualifiedName();
daoCollection.profilerDataTimeSeriesDao().delete(fqn, TABLE_PROFILE_EXTENSION);
daoCollection.profilerDataTimeSeriesDao().delete(fqn, SYSTEM_PROFILE_EXTENSION);
Comment on lines +1157 to +1161
Copy link

Copilot AI Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description/checklist says a test was added for this scenario, but there are no test changes in this PR branch/repo state that exercise the new hard-delete cleanup for profiler time series. Please either add an explicit regression test for hard-deleting a table and asserting profiler_data_time_series is cleaned, or update the PR description to point to the existing test that covers this behavior.

Copilot uses AI. Check for mistakes.
daoCollection
.profilerDataTimeSeriesDao()
.deleteByFQNPrefix(fqn, TABLE_COLUMN_PROFILE_EXTENSION);
}

public ResultList<TableProfile> getTableProfiles(String fqn, Long startTs, Long endTs) {
List<EntityProfile> entityProfiles;
List<TableProfile> tableProfiles;
Expand Down
Loading