Skip to content

Commit aeefa5b

Browse files
yan-3005claude
andauthored
fix(migration): v200 thread_entity guard, PostgreSQL jsonb cast, self-loop cycle detection (#27756)
* fix(migration): v200 thread_entity guard, PostgreSQL jsonb cast, self-loop cycle detection - Add tableExists guard to migrateThreadTasksToTaskEntity so it skips safely when thread_entity has been renamed to thread_entity_legacy by postDataMigration SQL, preventing crashes on fresh installs and migration re-runs - Fix PostgreSQL jsonb type mismatch in migrateSuggestionsToTaskEntity and migrateLegacyActivityThreadsToActivityStream: JDBI binds String as varchar which PostgreSQL rejects for jsonb columns; use ::jsonb cast when ConnectionType is POSTGRES - Re-enable migrateThreadTasksToTaskEntity for MySQL (was commented out as workaround) now that the tableExists guard makes it safe - Fix hasCycleDFS in WorkflowDefinitionRepository to skip self-loop edges so BPMN reassignment cycles (AssignedStage -> AssignedStage) are not flagged as cycles, allowing IncidentLifecycleWorkflow to seed without error - Fix initSeedDataFromResources in EntityRepository to continue seeding remaining entities when one entity fails, instead of aborting the entire seed operation - Add unit tests covering MySQL/PostgreSQL variants for all three migration methods, tableExists guard, and jsonb cast behavior (13 tests total, all passing) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(migration): address review feedback on tableExists and jsonb cast tests - Fix tableExists to use findFirst() instead of one(): one() throws on an empty result set (0 rows from an existing but empty table), causing the guard to incorrectly return false and log "table does not exist" - Replace vacuous jsonb cast tests with direct insertTask reflection tests: the previous tests mocked an empty thread list so insertTask was never reached, making assertions trivially true regardless of cast logic; now call insertTask directly via reflection so the ::jsonb cast path is actually exercised for POSTGRES and absent for MYSQL - Update tableExists mock stubs in tests from .one() to .findFirst() to match the corrected implementation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix v200 migration: insert entity_relationship rows for migrated tasks Thread tasks and suggestions migrated to task_entity were missing entity_relationship rows for assignees, createdBy, and about — causing the repository to return empty assignees and no createdBy in the API. Added insertTaskLinkRelationships helper called after each task INSERT in both migrateThreadTasksToTaskEntity and migrateSuggestionsToTaskEntity. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix v200 migration: UPSERT entity_relationship rows and handle thread_entity_legacy - Replace INSERT IGNORE/ON CONFLICT DO NOTHING with UPSERT so stale entity_relationship rows written by 1.12.x FeedRepository (toEntity='thread') get corrected to toEntity='task' on PK conflict instead of being silently skipped - Fall back to thread_entity_legacy when thread_entity is absent so subsequent server starts (after post-DDL rename) still process thread tasks - For tasks already in task_entity, still call insertTaskLinkRelationships instead of skipping entirely, ensuring relationship rows are backfilled idempotently Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Fix v200 migration: null-safe createdBy lookup in alreadyExists backfill Use threadJson.path("createdBy").asText("system") to avoid treating a JSON null value as the literal string "null" which would cause lookupUserId to fail and skip the CREATED entity_relationship row. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent a1e7086 commit aeefa5b

6 files changed

Lines changed: 436 additions & 38 deletions

File tree

openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/EntityRepository.java

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1228,7 +1228,16 @@ protected void setDefaultStatus(T entity, boolean update) {
12281228
public final void initSeedDataFromResources() throws IOException {
12291229
List<T> entities = getEntitiesFromSeedData();
12301230
for (T entity : entities) {
1231-
initializeEntity(entity);
1231+
try {
1232+
initializeEntity(entity);
1233+
} catch (Exception e) {
1234+
LOG.warn(
1235+
"Failed to initialize {} '{}': {}",
1236+
entityType,
1237+
entity.getFullyQualifiedName(),
1238+
e.getMessage(),
1239+
e);
1240+
}
12321241
}
12331242
}
12341243

openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/WorkflowDefinitionRepository.java

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -347,6 +347,9 @@ private boolean hasCycleDFS(
347347
List<String> neighbors = adjacencyList.get(node);
348348
if (neighbors != null) {
349349
for (String neighbor : neighbors) {
350+
if (neighbor.equals(node)) {
351+
continue;
352+
}
350353
if (hasCycleDFS(neighbor, adjacencyList, visited, recursionStack)) {
351354
return true;
352355
}

openmetadata-service/src/main/java/org/openmetadata/service/migration/mysql/v200/Migration.java

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
package org.openmetadata.service.migration.mysql.v200;
22

3+
import static org.openmetadata.service.jdbi3.locator.ConnectionType.MYSQL;
34
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.addTableColumnSearchSettings;
45
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.backfillAnnouncementRelationships;
56
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.migrateLegacyActivityThreadsToActivityStream;
67
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.migrateSuggestionsToTaskEntity;
8+
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.migrateThreadTasksToTaskEntity;
79

810
import lombok.SneakyThrows;
911
import org.openmetadata.service.migration.api.MigrationProcessImpl;
@@ -19,11 +21,9 @@ public Migration(MigrationFile migrationFile) {
1921
@SneakyThrows
2022
public void runDataMigration() {
2123
addTableColumnSearchSettings();
22-
migrateSuggestionsToTaskEntity(handle);
23-
// Causing issues with collate CI, needs to be fixed before enabling this migration
24-
// @harshach
25-
// migrateThreadTasksToTaskEntity(handle);
26-
migrateLegacyActivityThreadsToActivityStream(handle);
24+
migrateSuggestionsToTaskEntity(handle, MYSQL);
25+
migrateThreadTasksToTaskEntity(handle, MYSQL);
26+
migrateLegacyActivityThreadsToActivityStream(handle, MYSQL);
2727
backfillAnnouncementRelationships(handle);
2828
}
2929
}

openmetadata-service/src/main/java/org/openmetadata/service/migration/postgres/v200/Migration.java

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
package org.openmetadata.service.migration.postgres.v200;
22

3+
import static org.openmetadata.service.jdbi3.locator.ConnectionType.POSTGRES;
34
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.addTableColumnSearchSettings;
45
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.backfillAnnouncementRelationships;
56
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.migrateLegacyActivityThreadsToActivityStream;
67
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.migrateSuggestionsToTaskEntity;
8+
import static org.openmetadata.service.migration.utils.v200.MigrationUtil.migrateThreadTasksToTaskEntity;
79

810
import lombok.SneakyThrows;
911
import org.openmetadata.service.migration.api.MigrationProcessImpl;
@@ -19,11 +21,9 @@ public Migration(MigrationFile migrationFile) {
1921
@SneakyThrows
2022
public void runDataMigration() {
2123
addTableColumnSearchSettings();
22-
migrateSuggestionsToTaskEntity(handle);
23-
// Causing issues with collate CI, needs to be fixed before enabling this migration
24-
// @harshach
25-
// migrateThreadTasksToTaskEntity(handle);
26-
migrateLegacyActivityThreadsToActivityStream(handle);
24+
migrateSuggestionsToTaskEntity(handle, POSTGRES);
25+
migrateThreadTasksToTaskEntity(handle, POSTGRES);
26+
migrateLegacyActivityThreadsToActivityStream(handle, POSTGRES);
2727
backfillAnnouncementRelationships(handle);
2828
}
2929
}

0 commit comments

Comments
 (0)