Skip to content

Test failure: test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-hiveserde-tables-in-place-experimental] #4869

Description

@github-actions
❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-hiveserde-tables-in-place-experimental]: AssertionError: Workflow failed: migrate-external-hiveserde-tables-in-place-experimental (32m46.182s)
AssertionError: Workflow failed: migrate-external-hiveserde-tables-in-place-experimental
assert False
[gw5] linux -- Python 3.10.20 /home/runner/work/ucx/ucx/.venv/bin/python
06:51 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QyFb/config.yml) doesn't exist.
06:51 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
06:51 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
06:51 INFO [databricks.labs.ucx.install] Fetching installations...
06:51 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QyFb is corrupted. Skipping...
06:51 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
06:51 DEBUG [tests.integration.conftest] Waiting for clusters to start...
06:51 DEBUG [tests.integration.conftest] Waiting for clusters to start...
06:51 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+1220260611065109
06:51 INFO [databricks.labs.ucx.install] Creating ucx schemas...
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
06:51 INFO [databricks.labs.ucx.install] Creating dashboards...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QyFb/README for the next steps.
06:51 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
06:51 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/36050286855936
06:51 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
06:51 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/36050286855936/runs/388309970359796
06:51 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/36050286855936
06:51 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
07:03 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/369969558482730
07:03 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-hiveserde-tables-in-place-experimental job: None
07:03 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/369969558482730/runs/1119871642891889
07:03 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-hiveserde-tables-in-place-experimental workflow: https://DATABRICKS_HOST#job/369969558482730
07:03 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
06:51 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QyFb/config.yml) doesn't exist.
06:51 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
06:51 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
06:51 INFO [databricks.labs.ucx.install] Fetching installations...
06:51 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QyFb is corrupted. Skipping...
06:51 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
06:51 DEBUG [tests.integration.conftest] Waiting for clusters to start...
06:51 DEBUG [tests.integration.conftest] Waiting for clusters to start...
06:51 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+1220260611065109
06:51 INFO [databricks.labs.ucx.install] Creating ucx schemas...
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
06:51 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
06:51 INFO [databricks.labs.ucx.install] Creating dashboards...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
06:51 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
06:51 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
06:51 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QyFb/README for the next steps.
06:51 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
06:51 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/36050286855936
06:51 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
06:51 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/36050286855936/runs/388309970359796
06:51 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/36050286855936
06:51 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
07:03 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/369969558482730
07:03 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-hiveserde-tables-in-place-experimental job: None
07:03 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/369969558482730/runs/1119871642891889
07:03 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-hiveserde-tables-in-place-experimental workflow: https://DATABRICKS_HOST#job/369969558482730
07:03 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
07:23 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+1220260611065109 from https://DATABRICKS_HOST
07:23 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_sn2ucfwau
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=369969558482730, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=827314849086826, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=123034326128393, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=807103600142146, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=960121597900129, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=36050286855936, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=862781544698086, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=757689642455633, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1111290517152681, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=877694647090687, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1088709169503734, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1117601445617525, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=977952218234378, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=809122290480576, as it is no longer needed
07:23 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=35409213662259, as it is no longer needed
07:23 INFO [databricks.labs.ucx.install] Deleting cluster policy
07:23 INFO [databricks.labs.ucx.install] Deleting secret scope
07:23 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw5] linux -- Python 3.10.20 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #879

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions