Flyte & Flytekit version
flyte-core v1.16.4
Describe the bug
By default, datacatalog is enabled (here) which works with the default settings for catalog-cache (here).
However, this two settings are miles apart especially the catalog-cache setting is deeply embedded inside configmaps. So if datacatalog is disabled which is not uncommon to reduce operational complexity, the default settings on catalog-cache will cause DNS failure due to datacatalog is not there. This is a gotcha moment because users may not know there is a default setting on configmaps that explicitly sets the cache entrypoint to use datacatalog.
Expected behavior
These two related configs should be linked together. If datacatalog is disabled, flyte should automatically disable the catalog-cache instead of using datacatalog service which doesn't exist.
Additional context to reproduce
Disable datacatalog without changing catalog-cache settings in configmaps setting.
Screenshots
Workflow[xxxx] failed.
RuntimeExecutionError: max number of system retry attempts [31/30] exhausted.
Last known status message: Failed to check Catalog for previous results:
DataCatalog failed to get dataset for ID
resource_type:TASK
project:"xxxx"
domain:"xxxx"
name:"xxxx"
version:"xxxx":
rpc error: code = Unavailable
desc = connection error: desc = "transport: Error while dialing: dial tcp: lookup datacatalog on x.x.x.x:x no such host"
Are you sure this issue hasn't been raised already?
Have you read the Code of Conduct?
Flyte & Flytekit version
flyte-core v1.16.4
Describe the bug
By default, datacatalog is enabled (here) which works with the default settings for catalog-cache (here).
However, this two settings are miles apart especially the catalog-cache setting is deeply embedded inside configmaps. So if datacatalog is disabled which is not uncommon to reduce operational complexity, the default settings on catalog-cache will cause DNS failure due to datacatalog is not there. This is a gotcha moment because users may not know there is a default setting on configmaps that explicitly sets the cache entrypoint to use datacatalog.
Expected behavior
These two related configs should be linked together. If datacatalog is disabled, flyte should automatically disable the catalog-cache instead of using datacatalog service which doesn't exist.
Additional context to reproduce
Disable datacatalog without changing catalog-cache settings in configmaps setting.
Screenshots
Workflow[xxxx] failed.
RuntimeExecutionError: max number of system retry attempts [31/30] exhausted.
Last known status message: Failed to check Catalog for previous results:
DataCatalog failed to get dataset for ID
resource_type:TASK
project:"xxxx"
domain:"xxxx"
name:"xxxx"
version:"xxxx":
rpc error: code = Unavailable
desc = connection error: desc = "transport: Error while dialing: dial tcp: lookup datacatalog on x.x.x.x:x no such host"
Are you sure this issue hasn't been raised already?
Have you read the Code of Conduct?