Use-cases
We initially created cross account role for the workspace credential config and the s3 storage access role for the workspace storage config and did not make these roles region specific. We now think it makes sense to create both of these roles per region. This means we need new roles because the roles need new names which means new ARNs. New roles have been done that easily with CDK (new roles with existing roles also still in place). But when terraform gets the new ARN, the provider doesn't know about the API call to simply update the role in place on the existing credential/storage config. Because it's not "update aware", it's behavior is to delete and replace the entire credential/storage config. And that means the workspace itself has a property change. And again, the provider isn't smart enough to just know how to change that one property in an existing object. So it's plan is to delete the entire workspace and recreate it.
Attempted Solutions
We do a Databricks API update call on the credential and storage config to change the role ARN. Then, we have to do a "terraform rm" to remove that object from the terraform state, and then do a "terraform import" to bring it back in from Databricks with the new ARN. Not a good workaround because this also involves extra process because it will be done outside the pipeline process. We will need a manual change ticket documentation to get it done whereas if the terraform provider could do it, we could just implement it via a standard deploy.
Proposal
Have update capabilities on the terraform provider
References
Use-cases
We initially created cross account role for the workspace credential config and the s3 storage access role for the workspace storage config and did not make these roles region specific. We now think it makes sense to create both of these roles per region. This means we need new roles because the roles need new names which means new ARNs. New roles have been done that easily with CDK (new roles with existing roles also still in place). But when terraform gets the new ARN, the provider doesn't know about the API call to simply update the role in place on the existing credential/storage config. Because it's not "update aware", it's behavior is to delete and replace the entire credential/storage config. And that means the workspace itself has a property change. And again, the provider isn't smart enough to just know how to change that one property in an existing object. So it's plan is to delete the entire workspace and recreate it.
Attempted Solutions
We do a Databricks API update call on the credential and storage config to change the role ARN. Then, we have to do a "terraform rm" to remove that object from the terraform state, and then do a "terraform import" to bring it back in from Databricks with the new ARN. Not a good workaround because this also involves extra process because it will be done outside the pipeline process. We will need a manual change ticket documentation to get it done whereas if the terraform provider could do it, we could just implement it via a standard deploy.
Proposal
Have update capabilities on the terraform provider
References