[Bug]: Encrypted variables set in the UI are silently changed when other config is applied through OpenTofu

[JoeHutchins0n]

What happened?

Use case

We are managing pipelines and projects through OpenTofu, but don't want to manage encrypted variables there as it's not practical for our use case. The only thing feasible for us is to manage them in the Codefresh UI.

Expected behaviour

An encrypted variable is added or updated in a pipeline or project in the Codefresh UI. Because we don't want to manage encrypted variables in code, the lifecycle.ignore_changes property is set for spec[0].encrypted_variables. Any updates to other configuration then applied through OpenTofu does not change the value of that encrypted variable. Or if there is a change then it at least stages something in the plan.

Actual behaviour

When any configuration update for the pipeline or project is applied through OpenTofu, the values of encrypted variables that were set in the UI are changed to an empty string. This change is completely silent and isn't shown in the plan. This behaviour has broken important pipelines for us.

Test to reproduce

I created a pipeline variable called TEST_VAR, set the value to "foo" and encrypt the variable.

I have a simple pipeline step that checks if TEST_VAR is equal to "foo".

version: "1.0"
stages:
  - prepare

steps:
  test_var:
    title: Test encrypted variable
    image: alpine:latest
    type: freestyle
    stage: prepare
    commands:
      - if [[ "${TEST_VAR}" = "foo" ]]; then echo "Encrypted variable matches"; else echo "Encrypted variable does not match" && exit 1; fi

I run the pipeline and it passes, confirming that TEST_VAR is equal to "foo". No surprises there.

In my attached OpenTofu code, I update spec[0].concurrency from 1 to 2. I run a plan (snippet below), then apply it.

OpenTofu will perform the following actions:

  # codefresh_pipeline.cd["cicd-playground"] will be updated in-place
  ~ resource "codefresh_pipeline" "cd" {
        id         = "69a5a71add33bdcd4a61e4b0"
        name       = "cicd-playground/cicd-playground-cd"
        tags       = []
        # (3 unchanged attributes hidden)

      ~ spec {
          ~ concurrency                                           = 1 -> 2
            # (8 unchanged attributes hidden)

            # (3 unchanged blocks hidden)
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Then when I run the pipeline again, the step fails as the value of the encrypted variable is no longer "foo".

Some investigation I did afterwards showed that the value was reset to an empty string.

I would expect the same behaviour with Terraform as well, although I can't confirm that.

Version

1.2.0

Relevant Terraform Configuration

resource "codefresh_pipeline" "cd" {
  name     = "cd-pipeline"

  spec {
    concurrency         = 1
    branch_concurrency  = 1
    trigger_concurrency = 1
    spec_template {
      repo     = try("myuser/myrepo")
      path     = "./codefresh/cd.yml"
      revision = ""
      context  = "bitbucket"
    }

    trigger {
      branch_regex = "/main/gi"
      context      = "bitbucket"
      name         = "CD"
      description  = "Trigger for main branch"
      events       = ["push.heads"]
      repo         = "myuser/myrepo"
      provider     = "bitbucket"
      type         = "git"
    }

    runtime_environment {
      name   = "otrium-euc-runtime/codefresh"
      cpu    = "1"
      memory = "1024Mi"
    }
  }

  lifecycle {
    ignore_changes = [spec[0].encrypted_variables]
  }
}