Ability to fill dynamic values & secrets

[michaelhelmick]

Not sure if this should be one in the same issue, but; I wanted to deploy task definitions to two environments changing the CPU value. Generally this is static, but could change; I also have a lot of application settings that could change. I'm unsure of the ability of actions/core but it would be nice to be able to have syntax that could provide this functionality. Secondly, it might fall in the same category, but a list of secrets we have (using AWS Secret Manager) being inputed dynamically into the task def (as those could change too). Hopefully that use case was easy to follow. I'd love to help develop this if it's possible and looking to discuss how it could be accomplished.

[clareliguori]

Do you have an example of how you would want to provide the task def setting and value you want to update?

I think part of the challenge today is that GitHub Actions inputs are fairly inflexible. In GH Actions, the input keys are typically fixed in the action.yml, and then the input value must be a string. For example, you can't provide a map of task def keys you want to update mapped to your desired value as an input. There are some workarounds here, but neither of them seem particular user-friendly.
https://github.community/t5/GitHub-Actions/Can-action-inputs-be-arrays/td-p/33776

Just brainstorming here, there are a couple other tools that might be useful for accomplishing what you're looking to do. Something like envsubst can evaluate environment variables in a file and replace them with the current env variable value. So for example, in your task def:

"secrets": [
                {
                    "name": "username", 
                    "valueFrom": "$USERNAME_SECRET_ARN"
                }
            ], 

There are a few envsubst GitHub Actions in the marketplace, though I haven't tried any of them.
https://github.com/marketplace?type=actions&query=envsubst

You could also merge multiple files together to enable having stage-specific settings. For example, have a task-def-base.json, task-def-dev.json, task-def-prod.json. task-def-base.json can have most of your settings that are common across stages, with task-def-prod.json just containing a CPU setting override.

{
    "containerDefinitions": [
        {
            "name": "my-container", 
            "cpu": 1024, 
        }
     ]
}

A tool like yq can merge files together:
https://mikefarah.gitbook.io/yq/commands/merge
yq is mostly for yaml, but it should be able to merge json as well. It also has a GitHub Action:
https://github.com/mikefarah/yq/blob/master/action.yml

Please let me know if either of these work for you!

[michaelhelmick]

Hm, I can't get around to trying the suggestions for a bit, but they look usable.

Maybe providing a json file in the with: for key other_env_vars or something and this action could loop over each key value and update the task-definition.json that is provided?

ci.yml

name: Render Amazon ECS task definition
      id: render-web-container
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: task-definition.json
        container-name: web
        other-vars: other-vars.json
        image: amazon/amazon-ecs-sample:latest

other-vars.json

{
    "memoryReservation": 256,
    "cpu": 1024,
    "environment: [
        "LOG_LEVEL": "info",
        "OTHER_VAR": "abc123"
    ],
    "secrets": [
        "FACEBOOK_API_KEY": "arn:aws:secretsmanager:us-east-1:account_id:secret:project/FACEBOOK_API_KEY-abcDEF"}",
        "STRIPE_API_KEY": "arn:aws:secretsmanager:us-east-1:account_id:secret:project/STRIPE_API_KEY-defGHI"}"
    ]
}

After this action ran, task def would look like:

[michaelhelmick]

Okay, so as I wrote that, I kind of see that we'd prob want the task def to be separate for each, i.e.:

taskdefs
|_staging.json
|_production.json
|_other.json

and in the action, use

with:
    task-definition: ./taskdefs/{{ env.ENVIRONMENT }}.json

or something similar?

Would prob be easier than envsubst everywhere. Would you agree with the above method?

[clareliguori]

Ah, I see so similar to how yq merge works, the render-task-definition action could merge in a task def "fragment"?

So I think you could do:

task-def.json
staging-vars.json
prod-vars.json

And then:

    - name: Add image to Amazon ECS task definition
      id: render-image-in-task-def
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: task-def.json
        container-name: web
        image: amazon/amazon-ecs-sample:latest

    - name: Render Amazon ECS task definition for staging
      id: render-staging-task-def
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: ${{ steps.render-image-in-task-def.outputs.task-definition }}
        merge: staging-vars.json

    - name: Render Amazon ECS task definition for prod
      id: render-prod-task-def
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: ${{ steps.render-image-in-task-def.outputs.task-definition }}
        merge: prod-vars.json

    - name: Deploy to Staging
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        task-definition: ${{ steps.render-staging-task-def.outputs.task-definition }}
        service: my-staging-service
        cluster: my-staging-cluster

    - name: Deploy to Prod
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        task-definition: ${{ steps.render-prod-task-def.outputs.task-definition }}
        service: my-prod-service
        cluster: my-prod-cluster

[robbie1douglas]

Similarly, I want to pass account numbers in ARNs in as secrets to the task-definition.json to keep them out of the repo.

[adamhathcock]

Has any work been done this? Being able to merge in any value into the task definition or having environment variables exposed as a first class input would be great.

Thanks!

[jestjosh]

Bump because passing in account id, arns as secrets seems like a needed feature.