Skip to content

refactor(test): allow linux integ test to run in github action#8386

Merged
roger-zhangg merged 60 commits into
aws:developfrom
roger-zhangg:develop
Nov 4, 2025
Merged

refactor(test): allow linux integ test to run in github action#8386
roger-zhangg merged 60 commits into
aws:developfrom
roger-zhangg:develop

Conversation

@roger-zhangg

@roger-zhangg roger-zhangg commented Oct 31, 2025

Copy link
Copy Markdown
Member

Which issue(s) does this change fix?

Integ Test can not be run in Github Action (GA) before due to multiple reason.

  • Few tests (integration/buildcmd especially) takes too much disk size, GA runner only has 18GB of free space by default.
  • No good way to sync credential to GA
  • GA uses other GA to setup test environments

Why is this change necessary?

Provide faster and more stable testing experience

  • Uses OIDC to assume test accounts (Valid up to 6 hours)
  • Breaks down test even more by separating test that actually doesn't require docker. So for integration/buildcmd there are 3 category: docker/finch/no-container. This avoids test that doesn't require container to be run in container tests and provided better parallelization and efficiency at same time.
  • Break down integration/buildcmd test by language so each container test runner only need to download 2-3 types of sam builder image to save space.
  • Update few outdated test to use newer runtime
    • Go 1.18 -> 1.25
    • Node 18 -> Node 22
    • Ruby 3.2 -> Ruby 3.4
  • Fixed Few issues introduced by

How does it address the issue?

Test run: https://github.com/roger-zhangg/aws-sam-cli/actions/runs/18965218688
Newer test run: https://github.com/roger-zhangg/aws-sam-cli/actions/runs/19054600514
Takes only ~30 minute to finish all tests

Job Run time  
build-integ-arm64-java (docker) 19m 40s  
build-integ (docker) 12m 46s  
sync (docker) 25m 4s  
build-integ-java-python-provided (docker) 12m 33s  
other-and-e2e (docker) 23m 48s  
package-delete-deploy (docker) 29m 5s  
build-integ-dotnet-node-ruby (docker) 5m 0s  
local-start2 (docker) 11m 15s  
local-start (docker) 20m 47s  
build-integ-arm64 (docker) 10m 56s  
terraform-build (docker) 16m 48s  
local-invoke (docker) 31m 56s  
build-integ-arm64 (finch) 15m 23s  
build-integ-arm64-java (finch) 20m 39s  
terraform-build (finch) 15m 51s  
package-delete-deploy (finch) 29m 58s  
sync (finch) 24m 59s  
local-start (finch) 22m 41s  
build-integ-dotnet-node-ruby (finch) 5m 37s  
local-invoke (finch) 28m 42s  
build-integ (finch) 16m 14s  
build-integ (no-container) 10m 54s  
build-integ-java-python-provided (no-container) 17m 51s  
local-start2 (finch) 14m 23s  
build-integ-java-python-provided (finch) 10m 47s  
build-integ-arm64-java (no-container) 2m 30s  
build-integ-arm64 (no-container) 9m 1s  
build-integ-dotnet-node-ruby (no-container) 6m 12s  
other-and-e2e (finch) 25m 47s  
  8h 17m 7s  

What side effects does this change have?

  • 1-2 test being flaky in github test
  • We changed the default test account role assume logic. Before the process will delete and re-create the test account role before assuming (this need to wait for 90 second to make sure we are assuming to the new role as the role is using the same name). Now we skipped the role re-creation process so we can finish assume role process in 10 second. But this could have unknown side effect, if that happens, we should revert back to the default behavior by change: tests/get_testing_resources.py (by removing payload_data = json.dumps({"skip_role_deletion": True}))

Mandatory Checklist

PRs will only be reviewed after checklist is complete

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@roger-zhangg
roger-zhangg requested a review from a team as a code owner October 31, 2025 17:06
vicheey
vicheey previously approved these changes Nov 3, 2025
Comment thread tests/integration/buildcmd/build_integ_base.py Outdated
Comment thread tests/integration/buildcmd/build_integ_base.py Outdated
Comment thread tests/integration/buildcmd/build_integ_base.py Outdated
Comment thread tests/integration/buildcmd/test_build_cmd.py Outdated
@seshubaws

seshubaws commented Nov 4, 2025

Copy link
Copy Markdown
Contributor

In PR description says "Integ Test can not be run in github action due to multiple reason."

Could you expand on the reasons for posterity? Otherwise, LGTM

@roger-zhangg
roger-zhangg added this pull request to the merge queue Nov 4, 2025
Merged via the queue into aws:develop with commit 73695ce Nov 4, 2025
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants