Skip to content

Make unit/integ tests more reliable#2353

Merged
normj merged 6 commits intofeature/response-streamingfrom
normj/test-stability
May 5, 2026
Merged

Make unit/integ tests more reliable#2353
normj merged 6 commits intofeature/response-streamingfrom
normj/test-stability

Conversation

@normj
Copy link
Copy Markdown
Member

@normj normj commented May 1, 2026

Description of changes:
Work on making the tests more reliable and PR checks succeed without retries. After the PR changes I was able to run the PR checks 5 times in a row with no issues.

Changes:

  • Removed the OnCompletedCallback_IsExecuted integ test. The test was relying on making a second invoke to Lambda see if a state variable was set from previous run. There is no way to guarantee the second invokes will go to the same Lambda compute instance. The scenario being tested was checking if the ASP.NET Core OnComplete callbacks are called. But the OnComplete callbacks don't have access to the response stream so there is no way to write something back to the caller to say I was called. Hence the reason the AI generated integ test relied on the second call. The fact that the test did pass when the invokes did go to same Lambda compute environment show the feature does work but we don't want to have this flaky test.
  • Renamed the ResponseStreamFactory collection to RuntimeSupportStateCheck and applied it to more tests that are testing RuntimeSupport so they would not run in parallel. RuntimeSupport has logic via statics that assume there is only a single instance running a single invocation. All these unit tests run under a few seconds so I'm not worried about tests not running in parallel
  • Updated tests creating LambdaBootstrap to pass in mock environment variable collections to avoid bleeding environment variable between tests.
  • Removed the MidstreamError_SetsErrorStateWithExceptionDetails unit test. There were too many issues mock of the runtime api to get the timing a 100% right. The scenario being tested is covered in integ tests.

I also removed the .NET 11 targets from RuntimeSupport and SnapShotRestore because we don't need it for OCI images and it was causing issues building in VS.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@normj normj requested a review from a team as a code owner May 1, 2026 23:39
@normj normj marked this pull request as draft May 1, 2026 23:39
@normj normj added the Release Not Needed Add this label if a PR does not need to be released. label May 1, 2026
@normj normj marked this pull request as ready for review May 5, 2026 16:45
@normj normj requested review from a team and philasmar May 5, 2026 16:47
@normj normj merged commit 42eb363 into feature/response-streaming May 5, 2026
10 checks passed
@normj normj deleted the normj/test-stability branch May 5, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Release Not Needed Add this label if a PR does not need to be released.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants