Fix race condition in idp-cli generate-manifest --test-set (#193)#239
Merged
Fix race condition in idp-cli generate-manifest --test-set (#193)#239
Conversation
Contributor
Author
|
Can you review / verify this fix, @kaleko ? Many tx. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #193 — Race condition in
idp-cli generate-manifest --test-setthat causes test sets to be marked as FAILED with "Missing baseline files" errors when the UI is open during CLI uploads.Root Cause
When the CLI uploads input files then baselines sequentially to S3, the test set resolver's auto-detection (triggered by UI polling via
getTestSets) can validate the test set before all baseline files are uploaded. This creates a premature FAILED DynamoDB entry. The CLI's subsequent resolver call then skips re-validation because the entry already exists.Solution
Implemented a
.uploadingS3 marker file protocol:{test_set}/.uploadingin the test set bucket_is_valid_test_set_structure()checks for the marker viahead_objectand skips folders that have itChanges
nested/appsync/src/lambda/test_set_resolver/index.py.uploadingmarker check in_is_valid_test_set_structure()lib/idp_cli_pkg/idp_cli/cli.pygenerate_manifest()and_create_test_set_from_manifest()CHANGELOG.md[Unreleased]Backward Compatibility