Conversation
🤖 Augment PR SummarySummary: Adds a Changes:
🤖 Was this summary useful? React with 👍 or 👎 |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a build-time alternative for S3 snapshot storage via WITH_AWS_CLOUD, intending to use helio’s native AWS/S3 helpers instead of the AWS C++ SDK, and wires S3 initialization into the snapshot storage factory.
Changes:
- Extend S3 snapshot storage selection to allow
WITH_AWS_CLOUDbuilds. - Add
AwsS3SnapshotStorage::Init(), destructor, and aWITH_AWS_CLOUDimplementation ofListObjects(). - Update server linking/defines to include
WITH_AWS_CLOUDsupport.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/server/server_family.cc | Enables S3 snapshot storage creation for WITH_AWS_CLOUD and calls AwsS3SnapshotStorage::Init(). |
| src/server/detail/snapshot_storage.h | Adds helio AWS cloud includes and new members/API (Init, dtor, creds provider, SSL ctx). |
| src/server/detail/snapshot_storage.cc | Implements Init, dtor, and ListObjects for WITH_AWS_CLOUD; adjusts S3 read/write compilation guards. |
| src/server/CMakeLists.txt | Adds WITH_AWS_CLOUD define and links an additional AWS cloud library target. |
Comments suppressed due to low confidence (2)
src/server/detail/snapshot_storage.cc:618
- When building with WITH_AWS_CLOUD (and without WITH_AWS), this method always returns "AWS support not compiled in", so loading snapshots from s3:// will fail even though ListObjects is implemented. Add a WITH_AWS_CLOUD implementation (helio S3Storage-backed read file) or gate S3 snapshot storage on WITH_AWS until read support exists.
if (!bucket_path) {
return nonstd::make_unexpected(GenericError("Invalid S3 path"));
}
auto [bucket, key] = *bucket_path;
return new aws::S3ReadFile(bucket, key, s3_);
#else
return nonstd::make_unexpected(GenericError("AWS support not compiled in"));
#endif
}
io::Result<std::string, GenericError> AwsS3SnapshotStorage::LoadPath(std::string_view dir,
std::string_view dbfilename) {
if (dbfilename.empty())
src/server/detail/snapshot_storage.cc:536
- In WITH_AWS_CLOUD builds, the ec2_metadata and sign_payload constructor arguments are effectively ignored because the only code that uses them is under WITH_AWS. Either plumb these flags into AwsCredsProvider/S3Storage configuration (or env var handling) or remove the flags from this constructor in the WITH_AWS_CLOUD path to avoid silently changing behavior.
setenv("AWS_EC2_METADATA_DISABLED", "true", 0);
}
// S3ClientConfiguration may request configuration and credentials from
// EC2 metadata so must be run in a proactor thread.
Aws::S3::S3ClientConfiguration s3_conf;
s3_conf.checksumConfig.responseChecksumValidation =
Aws::Client::ResponseChecksumValidation::WHEN_REQUIRED;
Collaborator
Author
|
augment review |
bc9877e to
c3867a8
Compare
BorysTheDev
reviewed
Apr 3, 2026
BorysTheDev
reviewed
Apr 3, 2026
BorysTheDev
reviewed
Apr 3, 2026
BorysTheDev
previously approved these changes
Apr 3, 2026
Add alternative S3 implementation using helio's native S3Storage and AwsCredsProvider when WITH_AWS_CLOUD is ON, removing the dependency on the AWS C++ SDK. Changes: - Add AwsS3SnapshotStorage::Init() to initialize credentials and SSL context (matching GCS/Azure pattern) - Implement ListObjects using cloud::aws::S3Storage::List - Add destructor to free SSL context - Wire up Init call in CreateCloudSnapshotStorage - Update CMakeLists.txt to link aws_cloud_lib - Update helio submodule with S3Storage virtual-hosted-style URLs and SigV4 signing fixes This PR substitutes ListObjects implementation ONLY if WITH_AWS is disabled and WITH_AWS_CLOUD enabled. Signed-off-by: Roman Gershman <roman@dragonflydb.io>
BorysTheDev
approved these changes
Apr 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
S3StorageandAwsCredsProviderwhenWITH_AWS_CLOUD=ON, removing the dependency on the AWS C++ SDKAwsS3SnapshotStorage::Init(),ListObjects, and destructor following the GCS/Azure pattern🤖 Generated with Claude Code