Skip to content

Use a new S3 bucket instead of mciuploads#900

Merged
autarch merged 1 commit into
masterfrom
02-27-use_a_new_s3_bucket_instead_of_mciuploads_
Mar 2, 2026
Merged

Use a new S3 bucket instead of mciuploads#900
autarch merged 1 commit into
masterfrom
02-27-use_a_new_s3_bucket_instead_of_mciuploads_

Conversation

@autarch
Copy link
Copy Markdown
Collaborator

@autarch autarch commented Feb 27, 2026

This is being done for two reasons.

First, we want to move projects away from all using the shared mciuploads bucket.

Second, the credentials to access mciuploads are only available for the DB Tools Evergreen project for the master branch. When we make a PR off a different branch, we cannot see those credentials. That means those branches cannot interact with S3 in CI, which leads to lots of CI failures. Switching to a new bucket and using role_arn to access that bucket means this works on any branch.

This also changes all our S3 uploads to be private, but I don't think there was any reason for these to be publicly readable. People outside MongoDB can't see Evergreen logs, so they'd have no way to find the relevant URLs anyway.

Copy link
Copy Markdown
Collaborator Author

autarch commented Feb 27, 2026

This stack of pull requests is managed by Graphite. Learn more about stacking.

@autarch autarch force-pushed the 02-27-use_a_new_s3_bucket_instead_of_mciuploads_ branch 3 times, most recently from 86040a7 to 86ca04c Compare February 27, 2026 19:27
@autarch autarch marked this pull request as ready for review February 27, 2026 19:57
@autarch autarch requested a review from a team as a code owner February 27, 2026 19:57
@autarch autarch requested review from tdq45gj and removed request for a team February 27, 2026 19:57
bucket: mciuploads
permissions: public-read
bucket: evergreen-project-mongo-tools-i6qg5nn6nbm
permissions: private
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the uploaded file be available for download from Evergreen UI?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that the default visibility is "public" according to the doc, so it should be good.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I needed to add visibility: signed to make this work. The links showed up in Evergreen, but they resulted in permissions error without setting visibility.

Copy link
Copy Markdown
Contributor

@tdq45gj tdq45gj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

bucket: mciuploads
permissions: public-read
bucket: evergreen-project-mongo-tools-i6qg5nn6nbm
permissions: private
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that the default visibility is "public" according to the doc, so it should be good.

@autarch autarch force-pushed the 02-27-use_a_new_s3_bucket_instead_of_mciuploads_ branch 2 times, most recently from 2f6832c to b61ba1d Compare March 2, 2026 17:41
Copy link
Copy Markdown
Collaborator Author

autarch commented Mar 2, 2026

Merge activity

  • Mar 2, 8:38 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Mar 2, 8:39 PM UTC: Graphite rebased this pull request as part of a merge.
  • Mar 2, 8:40 PM UTC: @autarch merged this pull request with Graphite.

@autarch autarch changed the base branch from 02-25-tools-4102_add_a_mise.toml_with_all_of_our_tools to graphite-base/900 March 2, 2026 20:38
@autarch autarch changed the base branch from graphite-base/900 to master March 2, 2026 20:38
This is being done for two reasons.

First, we want to move projects away from all using the shared `mciuploads` bucket.

Second, the credentials to access `mciuploads` are only available for the DB Tools Evergreen project for the `master` branch. When we make a PR off a different branch, we cannot see those credentials. That means those branches cannot interact with S3 in CI, which leads to lots of CI failures. Switching to a new bucket and using `ec2.assume_role` to access that bucket means this works on any branch.
@autarch autarch force-pushed the 02-27-use_a_new_s3_bucket_instead_of_mciuploads_ branch from b61ba1d to d4e8fa0 Compare March 2, 2026 20:39
@autarch autarch merged commit 5745e79 into master Mar 2, 2026
2 of 3 checks passed
autarch added a commit that referenced this pull request Mar 2, 2026
This reverts commit 5745e79.

I realized that this may break the release process. It's not clear to me that using this new bucket will work for that case. We don't have any good way to test the release process, so I'll just revert this for now.
autarch added a commit that referenced this pull request Mar 3, 2026
This reverts commit 5745e79.

I realized that this may break the release process. It's not clear to me that using this new bucket will work for that case. We don't have any good way to test the release process, so I'll just revert this for now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants