Paginate S3 prefix deletes

[tomassrnka]

DeleteObjectsWithPrefix only deleted the first ListObjectsV2 page (1000 keys max), so prefixes with more than 1000 objects left the rest behind. Now it pages through all of them. Also refuses an empty prefix so a bad call can't wipe the whole bucket.

[cursor]

PR Summary

Medium Risk
Changes bulk S3 deletion behavior; the empty-prefix guard reduces catastrophic-delete risk, but any bug in pagination or batching could still leave objects behind or delete the wrong scope.

Overview
DeleteObjectsWithPrefix used a single ListObjectsV2 call, so only the first page of keys (up to 1000) was removed under large prefixes. It now walks every page with ListObjectsV2Paginator, deletes each batch via a new deleteObjects helper, and applies awsOperationTimeout per list/delete round-trip instead of one timeout over the whole walk. An empty prefix returns an error so a bad call cannot target the whole bucket. A unit test covers the empty-prefix rejection.

^{Reviewed by Cursor Bugbot for commit 7624012. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
3177	2	3175	6

View the top 2 failed test(s) by shortest run time

github.com/e2b-dev/infra/tests/integration/internal/tests/api/metrics::TestSandboxMetrics

Stack Traces | 14.7s run time

=== RUN   TestSandboxMetrics
=== PAUSE TestSandboxMetrics
=== CONT  TestSandboxMetrics
    sandbox_metrics_test.go:45: 
        	Error Trace:	.../api/metrics/sandbox_metrics_test.go:45
        	Error:      	Should NOT be empty, but was 0
        	Test:       	TestSandboxMetrics
--- FAIL: TestSandboxMetrics (14.72s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/templates::TestTemplateBuildCache

Stack Traces | 163s run time

=== RUN   TestTemplateBuildCache
=== PAUSE TestTemplateBuildCache
=== CONT  TestTemplateBuildCache
    build_template_test.go:133: test-ubuntu-cache: [info] Building template 2wxpn19647it1renypqo/51df51f7-96db-4924-b843-2d337ca556cd
    build_template_test.go:133: test-ubuntu-cache: [info] [base] FROM ubuntu:22.04 [f9f564014e009a9561a82bf8c84f9314242971e833fb019936654ecba452f184]
    build_template_test.go:133: test-ubuntu-cache: [info] Base Docker image size: 30 MB
    build_template_test.go:133: test-ubuntu-cache: [info] Creating file system and pulling Docker image
    build_template_test.go:133: test-ubuntu-cache: [info] Uncompressing layer sha256:d6834b4a794c03efa2c998853e64969fa8851b11b2ade63292268872a37759d0 30 MB
    build_template_test.go:133: test-ubuntu-cache: [info] Uncompressing layer sha256:8f87ff6b97fa2ea201417a33f0f89a604504a594a950602d6f98c298e9a8fe72 13 MB
    build_template_test.go:133: test-ubuntu-cache: [info] Uncompressing layer sha256:8c4b1b28875140ed3abacaf16ad0d696f6bef912f52d2148f261a23e3349465b 168 B
    build_template_test.go:133: test-ubuntu-cache: [info] Layers extracted
    build_template_test.go:133: test-ubuntu-cache: [info] Root filesystem structure: bin, boot, dev, etc, home, lib, lib32, lib64, libx32, media, mnt, opt, proc, root, run, sbin, srv, sys, tmp, usr, var
    build_template_test.go:133: test-ubuntu-cache: [info] Provisioning sandbox template
    build_template_test.go:133: test-ubuntu-cache: [info] Provisioning was successful, cleaning up
    build_template_test.go:133: test-ubuntu-cache: [info] Sandbox template provisioned
    build_template_test.go:133: test-ubuntu-cache: [info] [base] DEFAULT USER user [49e586c2171254c6bc4a09e84eedac32dbcf113a158c24248129af2f49cbed74]
    build_template_test.go:133: test-ubuntu-cache: [info] [builder 1/2] ENV ENV_VAR Hello, World! [ad43bf60323a0a7d9f49eadd0718478e4d275aa2c02c7eada236adf0c0efdfa6]
    build_template_test.go:133: test-ubuntu-cache: [info] [builder 2/2] RUN : "${ENV_VAR:?ENV_VAR is not set or empty}"; echo "$ENV_VAR" [c84aac682e6480ff86381017502b99e452b0ddf9728976d44ef327c3219270b7]
    build_template_test.go:133: test-ubuntu-cache: [info] [builder 2/2] [stdout]: Hello, World!
    build_template_test.go:133: test-ubuntu-cache: [info] [finalize] Finalizing template build [731c9a392e13ebcab75a37dec5ecdfa1768e0e0ec8a3080ff14028a9803da196]
    build_template_test.go:133: test-ubuntu-cache: [error] Build failed: An internal error occurred. Please try again or contact support with the build ID.
    build_template_test.go:451: Build failed: {<nil> An internal error occurred. Please try again or contact support with the build ID. <nil>}
--- FAIL: TestTemplateBuildCache (162.58s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[gemini-code-assist]

Code Review

This pull request refactors DeleteObjectsWithPrefix in the AWS storage implementation to use S3 pagination, preventing timeouts on large prefixes, and adds a safeguard to reject empty prefixes. A corresponding unit test was also added. The review feedback suggests two improvements: using aws.String for pointer conversion to maintain consistency with the AWS SDK, and simplifying the loop by removing the anonymous function used for scoping the context timeout.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[claude]

LGTM - correct use of the S3 paginator with per-page timeouts, plus a sensible empty-prefix guard.

Extended reasoning...

Overview

Small bug fix in packages/shared/pkg/storage/storage_aws.go (~40 LOC changed). DeleteObjectsWithPrefix previously invoked ListObjectsV2 once and deleted only the first page (max 1000 keys), silently leaving the rest. It now walks all pages via s3.NewListObjectsV2Paginator and delegates each page's DeleteObjects call to a new deleteObjects helper. An explicit guard rejects an empty prefix. A test asserts the empty-prefix guard.

Security risks

The empty-prefix guard is a defensive improvement — a bad caller can no longer accidentally enumerate/delete the entire bucket. No auth, crypto, or permission surface is touched. Nothing user-controllable reaches the S3 keys here.

Level of scrutiny

Low. This is a straightforward fix in a shared storage helper with an obvious real bug (1000-key ceiling) and a well-contained mitigation. Pagination is implemented with the SDK's own paginator (standard pattern), and each round-trip carries the same 5s timeout previously used for a single call. The deleted flag correctly preserves the pre-existing warn-on-empty behavior.

Other factors

• Per-page context.WithTimeout inside an IIFE-with-defer cancel() is the right shape to avoid leaking cancels across many pages.
• Delete errors abort the walk mid-way, which is consistent with the previous behavior and appropriate for this call site.
• Test coverage for the paginator path itself is not added, but that requires mocking the S3 client and the added risk is low.

[tomassrnka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e6a25b4. Configure here.}