Skip to content

Commit 841488b

Browse files
committed
refactor(serverless): address PR review feedback for boto3 lazy-loading
Address Copilot review comments from PR #466: 1. Extract boto3 import logic into shared helper function - Created _import_boto3_dependencies() to reduce duplication - Used by both get_boto_client() and bucket_upload() - Consistent error handling across all S3 upload functions 2. Add TransferConfig import to bucket_upload() - Now imports all boto3 dependencies via shared helper - Maintains consistency with get_boto_client() 3. Clarify documentation about fallback directories - Documented that upload_image() uses simulated_uploaded/ - Documented that public API functions use local_upload/ - Added context about when fallback behavior occurs All 358 tests pass with 97% coverage.
1 parent 73dc607 commit 841488b

2 files changed

Lines changed: 23 additions & 9 deletions

File tree

docs/serverless/utils/rp_upload.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@ The upload utility provides functions to upload files and in-memory objects to a
88

99
The upload utility requires [boto3](https://pypi.org/project/boto3/) for S3 functionality. boto3 is lazy-loaded to minimize initial import time and memory footprint.
1010

11-
If you attempt to use S3 upload features without boto3 installed, you'll receive a warning and files will be saved to local disk instead (`simulated_uploaded/` or `local_upload/` directories).
11+
If you attempt to use S3 upload features without boto3 installed or S3 credentials are not configured, files will be saved to local disk instead:
12+
- `upload_image()` saves to `simulated_uploaded/` directory
13+
- `upload_file_to_bucket()` and `upload_in_memory_object()` save to `local_upload/` directory
1214

1315
## Bucket Credentials
1416

runpod/serverless/utils/rp_upload.py

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,23 @@
2020
logging.basicConfig(level=logging.INFO, format=FMT, handlers=[logging.StreamHandler()])
2121

2222

23+
def _import_boto3_dependencies():
24+
"""
25+
Lazy-load boto3 dependencies.
26+
Returns tuple of (session, TransferConfig, Config) or raises ImportError.
27+
"""
28+
try:
29+
from boto3 import session
30+
from boto3.s3.transfer import TransferConfig
31+
from botocore.config import Config
32+
return session, TransferConfig, Config
33+
except ImportError as e:
34+
raise ImportError(
35+
"boto3 is required for S3 upload functionality. "
36+
"Install with: pip install boto3"
37+
) from e
38+
39+
2340
def extract_region_from_url(endpoint_url):
2441
"""
2542
Extracts the region from the endpoint URL.
@@ -45,9 +62,7 @@ def get_boto_client(
4562
Lazy-loads boto3 to reduce initial import time.
4663
"""
4764
try:
48-
from boto3 import session
49-
from boto3.s3.transfer import TransferConfig
50-
from botocore.config import Config
65+
session, TransferConfig, Config = _import_boto3_dependencies()
5166
except ImportError:
5267
logger.warning(
5368
"boto3 not installed. S3 upload functionality disabled. "
@@ -187,16 +202,13 @@ def bucket_upload(job_id, file_list, bucket_creds): # pragma: no cover
187202
Uploads files to bucket storage.
188203
"""
189204
try:
190-
from boto3 import session
191-
from botocore.config import Config
205+
session, _, Config = _import_boto3_dependencies()
192206
except ImportError:
193207
logger.error(
194208
"boto3 not installed. Cannot upload to S3 bucket. "
195209
"Install with: pip install boto3"
196210
)
197-
raise ImportError(
198-
"boto3 is required for bucket_upload. Install with: pip install boto3"
199-
)
211+
raise
200212

201213
temp_bucket_session = session.Session()
202214

0 commit comments

Comments
 (0)