Skip to content

Commit 3ecf467

Browse files
committed
fix(integ): Use SAGEMAKER_REGION for cross-region training job lookup
The SageMaker SDK's SageMakerClient reads SAGEMAKER_REGION env var at init time and caches the region for all subsequent API calls. The cleanup_e2e_endpoints session fixture was the first to create a SageMakerClient (in the default region), which then poisoned all subsequent TrainingJob.get calls regardless of the region parameter. Fix by setting SAGEMAKER_REGION=us-west-2 in cleanup_e2e_endpoints before any SDK session is created, since all resources in this test file live in us-west-2. The env var is restored after cleanup. In CodeBuild (us-west-2) this is a no-op since the default region already matches. The other test files (triton, tei, tgi) are not affected since they have their own fixtures and don't import from this file.
1 parent 7e54110 commit 3ecf467

1 file changed

Lines changed: 18 additions & 5 deletions

File tree

sagemaker-serve/tests/integ/test_model_customization_deployment.py

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -52,12 +52,15 @@ def endpoint_name():
5252
def cleanup_e2e_endpoints():
5353
"""Cleanup e2e endpoints before and after tests."""
5454
import os
55-
from sagemaker.core.resources import Endpoint
5655
from botocore.exceptions import ClientError
5756

58-
# Ensure region is set before any SageMaker session is created
59-
if "AWS_DEFAULT_REGION" not in os.environ:
60-
os.environ["AWS_DEFAULT_REGION"] = "us-west-2"
57+
# This file's tests use us-west-2 resources. Set SAGEMAKER_REGION so the
58+
# SDK's SageMakerClient creates sessions in the correct region from the start.
59+
# Save/restore to avoid leaking into other test files.
60+
original_sm_region = os.environ.get("SAGEMAKER_REGION")
61+
os.environ["SAGEMAKER_REGION"] = "us-west-2"
62+
63+
from sagemaker.core.resources import Endpoint
6164

6265
# Cleanup before tests
6366
try:
@@ -83,6 +86,12 @@ def cleanup_e2e_endpoints():
8386
except (ClientError, Exception):
8487
pass
8588

89+
# Restore original SAGEMAKER_REGION
90+
if original_sm_region:
91+
os.environ["SAGEMAKER_REGION"] = original_sm_region
92+
elif "SAGEMAKER_REGION" in os.environ:
93+
del os.environ["SAGEMAKER_REGION"]
94+
8695

8796
@pytest.fixture(scope="module")
8897
def cleanup_endpoints():
@@ -534,7 +543,11 @@ def test_zzz_cleanup_deployed_model(self, bedrock_client):
534543

535544

536545
def test_model_customization_workflow(training_job_name):
537-
"""Standalone test function for pytest discovery."""
546+
"""Standalone test function for pytest discovery.
547+
548+
Relies on SAGEMAKER_REGION being set by the cleanup_e2e_endpoints
549+
session fixture (us-west-2).
550+
"""
538551
config = {
539552
"training_job_name": training_job_name,
540553
"region": "us-west-2",

0 commit comments

Comments
 (0)