Merge branch 'main' into promptless/serverless-pricing-worker-states-edits

lavanya-gunreddi · web-flow · commit 53ed2b25efc9 · 2026-07-02T18:26:46.000-04:00
diff --git a/docs.json b/docs.json
@@ -354,7 +354,7 @@
               "tutorials/pods/run-your-first",
               "tutorials/pods/comfyui",
               "tutorials/pods/run-ollama",
-              "tutorials/pods/build-docker-images"
+              "tutorials/pods/build-docker-images"              
             ]
           },
           {
diff --git a/release-notes.mdx b/release-notes.mdx
diff --git a/serverless/endpoints/endpoint-configurations.mdx b/serverless/endpoints/endpoint-configurations.mdx
@@ -77,6 +77,16 @@ Number of GPUs assigned to each worker instance. Default is 1. Generally priorit
 
 How long a worker stays active after completing a request before shutting down. You're billed during idle time, but the worker remains warm for immediate processing. Default: 5 seconds.
 
+### Idle endpoint scale-down
+
+Runpod automatically scales down endpoints that go a long time without any requests, so unused endpoints don't keep consuming your account balance. 
+* After 3 days with no requests, the endpoint's max workers is reduced to 2 and Runpod sends you an email notification. 
+* After 7 days with no requests, max workers is set to 0. 
+
+This scale-down is automatic and system-driven, and the timer is based on request activity, so any incoming request resets it.
+
+Once an endpoint has been scaled down this way, it stays at its reduced max workers until you raise the value yourself. To use the endpoint again, increase its max workers in the Runpod console. To prevent an endpoint from scaling down in the first place, make sure it continues to receive requests.
+
 ### Execution timeout
 
 Maximum duration for a single job. When exceeded, the job fails and the worker stops. Keep enabled to prevent runaway jobs. Default: 600s (10 min). Range: 5s to 7 days.
diff --git a/serverless/endpoints/overview.mdx b/serverless/endpoints/overview.mdx
@@ -44,15 +44,18 @@ Before creating an endpoint, ensure you have a [handler function](/serverless/wo
 <Tab title="Web">
 
 1. Navigate to the [Serverless section](https://www.console.runpod.io/serverless) and click **New Endpoint**.
-2. Choose your deployment source:
-   - **Import Git Repository**: See [Deploy from GitHub](/serverless/workers/github-integration)
-   - **Import from Docker Registry**: See [Deploy from Docker Hub](/serverless/workers/deploy)
-   - **Ready-to-Deploy Repos**: Select a preconfigured endpoint
-3. Configure your endpoint:
-   - **Endpoint Name** and **Type** (Queue-based or Load balancer)
-   - **GPU Configuration** and worker settings
+2. Choose your deployment path:
+   - **Hello World**: Runpod forks a starter worker template into a new GitHub repo in your account. Choose Queue-based or Load balancing, then click **Deploy**.
+   - **Hugging Face LLM**: Search for any text-generation model on Hugging Face (for example, type "Gemma" to find Gemma 4), select it, and click **Create Endpoint**. Runpod deploys a vLLM endpoint for you.
+   - **Docker**: Deploy from a container image. Select a saved Serverless template to fill in the container configuration automatically, or skip the template and enter an image name manually. See [Deploy from Docker](/serverless/workers/deploy).
+   - **GitHub**: Select a repository, filtering by code owner if needed. Runpod checks for a Dockerfile and runs a background check on your handler: queue-based endpoints check for handler files, and load balancing endpoints check for a `/ping` path. See [Deploy from GitHub](/serverless/workers/github-integration).
+   - **Hub**: Opens the Hub browser, where you can browse and deploy prebuilt workers. This replaces the previous "Ready-to-Deploy Repos" option. See [Hub overview](/hub/overview).
+   - **Flash**: A guided setup flow for [Flash](/flash/overview) that walks you through installing the SDK, initializing your project, and sending your first command. Steps complete automatically as you progress.
+3. For the GitHub, Docker, and Hello World paths, configure your endpoint before deploying:
+   - **Endpoint name** and **type** ([Queue-based](/flash/create-endpoints#queue-based-endpoints) or [Load balancing](/flash/create-endpoints#load-balanced-endpoints))
+   - **GPU** configuration and worker scaling
    - **Model** (optional): Enter a Hugging Face URL for [cached models](/serverless/endpoints/model-caching)
-   - **Environment Variables**: See [environment variables](/serverless/development/environment-variables)
+   - **Environment variables** and container configuration. See [environment variables](/serverless/development/environment-variables).
 4. Click **Deploy Endpoint**.
 
 </Tab>
diff --git a/serverless/troubleshooting.mdx b/serverless/troubleshooting.mdx
@@ -51,6 +51,17 @@ Check the job status response for error details. Common causes:
 - **OOM (Out of Memory)**: Model or batch size exceeds GPU memory. Reduce batch size or use a larger GPU.
 - **Timeout**: Job exceeded execution timeout. Increase timeout or optimize processing.
 
+## Endpoint scaling issues
+
+### My endpoint was scaled down unexpectedly
+
+If your endpoint's max workers dropped without any change on your end, Runpod scaled the endpoint down automatically. This happens in two situations:
+
+- **Prolonged inactivity**: When an endpoint receives no requests for 3 days, its max workers is reduced to 2, and after 7 days its max workers is set to 0. Runpod emails you when the first reduction happens. For more details, see [idle endpoint scale-down](/serverless/endpoints/endpoint-configurations#idle-endpoint-scale-down).
+- **Repeated unhealthy workers**: When an endpoint consistently produces unhealthy (crashing) workers, Runpod scales it down to stop billing and reduce thrashing, and sends you an email.
+
+To bring the endpoint back, increase its max workers in the [Runpod console](https://www.console.runpod.io/serverless). If the scale-down was caused by unhealthy workers, fix the underlying problem first, or the endpoint may be scaled down again. Check the [logs](/serverless/development/logs) for crash errors, and verify your worker using [local testing](/serverless/development/local-testing).
+
 ## Cold start issues
 
 ### Slow cold starts
diff --git a/serverless/workers/deploy.mdx b/serverless/workers/deploy.mdx
@@ -76,6 +76,8 @@ Versioning best practices:
 * Document the specific image version or SHA in your deployment documentation.
 * Keep images as small as possible for faster startup times.
 
+Runpod validates your image name when you create an endpoint, and a reference that doesn't resolve to a published image is rejected at creation time. For example, `runpod/pytorch:latest` is rejected because `:latest` is not a published tag for Runpod's base images. When you deploy a Runpod base image, specify a tag that actually exists. You can browse the available tags for each image on [Docker Hub](https://hub.docker.com/u/runpod). For your own worker images, use a specific version or SHA tag as described above rather than `:latest`.
+
 ## Deploy an endpoint
 
 <Tip>
diff --git a/serverless/workers/overview.mdx b/serverless/workers/overview.mdx
@@ -56,6 +56,10 @@ The system may also spin up **extra workers** during traffic spikes when Docker
 | **Outdated** | Marked for replacement after update | Yes (while processing) |
 | **Unhealthy** | Crashed; auto-retries for up to 7 days | No |
 
+<Note>
+If an endpoint repeatedly produces unhealthy workers, Runpod automatically scales it down; see [My endpoint was scaled down unexpectedly](/serverless/troubleshooting#my-endpoint-was-scaled-down-unexpectedly). 
+</Note>
+
 View worker states in the **Workers** tab of your endpoint in the [Runpod console](https://www.console.runpod.io/serverless).
 
 ## Max worker limits
diff --git a/tutorials/pods/use-private-ecr-images.mdx b/tutorials/pods/use-private-ecr-images.mdx
@@ -0,0 +1,213 @@
+---
+title: "Use private AWS ECR images"
+sidebarTitle: "Private ECR images"
+description: "Pull container images from private AWS ECR repositories using cross-account delegation."
+tag: "BETA"
+hidden: true
+---
+
+<Note>
+ECR image support is currently in beta. [Join our Discord](https://discord.gg/cUpRmau42V) to provide feedback and get support.
+</Note>
+
+import { PodTooltip } from "/snippets/tooltips.jsx";
+
+This tutorial shows how to deploy <PodTooltip />s using container images stored in private AWS Elastic Container Registry (ECR) repositories. You configure cross-account IAM delegation so Runpod can pull images on your behalf — no credential sharing required, and no expiring tokens to manage.
+
+<Note>
+This tutorial covers ECR integration for Pods. If you're deploying Serverless workers from ECR, see [Deploying from AWS ECR](/serverless/workers/deploy#deploying-from-aws-ecr) — Serverless endpoints use a different credential mechanism.
+</Note>
+
+## What you'll learn
+
+- How to build and push a Docker image to a private ECR repository.
+- How to configure an AWS ECR repository policy for cross-account access.
+- How to add an ECR credential in the Runpod console.
+- How to deploy a <PodTooltip /> using your private ECR image.
+
+## Requirements
+
+- A Runpod account with credits.
+- An AWS account with ECR enabled.
+- An AWS IAM user or role with `ecr:SetRepositoryPolicy` permission on the repository.
+- Your AWS Account ID — a 12-digit number found in the top-right dropdown of the AWS Console.
+- Docker installed on your local machine.
+- AWS CLI installed (required to push images; optional if your image is already in ECR).
+
+## Step 1: Push your image to ECR
+
+Skip this step if your image is already in ECR.
+
+### Configure AWS CLI credentials
+
+Run the following command and enter your access key, secret key, and session token when prompted:
+
+```bash
+aws configure
+```
+
+You can find these credentials under **IAM → Security credentials** in the AWS Console.
+
+### Create a private ECR repository
+
+1. Open the [Amazon ECR console](https://console.aws.amazon.com/ecr/).
+2. Under **Private registry**, select **Repositories** and click **Create repository**.
+3. Enter a name and click **Create repository**.
+4. Note the **URI** shown for your new repository — you'll use it in the following steps.
+
+### Build, tag, and push your image
+
+Authenticate Docker with ECR, then build and push your image:
+
+```bash
+# Authenticate Docker to your ECR registry
+aws ecr get-login-password --region YOUR_REGION | docker login \
+  --username AWS \
+  --password-stdin YOUR_ACCOUNT_ID.dkr.ecr.YOUR_REGION.amazonaws.com
+
+# Build your image
+docker build -t YOUR_REPOSITORY_NAME .
+
+# Tag it with your ECR URI
+docker tag YOUR_REPOSITORY_NAME:latest \
+  YOUR_ACCOUNT_ID.dkr.ecr.YOUR_REGION.amazonaws.com/YOUR_REPOSITORY_NAME:latest
+
+# Push it
+docker push \
+  YOUR_ACCOUNT_ID.dkr.ecr.YOUR_REGION.amazonaws.com/YOUR_REPOSITORY_NAME:latest
+```
+
+Replace `YOUR_ACCOUNT_ID`, `YOUR_REGION`, and `YOUR_REPOSITORY_NAME` with your values.
+
+<Warning>
+Avoid using the `latest` tag in production. Runpod warns against it during template creation because `latest` tags are considered unstable. Use a specific version tag for production deployments.
+</Warning>
+
+## Step 2: Configure your ECR repository policy
+
+To pull images from your private ECR repository, Runpod needs cross-account access. You grant this by adding an IAM policy to your repository.
+
+1. Open the [Amazon ECR console](https://console.aws.amazon.com/ecr/).
+2. Select the repository containing your container image.
+3. In the left navigation, select **Permissions**.
+4. Click **Edit policy JSON** and add the following policy statement:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Sid": "AllowRunpodPull",
+      "Effect": "Allow",
+      "Principal": "*",
+      "Action": [
+        "ecr:GetAuthorizationToken",
+        "ecr:BatchCheckLayerAvailability",
+        "ecr:GetDownloadUrlForLayer",
+        "ecr:BatchGetImage"
+      ],
+      "Condition": {
+        "StringEquals": {
+          "aws:PrincipalArn": "arn:aws:iam::550005742258:role/prod-us-east-1-deployment-role"
+        }
+      }
+    }
+  ]
+}
+```
+
+5. Click **Save**.
+
+<Note>
+The `aws:PrincipalArn` value (`arn:aws:iam::550005742258:role/prod-us-east-1-deployment-role`) is Runpod's fixed AWS deployment role — not something you create or manage. `550005742258` is Runpod's AWS account ID, and this ARN is the same for all Runpod users. Do not modify it. Its permissions are intentionally limited to pulling images only, so Runpod cannot access any other part of your AWS account.
+</Note>
+
+<Note>
+`ecr:GetAuthorizationToken` is an account-level AWS action, not repository-scoped. If your organization uses Service Control Policies (SCPs), confirm this action is not blocked at the org level.
+</Note>
+
+### Alternative: Configure via AWS CLI
+
+```bash
+aws ecr set-repository-policy \
+    --repository-name YOUR_REPOSITORY_NAME \
+    --policy-text '{
+      "Version": "2012-10-17",
+      "Statement": [
+        {
+          "Sid": "AllowRunpodPull",
+          "Effect": "Allow",
+          "Principal": "*",
+          "Action": [
+            "ecr:GetAuthorizationToken",
+            "ecr:BatchCheckLayerAvailability",
+            "ecr:GetDownloadUrlForLayer",
+            "ecr:BatchGetImage"
+          ],
+          "Condition": {
+            "StringEquals": {
+              "aws:PrincipalArn": "arn:aws:iam::550005742258:role/prod-us-east-1-deployment-role"
+            }
+          }
+        }
+      ]
+    }'
+```
+
+Replace `YOUR_REPOSITORY_NAME` with the name of your ECR repository.
+
+## Step 3: Add your ECR credential to Runpod
+
+1. Navigate to [Settings](https://www.runpod.io/console/user/settings) in the Runpod console.
+2. Scroll down to **Container Registry Authentication** and click **Add Credential**.
+3. Select **AWS ECR** as the registry type.
+4. Enter a **Name** for this credential (for example, `my-ecr-repo`).
+5. Enter the **ECR Image URI** in the format `ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com/REPOSITORY_NAME:TAG`. You must include the tag (for example, `:latest` or `:v1.0.0`). Runpod requires the tag to store the delegation.
+6. Click **Create**.
+
+<Warning>
+Credential creation fails if the ECR repository policy from Step 2 is not correctly configured. Runpod validates the delegation at registration time — if the policy is missing, the credential is rejected. Set up the policy before proceeding.
+</Warning>
+
+## Step 4: Deploy a <PodTooltip /> with your private image
+
+You can deploy using a template or directly from the <PodTooltip /> deploy page.
+
+### Option A: Deploy directly
+
+1. Navigate to [Pods](https://www.runpod.io/console/pods) and select **Deploy**.
+2. Choose your GPU configuration.
+3. Under **Container Image**, enter your full ECR image URI (for example, `123456789012.dkr.ecr.us-east-2.amazonaws.com/my-app:latest`).
+4. Configure any additional settings such as environment variables or exposed ports.
+5. Click **Deploy**.
+
+### Option B: Deploy via a template
+
+1. Navigate to [Templates](https://www.runpod.io/console/user/templates).
+2. Create a new template or update an existing one.
+3. Set the **Container Image** to your ECR image URI.
+4. Save the template, then deploy a <PodTooltip /> from it.
+
+Runpod uses the registered credential to authenticate and pull your private image. To confirm the pull succeeded, open your <PodTooltip />'s logs — you should see layer download entries from your ECR registry.
+
+<Check>
+You've configured cross-account ECR delegation and deployed a <PodTooltip /> using a private container image.
+</Check>
+
+## Troubleshooting
+
+**Credential creation fails:**
+- Confirm the repository policy JSON is saved: ECR console → your repository → **Permissions** tab.
+- Check that the `aws:PrincipalArn` value matches exactly — no trailing spaces or modified characters.
+- Verify your IAM user has `ecr:SetRepositoryPolicy` permission.
+
+**<PodTooltip /> fails to start with an image pull error:**
+- Confirm the ECR image URI in the <PodTooltip /> or template matches the one registered in Step 3.
+- Check that the image tag exists in your ECR repository.
+- Verify the repository policy is applied to the correct repository.
+
+## Next steps
+
+- Learn how to [create custom templates](/pods/templates/create-custom-template) from your container images.
+- Explore [environment variables](/pods/templates/environment-variables) for configuring your containers.
+- Set up [network volumes](/storage/network-volumes) for persistent storage.
diff --git a/tutorials/serverless/generate-sdxl-turbo.mdx b/tutorials/serverless/generate-sdxl-turbo.mdx
@@ -248,8 +248,10 @@ async function generateImage() {
         if (data && data.output && data.output.images && data.output.images.length > 0) {
             const imageBase64 = data.output.images[0].image;
             const imageUrl = `data:image/png;base64,${imageBase64}`;
-            document.getElementById("imageResult").innerHTML =
-                `<img src="${imageUrl}" alt="Generated Image" />`;
+            const img = document.createElement("img");
+            img.src = imageUrl;
+            img.alt = "Generated Image";
+            document.getElementById("imageResult").replaceChildren(img);
         } else if (data && data.error) {
             alert(`Error: ${data.error}`);
         } else {

Original file line number	Diff line number	Diff line change
`@@ -354,7 +354,7 @@`
`354`	`354`	`"tutorials/pods/run-your-first",`
`355`	`355`	`"tutorials/pods/comfyui",`
`356`	`356`	`"tutorials/pods/run-ollama",`
`357`		`- "tutorials/pods/build-docker-images"`
	`357`	`+ "tutorials/pods/build-docker-images"`
`358`	`358`	`]`
`359`	`359`	`},`
`360`	`360`	`{`