Retry RTX PRO 6000 GGUF deploy

Author: cderinbogaz

experiments/002-kimi-k26-gguf-q2/2026-04-25-runpod-rtxpro6000x4-q2-attempts.md

@@ -66,9 +66,31 @@ Observed behavior:
 - `uptimeSeconds` remained `0`
 - No benchmark was possible because the runtime never became reachable
 
+## Attempt 3
+
+| Field | Value |
+| --- | --- |
+| Pod name | `kimi-k26-gguf-q2-rtxpro6000x4-20260425-3` |
+| Pod ID | `2qlu7nwua4ndd8` |
+| Cost | `$6.76/hr` |
+| Requested datacenter | `CA-MTL-3` |
+| Machine ID | `9ti6j8484pn1` |
+
+Observed behavior:
+
+- Pod allocated again onto the same machine ID as attempts 1 and 2
+- RunPod later exposed SSH metadata: `107.150.186.62:13262`
+- Direct SSH attempts still returned `Connection refused`
+- `uptimeSeconds` remained `0`
+- No benchmark was possible because the runtime never became reachable
+
+## Launcher Fix
+
+During the third retry, the pod launcher was corrected to pass runtime overrides such as `HF_MODEL`, `CONTEXT_LENGTH`, `SPLIT_MODE`, and `TENSOR_SPLIT` into the pod environment. That means later GGUF retries now accurately reflect the requested serving topology instead of silently falling back to startup-script defaults.
+
 ## Interpretation
 
-This is another provider readiness failure. The second attempt is especially useful because it shows that even after RunPod exposed SSH metadata, the host still was not accepting TCP connections.
+This is another provider readiness failure. The second and third attempts are especially useful because they show that even after RunPod exposed SSH metadata, the host still was not accepting TCP connections.
 
 That means:
 

experiments/002-kimi-k26-gguf-q2/README.md

@@ -62,7 +62,7 @@ This failure mode points to RunPod host readiness, not model fit. The deployment
 
 ## 2026-04-25 4x RTX PRO 6000 Attempts
 
-Two `4x RTX PRO 6000 Blackwell Server Edition` attempts were made on 2026-04-25 for the same `UD-Q2_K_XL` GGUF using llama.cpp CUDA with explicit multi-GPU sharding enabled.
+Three `4x RTX PRO 6000 Blackwell Server Edition` attempts were made on 2026-04-25 for the same `UD-Q2_K_XL` GGUF using llama.cpp CUDA with explicit multi-GPU sharding enabled.
 
 Configuration:
 
@@ -83,10 +83,12 @@ Outcome:
 
 - First pod: `eleak5xoojla2a`
 - Second pod: `qm93vevzo0cz1j`
-- Both landed on the same machine family: `9ti6j8484pn1`
+- Third pod: `2qlu7nwua4ndd8`
+- All three landed on the same machine family: `9ti6j8484pn1`
 - First attempt never exposed reachable SSH
 - Second attempt later exposed SSH metadata (`107.150.186.62:13340`) but direct SSH still returned `Connection refused`
-- In both cases `uptimeSeconds` remained `0`, so the runtime never transitioned into a usable state
+- Third attempt later exposed SSH metadata (`107.150.186.62:13262`) but direct SSH still returned `Connection refused`
+- In all three cases `uptimeSeconds` remained `0`, so the runtime never transitioned into a usable state
 
 These attempts confirm that the current RunPod `4x RTX PRO 6000` allocator is also returning a non-ready host for this workflow.
 

scripts/runpod/create_kimi_k26_gguf_pod_rest.sh

@@ -72,6 +72,25 @@ if [[ -n "${HF_TOKEN:-}" ]]; then
   pod_env="$(jq --arg token "$HF_TOKEN" '. + {HF_TOKEN: $token}' <<<"$pod_env")"
 fi
 
+for key in \
+  HF_MODEL \
+  HOST \
+  PORT \
+  CONTEXT_LENGTH \
+  GPU_LAYERS \
+  PARALLEL \
+  SPLIT_MODE \
+  TENSOR_SPLIT \
+  LLAMA_CPP_DIR \
+  CUDA_ARCHITECTURES \
+  BUILD_THREADS \
+  ROCR_VISIBLE_DEVICES
+do
+  if [[ -n "${!key:-}" ]]; then
+    pod_env="$(jq --arg key "$key" --arg value "${!key}" '. + {($key): $value}' <<<"$pod_env")"
+  fi
+done
+
 payload="$(
   jq -n \
     --arg name "$NAME" \