Skip to content

Commit 920873d

Browse files
authored
fix(backend): fix kubeconfig injection for slow-starting workspaces (CRW-11193) (#1620)
* fix(backend): start polling fallback when watch times out in PostStartInjector The 60-second watch timeout called cleanup() and silently abandoned injection without starting the polling fallback. Any workspace that takes more than 60 seconds to reach Running — which is common — never received kubeconfig or podman credentials. Fix: start startPollingFallback() from the timeout handler so injection can still complete via GET polling (10 s interval, up to 5 minutes). Fixes: https://redhat.atlassian.net/browse/CRW-11193 Assisted-by: Claude Sonnet 4.6 Signed-off-by: Oleksii Orel <oorel@redhat.com> * fix(backend): increase watch timeout to 120s and log polling elapsed time - Raise INJECTION_TIMEOUT_MS from 60s to 120s. The watch is a cheap long-lived HTTP connection, while the polling fallback fires a GET every 10 seconds. Keeping the watch alive longer reduces unnecessary polling for workspaces that start between 60s and 120s. - Log how long the workspace was in Starting phase when injection completes via polling (e.g. "workspace ready after 95s"), which helps operators diagnose slow-start clusters. - Add a comment in startPollingFallback() explaining why the same devworkspaceApi instance is safe to reuse across watch and polling (getByName is a standalone REST call, independent of watch state). Assists: https://redhat.atlassian.net/browse/CRW-11193 Assisted-by: Claude Sonnet 4.6 Signed-off-by: Oleksii Orel <oorel@redhat.com> * fix(backend): increase watch timeout to 300s to match startTimeout Align INJECTION_TIMEOUT_MS with the server-side startTimeout default (300 s). The watch is a cheap long-lived HTTP connection that should stay alive for the full startup window rather than falling back to polling at 120 s. Assists: https://redhat.atlassian.net/browse/CRW-11193 Assisted-by: Claude Sonnet 4.6 Signed-off-by: Oleksii Orel <oorel@redhat.com> --------- Signed-off-by: Oleksii Orel <oorel@redhat.com>
1 parent 38203bd commit 920873d

2 files changed

Lines changed: 44 additions & 6 deletions

File tree

packages/dashboard-backend/src/services/PostStartInjector.ts

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ import { IDevWorkspaceApi, IKubeConfigApi, IPodmanApi } from '@/devworkspaceClie
1616
import { MessageListener } from '@/services/types/Observer';
1717
import { logger } from '@/utils/logger';
1818

19-
const INJECTION_TIMEOUT_MS = 60000;
19+
const INJECTION_TIMEOUT_MS = 300000;
2020
const POLL_INTERVAL_MS = 10000;
2121
const POLL_TIMEOUT_MS = 300000;
2222

@@ -68,8 +68,15 @@ export class PostStartInjector {
6868
PostStartInjector.activeWatches.set(key, cleanup);
6969

7070
const timeoutHandle = setTimeout(() => {
71-
logger.warn(`PostStartInjector: timed out waiting for ${key} to reach Running`);
71+
logger.warn(`PostStartInjector: watch timed out for ${key}, starting polling fallback`);
7272
cleanup();
73+
PostStartInjector.startPollingFallback(
74+
namespace,
75+
workspaceName,
76+
devworkspaceApi,
77+
kubeConfigApi,
78+
podmanApi,
79+
);
7380
}, INJECTION_TIMEOUT_MS);
7481

7582
const cleanupWithTimeout = () => {
@@ -157,6 +164,9 @@ export class PostStartInjector {
157164
};
158165

159166
// Re-register so duplicate watchAndInject calls are still blocked during polling.
167+
// Note: the same devworkspaceApi instance is reused here. stopWatching() was already
168+
// called before startPollingFallback(), but getByName() is a standalone REST call
169+
// with no dependency on watch state, so the instance is safe to reuse.
160170
PostStartInjector.activeWatches.set(key, cleanup);
161171

162172
let elapsed = 0;
@@ -195,6 +205,7 @@ export class PostStartInjector {
195205
kubeConfigApi,
196206
podmanApi,
197207
key,
208+
elapsed,
198209
);
199210
} else {
200211
logger.info(`PostStartInjector: ${key} is in phase ${phase}, stopping poll`);
@@ -212,8 +223,13 @@ export class PostStartInjector {
212223
kubeConfigApi: IKubeConfigApi,
213224
podmanApi: IPodmanApi,
214225
key: string,
226+
elapsedMs?: number,
215227
): Promise<void> {
216-
logger.info(`PostStartInjector: ${key} is Running, injecting kubeconfig and podman login`);
228+
const elapsed =
229+
elapsedMs !== undefined ? ` (workspace ready after ${Math.round(elapsedMs / 1000)}s)` : '';
230+
logger.info(
231+
`PostStartInjector: ${key} is Running, injecting kubeconfig and podman login${elapsed}`,
232+
);
217233
try {
218234
await kubeConfigApi.injectKubeConfig(namespace, devworkspaceId);
219235
} catch (e) {

packages/dashboard-backend/src/services/__tests__/PostStartInjector.spec.ts

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -196,12 +196,34 @@ describe('PostStartInjector', () => {
196196

197197
// ── timeout ───────────────────────────────────────────────────────────────
198198

199-
test('cleans up on 60 s timeout', () => {
199+
test('stops watch and starts polling fallback on 300 s timeout', () => {
200+
(devworkspaceApi.getByName as jest.Mock).mockResolvedValue({
201+
status: { phase: 'Starting' },
202+
});
203+
200204
invoke();
201-
jest.advanceTimersByTime(60000);
205+
jest.advanceTimersByTime(300000);
202206

207+
// Watch stopped
203208
expect(devworkspaceApi.stopWatching).toHaveBeenCalled();
204-
expect((PostStartInjector as any).activeWatches.has(key)).toBe(false);
209+
// Polling fallback re-registers the key so injection can still complete
210+
expect((PostStartInjector as any).activeWatches.has(key)).toBe(true);
211+
});
212+
213+
test('injects credentials via polling fallback after 60 s watch timeout', async () => {
214+
(devworkspaceApi.getByName as jest.Mock).mockResolvedValue({
215+
status: { phase: 'Running', devworkspaceId: 'ws-timeout-id' },
216+
});
217+
218+
invoke();
219+
jest.advanceTimersByTime(300000);
220+
221+
// Advance polling interval so getByName is called
222+
jest.advanceTimersByTime(10000);
223+
await Promise.resolve().then(() => Promise.resolve());
224+
225+
expect(kubeConfigApi.injectKubeConfig).toHaveBeenCalledWith(namespace, 'ws-timeout-id');
226+
expect(podmanApi.podmanLogin).toHaveBeenCalledWith(namespace, 'ws-timeout-id');
205227
});
206228

207229
// ── polling fallback (triggered by ERROR event) ───────────────────────────

0 commit comments

Comments
 (0)