Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .changeset/me-permissions-retry-transient.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
"@object-ui/permissions": minor
"@object-ui/console": patch
---

fix(permissions,console): `MePermissionsProvider` retries a transient `/me/permissions` failure instead of stranding the app on its loading state

"Not now" is a real answer from this endpoint. On a multi-tenant host it is served
by the environment kernel that owns the session, and a COLD one answers `503` +
`Retry-After` while it warms (objectstack#4159 / cloud#927). The provider treated
that like any other failure: it set `error` — and a consumer that passes no
`errorFallback` renders `loadingFallback` for the error state too. The console
does exactly that (`loadingFallback={<LoadingScreen />}`, no `errorFallback`), so
the app sat on its spinner indefinitely, with a `retry` nobody could reach.

The fetch now re-attempts a **transient** failure — `408`, `425`, `429`, `502`,
`503`, `504`, or a thrown fetch (offline / DNS / aborted), which never got an
answer at all. A server-stated `Retry-After` wins over the exponential backoff
(both wire forms are read, and clamped to 30s so a hostile value cannot park the
UI); otherwise the delay doubles from `retryBaseDelayMs`. `loading` stays true
across the waits, so the fail-closed loading state holds and consumers never see
a permissive flash mid-recovery.

Unchanged for a real answer about the caller: `401`, `403`, `404` and `500` fail
on the first attempt exactly as before. `500` is deliberately not retried — a
genuine server fault neither benefits from hammering nor should be hidden behind
a spinner.

**New props**, both optional and defaulted so no call site needs to change:

- `maxRetries` (default `3`) — `0` restores the previous single-attempt
behaviour.
- `retryBaseDelayMs` (default `500`) — base for the exponential backoff.

Also fixes a latent race the retries made much wider: the in-flight fetch is now
cancelled when the effect tears down, so a slow answer for a previous `endpoint`
or `fetcher` can no longer overwrite a fast answer for the current one. The retry
primitives (`parseRetryAfterMs`, `backoffMs`, `isTransientFailure`,
`TRANSIENT_STATUS`, `PermissionsFetchError`) live in a new internal `./retry`
module — not exported from the package root.

**The console now passes an `errorFallback`.** Retrying narrows the window but
cannot close it — a kernel build slower than the retry budget still lands in the
error state, and rendering `loadingFallback` there is what produced the eternal
spinner. It now renders `<LoadingScreen error={...} onRetry={retry} />`, using the
error + retry affordance that component has carried all along, so a user is never
left with a spinner and no way forward.
13 changes: 12 additions & 1 deletion apps/console/src/AppContent.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,18 @@ export function AppContent() {
// rendering fell open (restricted fields rendered editable).
const authFetch = useMemo(() => createAuthenticatedFetch(), []);
return (
<MePermissionsProvider endpoint={endpoint} fetcher={authFetch} loadingFallback={<LoadingScreen />}>
<MePermissionsProvider
endpoint={endpoint}
fetcher={authFetch}
loadingFallback={<LoadingScreen />}
// Without this, the provider's error state renders `loadingFallback` too —
// an eternal spinner with a `retry` nobody can reach. The provider already
// re-attempts a transient failure (a cold environment kernel answers 503 +
// `Retry-After` while it warms, objectstack#4159); this is what the user
// sees once those are exhausted, and `LoadingScreen` has carried the
// error + retry affordance all along.
errorFallback={(err, retry) => <LoadingScreen error={err.message} onRetry={retry} />}
>
<LocalizationFetchProvider endpoint={localizationEndpoint}>
<UploadProvider adapter={uploadAdapter}>
<DefaultAppContent extraRoutes={systemRoutes} extraRoutesNoApp={systemRoutes} />
Expand Down
107 changes: 88 additions & 19 deletions packages/permissions/src/MePermissionsProvider.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@
*/

import React, { useEffect, useMemo, useState, useCallback } from 'react';
import {
PermissionsFetchError,
backoffMs,
isTransientFailure,
parseRetryAfterMs,
sleep,
} from './retry';
import type {
PermissionAction,
PermissionCheckResult,
Expand Down Expand Up @@ -65,6 +72,27 @@ export interface MePermissionsProviderProps {
loadingFallback?: React.ReactNode;
/** Rendered when load fails */
errorFallback?: (err: Error, retry: () => void) => React.ReactNode;
/**
* How many times to re-attempt a TRANSIENT failure before giving up — a
* response that says "not now" (`TRANSIENT_STATUS` in `./retry`) or a network
* error. `0` disables retrying.
*
* This exists because "not now" is a real answer from this endpoint. On a
* multi-tenant host it is served by the environment kernel that owns the
* session, and a cold one answers `503` + `Retry-After` while it warms
* (objectstack#4159). Without a retry the provider keeps `loadingFallback`
* on screen forever, because a consumer that passes no `errorFallback` — the
* console does exactly that — renders the loading node for the error state
* too, and nothing ever calls `retry`.
*
* @default 3
*/
maxRetries?: number;
/**
* Base for the exponential backoff between retries, in ms. A `Retry-After`
* header on the response wins over it. @default 500
*/
retryBaseDelayMs?: number;
/** Children */
children: React.ReactNode;
}
Expand All @@ -88,34 +116,75 @@ export function MePermissionsProvider({
initialPermissions,
loadingFallback = null,
errorFallback,
maxRetries = 3,
retryBaseDelayMs = 500,
children,
}: MePermissionsProviderProps) {
const [data, setData] = useState<MePermissionsResponse | null>(initialPermissions ?? null);
const [error, setError] = useState<Error | null>(null);
const [loading, setLoading] = useState(!initialPermissions);

const fetchPermissions = useCallback(async () => {
setLoading(true);
setError(null);
try {
/**
* Run the fetch, re-attempting a transient failure.
*
* `token` cancels an in-flight attempt. Retries put real time (a backoff, or
* a server-stated `Retry-After`) between the request and the `setState`, and
* during that window the effect may be torn down — by an unmount, or by
* `endpoint`/`fetcher` changing. Without the token a superseded attempt would
* still land, so a slow answer for the OLD endpoint could overwrite a fast
* answer for the new one.
*/
const fetchPermissions = useCallback(
async (token?: { cancelled: boolean }) => {
const live = () => !token?.cancelled;
setLoading(true);
setError(null);
const doFetch = fetcher ?? fetch;
const res = await doFetch(endpoint, {
credentials: 'include',
headers: { Accept: 'application/json' },
});
if (!res.ok) throw new Error(`Permissions endpoint returned ${res.status}`);
const json = (await res.json()) as MePermissionsResponse;
setData(json);
} catch (e) {
setError(e instanceof Error ? e : new Error(String(e)));
} finally {
setLoading(false);
}
}, [endpoint, fetcher]);
const attempts = Math.max(0, maxRetries) + 1;
for (let attempt = 0; attempt < attempts; attempt++) {
try {
const res = await doFetch(endpoint, {
credentials: 'include',
headers: { Accept: 'application/json' },
});
if (!res.ok) {
throw new PermissionsFetchError(
res.status,
parseRetryAfterMs(res.headers?.get?.('Retry-After'), Date.now()),
);
}
const json = (await res.json()) as MePermissionsResponse;
if (!live()) return;
setData(json);
setLoading(false);
return;
} catch (e) {
const err = e instanceof Error ? e : new Error(String(e));
if (!isTransientFailure(err) || attempt === attempts - 1) {
if (!live()) return;
setError(err);
setLoading(false);
return;
}
// `loading` stays true across the wait, so the fail-closed loading
// state holds and the recovery is invisible to consumers.
const stated = err instanceof PermissionsFetchError ? err.retryAfterMs : undefined;
await sleep(backoffMs(attempt, retryBaseDelayMs, stated));
if (!live()) return;
}
}
},
[endpoint, fetcher, maxRetries, retryBaseDelayMs],
);

/** The `retry` handed to `errorFallback` — uncancelled, it is user-initiated. */
const retry = useCallback(() => { void fetchPermissions(); }, [fetchPermissions]);

useEffect(() => {
if (initialPermissions) return;
void fetchPermissions();
const token = { cancelled: false };
void fetchPermissions(token);
return () => { token.cancelled = true; };
}, [fetchPermissions, initialPermissions]);

const checkField = useCallback(
Expand Down Expand Up @@ -232,7 +301,7 @@ export function MePermissionsProvider({

if (loading && !data) return <>{loadingFallback}</>;
if (error && !data) {
if (errorFallback) return <>{errorFallback(error, fetchPermissions)}</>;
if (errorFallback) return <>{errorFallback(error, retry)}</>;
return <>{loadingFallback}</>;
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
/**
* ObjectUI
* Copyright (c) 2024-present ObjectStack Inc.
*
* "Not now" is a real answer from `/me/permissions`, and it used to strand the
* console.
*
* On a multi-tenant host the endpoint is served by the environment kernel that
* owns the session, and a COLD one answers `503` + `Retry-After` while it warms
* (objectstack#4159 / cloud#927). The provider treated that like any other
* failure: it set `error`, and a consumer that passes no `errorFallback` — the
* console passes only `loadingFallback={<LoadingScreen />}` — renders the
* loading node for the error state too. So the app sat on its spinner forever,
* with a `retry` nobody could reach.
*
* These pin the retry: transient answers are re-attempted (server-stated delay
* first), real answers about the caller are not, and the fail-closed loading
* state holds throughout rather than flashing permissive.
*/

import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { render, screen, waitFor } from '@testing-library/react';
import React from 'react';
import { MePermissionsProvider } from '../MePermissionsProvider';
import { parseRetryAfterMs } from '../retry';
import { usePermissions } from '../usePermissions';

function Probe() {
const { isLoaded, check } = usePermissions();
return (
<div>
<span data-testid="loaded">{String(isLoaded)}</span>
<span data-testid="edit">{String(check('account', 'update').allowed)}</span>
</div>
);
}

const PAYLOAD = {
authenticated: true,
userId: 'u1',
tenantId: 't1',
roles: [],
permissionSets: ['member_default'],
objects: { account: { allowRead: true, allowEdit: false } },
fields: {},
};

const ok = () => ({ ok: true, status: 200, headers: new Headers(), json: async () => PAYLOAD });
const fail = (status: number, headers: Record<string, string> = {}) => ({
ok: false,
status,
headers: new Headers(headers),
json: async () => ({}),
});

/** Render with retries fast enough that the tests need no timer control. */
const renderProvider = (props: Record<string, unknown> = {}) =>
render(
<MePermissionsProvider retryBaseDelayMs={1} {...props}>
<Probe />
</MePermissionsProvider>,
);

describe('MePermissionsProvider retries a transient failure', () => {
beforeEach(() => { vi.spyOn(global, 'fetch' as any); });
afterEach(() => { vi.restoreAllMocks(); });

it('recovers from a warming 503 and serves the real answer', async () => {
(global.fetch as any)
.mockResolvedValueOnce(fail(503, { 'Retry-After': '0' }))
.mockResolvedValueOnce(ok());

renderProvider();

await waitFor(() => expect(screen.getByTestId('loaded').textContent).toBe('true'));
// The permission answer really arrived — not the anonymous fail-open.
expect(screen.getByTestId('edit').textContent).toBe('false');
expect((global.fetch as any)).toHaveBeenCalledTimes(2);
});

it('prefers the server-stated Retry-After over its own backoff', async () => {
// Timer-free proof: the backoff base is 100s, so if the header were ignored
// this test could not finish. `Retry-After: 0` says "now".
(global.fetch as any)
.mockResolvedValueOnce(fail(503, { 'Retry-After': '0' }))
.mockResolvedValueOnce(ok());

renderProvider({ retryBaseDelayMs: 100_000 });

await waitFor(() => expect(screen.getByTestId('loaded').textContent).toBe('true'));
});

it('retries a thrown fetch (offline / DNS / aborted) too', async () => {
(global.fetch as any)
.mockRejectedValueOnce(new TypeError('Failed to fetch'))
.mockResolvedValueOnce(ok());

renderProvider();

await waitFor(() => expect(screen.getByTestId('loaded').textContent).toBe('true'));
});

it('holds the fail-closed loading state while retrying — never a permissive flash', async () => {
(global.fetch as any)
.mockResolvedValueOnce(fail(503))
.mockResolvedValueOnce(fail(503))
.mockResolvedValueOnce(ok());

renderProvider({ loadingFallback: <span data-testid="spinner" /> });

// Children (and therefore any permissive default) are not rendered yet.
expect(screen.getByTestId('spinner')).toBeTruthy();
expect(screen.queryByTestId('loaded')).toBeNull();
await waitFor(() => expect(screen.getByTestId('loaded').textContent).toBe('true'));
});

it('gives up after maxRetries and surfaces the error', async () => {
(global.fetch as any).mockResolvedValue(fail(503));

const errorFallback = vi.fn((err: Error) => <span data-testid="err">{err.message}</span>);
renderProvider({ maxRetries: 2, errorFallback });

await waitFor(() => expect(screen.getByTestId('err')).toBeTruthy());
expect(screen.getByTestId('err').textContent).toContain('503');
expect((global.fetch as any)).toHaveBeenCalledTimes(3); // 1 attempt + 2 retries
});
});

describe('MePermissionsProvider does NOT retry a real answer about the caller', () => {
beforeEach(() => { vi.spyOn(global, 'fetch' as any); });
afterEach(() => { vi.restoreAllMocks(); });

it.each([401, 403, 404, 500])('gives up immediately on %i', async (status) => {
(global.fetch as any).mockResolvedValue(fail(status));

const errorFallback = vi.fn((err: Error) => <span data-testid="err">{err.message}</span>);
renderProvider({ errorFallback });

await waitFor(() => expect(screen.getByTestId('err')).toBeTruthy());
expect((global.fetch as any)).toHaveBeenCalledTimes(1);
});

it('maxRetries: 0 disables retrying entirely', async () => {
(global.fetch as any).mockResolvedValue(fail(503));

const errorFallback = vi.fn((err: Error) => <span data-testid="err">{err.message}</span>);
renderProvider({ maxRetries: 0, errorFallback });

await waitFor(() => expect(screen.getByTestId('err')).toBeTruthy());
expect((global.fetch as any)).toHaveBeenCalledTimes(1);
});
});

describe('parseRetryAfterMs', () => {
const NOW = Date.parse('2026-07-30T12:00:00Z');

it('reads delta-seconds', () => {
expect(parseRetryAfterMs('4', NOW)).toBe(4000);
expect(parseRetryAfterMs('0', NOW)).toBe(0);
expect(parseRetryAfterMs(' 7 ', NOW)).toBe(7000);
});

it('reads an HTTP-date, relative to now', () => {
expect(parseRetryAfterMs('Thu, 30 Jul 2026 12:00:03 GMT', NOW)).toBe(3000);
});

it('clamps a far-future value so it cannot park the UI', () => {
expect(parseRetryAfterMs('999999', NOW)).toBe(30_000);
expect(parseRetryAfterMs('Fri, 31 Jul 2026 12:00:00 GMT', NOW)).toBe(30_000);
});

it('ignores absent, empty, unparseable and past values', () => {
for (const v of [null, undefined, '', ' ', 'soon', 'Thu, 30 Jul 2026 11:59:00 GMT']) {
expect(parseRetryAfterMs(v, NOW), String(v)).toBeUndefined();
}
});
});
Loading
Loading