Skip to content

Commit a6bb1bd

Browse files
Merge pull request #1257 from objectstack-ai/copilot/minimize-metadata-loading
refactor(console): lazy-load metadata in MetadataProvider
2 parents 532e4cd + b01f92a commit a6bb1bd

5 files changed

Lines changed: 892 additions & 77 deletions

File tree

CHANGELOG.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Changed
11+
12+
- **`MetadataProvider` now lazy-loads metadata.** Previously the console
13+
fetched the full `app`, `object`, `dashboard`, `report`, and `page`
14+
lists in parallel at app startup, so first-paint cost scaled linearly
15+
with tenant size. The provider now only fetches the `app` list
16+
eagerly (required by the router and the App switcher); the other
17+
buckets are loaded on demand the first time a consumer reads them.
18+
Concurrent reads share a single in-flight request, results are cached
19+
with a 5 minute TTL, and the eager `app` list is hydrated from
20+
`sessionStorage` on reload for an instant first paint.
21+
- New context API: `ensureType(type)`, `getItem(type, name)`,
22+
`invalidate(type, name?)`, `refresh(type?)` (per-type form).
23+
- New hooks: `useMetadataType(type)`, `useMetadataItem(type, name)`.
24+
- The legacy `useMetadata()` shape (`apps`, `objects`, `dashboards`,
25+
`reports`, `pages`, `loading`, `error`, `refresh`,
26+
`getItemsByType`) is preserved — reading any of the lazy array
27+
properties transparently triggers `ensureType` so existing
28+
components keep working without changes. The `loading` flag now
29+
reflects only the initial `app` load, not lazy buckets.
30+
1031
### Fixed
1132

33+
- **"Object Not Found" / React "Rendered more hooks" crash on app entry.**
34+
A regression introduced by the lazy `MetadataProvider` refactor above:
35+
`useMetadata().objects` started empty while the lazy fetch was in flight,
36+
so `ObjectView` hit its `if (!objectDef) return <Empty/>` early return on
37+
the first render; once the fetch resolved, the hooks declared below that
38+
early return ran and React threw _"Rendered more hooks than during the
39+
previous render"_. `AppContent` now preloads the `object`, `dashboard`,
40+
`report`, and `page` buckets (via `ensureType`) before rendering the
41+
routes under `/apps/:appName/*`, so legacy components that assume
42+
metadata is fully loaded by render time continue to work. `/home`,
43+
`/login` and `/register` do not go through `AppContent`, so the Phase 1
44+
benefit of fetching only the `app` list at boot is preserved.
45+
1246
- **Record detail header** no longer renders two separate "More" (⋯) overflow
1347
menus when an object defines more `record_header` actions than
1448
`maxVisible`. The hardcoded `<DropdownMenu>` inside

ROADMAP.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1410,6 +1410,42 @@ Plugin architecture refactoring to support true modular development, plugin isol
14101410

14111411
## 🐛 Bug Fixes
14121412

1413+
### Console Metadata Lazy Loading (Phase 1) (April 2026)
1414+
1415+
**Root Cause:** The console's `MetadataProvider` fetched the full `app`,
1416+
`object`, `dashboard`, `report`, and `page` lists in parallel at startup
1417+
and held them in five top-level arrays via `useMetadata()`. As tenant
1418+
metadata grew, first-paint TTI, browser memory, and API pressure all
1419+
scaled linearly with total catalogue size — even when entering a single
1420+
app that only needed a fraction of it.
1421+
1422+
**Fix (Phase 1):** Refactored `MetadataProvider` to a per-type cache map
1423+
(`status / items / byName / fetchedAt / promise`) with these
1424+
characteristics:
1425+
1426+
1. Only the `app` list is fetched on mount (required by router and App
1427+
switcher); other types are fully lazy.
1428+
2. New imperative API on the context value: `ensureType(type)` (in-flight
1429+
de-duplicated), `getItem(type, name)` (single-record fetch + cache),
1430+
`invalidate(type, name?)`, and per-type `refresh(type?)`.
1431+
3. New hooks: `useMetadataType(type)` and `useMetadataItem(type, name)`.
1432+
4. 5-minute TTL freshness on list buckets; `app` list is hydrated from
1433+
`sessionStorage` on reload for instant first paint.
1434+
5. Fully backward compatible: `apps / objects / dashboards / reports /
1435+
pages` are now lazy getters that auto-trigger `ensureType` on first
1436+
read, so all 20+ existing call sites keep working unchanged.
1437+
6. Dev-only timing/cache-hit logs.
1438+
1439+
**Tests:** 7 new `MetadataProviderLazy.test.tsx` cases (only `app`
1440+
fetched on mount, lazy fetch on getter access, in-flight de-duplication,
1441+
single-item fetch & cache, per-record invalidation, scoped `refresh`,
1442+
`useMetadataItem` hook). Full console suite: 891 passed / 1 skipped.
1443+
1444+
**Follow-ups (deferred to later phases of the original plan):** route-
1445+
scoped `AppScopeProvider`, per-component migration to
1446+
`useMetadataItem`, route-level `React.lazy` chunking, and server-side
1447+
`?app=` filter pushdown / ETag support.
1448+
14131449
### Record Detail "Record Not Found" in External Metadata Environments (March 2026)
14141450

14151451
**Root Cause:** Three compounding issues caused "Record not found" when navigating from a list to a record detail page in external metadata environments:

apps/console/src/App.tsx

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,9 +99,42 @@ export function AppContent() {
9999
const navigate = useNavigate();
100100
const location = useLocation();
101101
const { appName } = useParams();
102-
const { apps, objects: allObjects, loading: metadataLoading } = useMetadata();
102+
const { apps, objects: allObjects, loading: metadataLoading, ensureType } = useMetadata();
103103
const { t } = useObjectTranslation();
104104
const { objectLabel } = useObjectLabel();
105+
106+
// Preload the metadata buckets that the routes under /apps/:appName/* assume
107+
// are fully loaded by render time. The Phase-1 MetadataProvider refactor
108+
// made these buckets lazy (only `app` is eager); without this preload some
109+
// legacy consumers (notably `ObjectView`) would mount with `objects=[]`,
110+
// hit an `if (!objectDef) return <Empty/>` early-return, then break React's
111+
// rules-of-hooks once the lazy fetch resolves and the hooks below the early
112+
// return start running. /home and /login don't go through AppContent, so
113+
// they keep the Phase-1 benefit of fetching only the `app` list at boot.
114+
//
115+
// `ensureType` is optional at the destructure site so that the many test
116+
// files which provide a partial `useMetadata()` mock continue to work
117+
// unchanged; in those environments the preload is a no-op (the mocked
118+
// metadata arrays are already populated synchronously).
119+
const [scopeMetaReady, setScopeMetaReady] = useState(!ensureType);
120+
useEffect(() => {
121+
if (!ensureType) {
122+
setScopeMetaReady(true);
123+
return;
124+
}
125+
let cancelled = false;
126+
Promise.all([
127+
ensureType('object'),
128+
ensureType('dashboard'),
129+
ensureType('report'),
130+
ensureType('page'),
131+
]).finally(() => {
132+
if (!cancelled) setScopeMetaReady(true);
133+
});
134+
return () => {
135+
cancelled = true;
136+
};
137+
}, [ensureType]);
105138

106139
// Determine active app based on URL
107140
const activeApps = apps.filter((a: any) => a.active !== false);
@@ -253,7 +286,7 @@ export function AppContent() {
253286
[user, activeApp, editingRecord]
254287
);
255288

256-
if (!dataSource || metadataLoading) return <LoadingScreen />;
289+
if (!dataSource || metadataLoading || !scopeMetaReady) return <LoadingScreen />;
257290

258291
// Allow create-app route even when no active app exists
259292
const isCreateAppRoute = location.pathname.endsWith('/create-app');
Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
/**
2+
* MetadataProvider lazy-loading tests
3+
*
4+
* Verifies the Phase 1 refactor: only the `app` type is fetched at
5+
* provider mount; other metadata types are deferred until a consumer
6+
* actually reads them. Also covers in-flight de-duplication and
7+
* per-type invalidation/refresh behaviour.
8+
*/
9+
10+
import { describe, it, expect, vi, beforeEach } from 'vitest';
11+
import { render, act, waitFor } from '@testing-library/react';
12+
import { MetadataProvider, useMetadata, useMetadataItem } from '../context/MetadataProvider';
13+
14+
// ---------------------------------------------------------------------------
15+
// Helpers
16+
// ---------------------------------------------------------------------------
17+
18+
function createMockAdapter() {
19+
const meta = {
20+
getItems: vi.fn().mockImplementation((type: string) => {
21+
// Each type returns a distinguishable single-item payload.
22+
return Promise.resolve({ items: [{ name: `${type}-1` }] });
23+
}),
24+
getItem: vi.fn().mockImplementation((type: string, name: string) => {
25+
return Promise.resolve({ item: { name, type } });
26+
}),
27+
};
28+
const adapter = {
29+
getClient: () => ({ meta }),
30+
} as any;
31+
return { adapter, meta };
32+
}
33+
34+
function flushAll() {
35+
// Two microtask ticks are needed: one for the `ensureType` promise
36+
// chain, another for the React state update that bumps the provider's
37+
// version counter. `waitFor` is used elsewhere for assertion polling;
38+
// `flushAll` just gets us past the initial mount transition.
39+
return act(async () => {
40+
await Promise.resolve();
41+
await Promise.resolve();
42+
});
43+
}
44+
45+
// Component that captures the latest context value into a ref-like object
46+
// so assertions can run outside of render.
47+
function Capture({ sink, read }: { sink: { value: any }; read?: (ctx: any) => void }) {
48+
const ctx = useMetadata();
49+
sink.value = ctx;
50+
if (read) read(ctx);
51+
return null;
52+
}
53+
54+
beforeEach(() => {
55+
if (typeof sessionStorage !== 'undefined') sessionStorage.clear();
56+
});
57+
58+
// ---------------------------------------------------------------------------
59+
// Tests
60+
// ---------------------------------------------------------------------------
61+
62+
describe('MetadataProvider — lazy loading (Phase 1)', () => {
63+
it('fetches only the app list on initial mount', async () => {
64+
const { adapter, meta } = createMockAdapter();
65+
const sink = { value: null as any };
66+
67+
render(
68+
<MetadataProvider adapter={adapter}>
69+
<Capture sink={sink} />
70+
</MetadataProvider>,
71+
);
72+
73+
await flushAll();
74+
await waitFor(() => expect(sink.value.loading).toBe(false));
75+
76+
// Only `app` should have been fetched at this point.
77+
const calledTypes = meta.getItems.mock.calls.map((c: any[]) => c[0]);
78+
expect(calledTypes).toEqual(['app']);
79+
expect(sink.value.apps).toEqual([{ name: 'app-1' }]);
80+
81+
// Lazy buckets are still empty arrays — no fetch yet.
82+
expect(meta.getItems).toHaveBeenCalledTimes(1);
83+
});
84+
85+
it('lazily fetches other types only when accessed via the legacy getter', async () => {
86+
const { adapter, meta } = createMockAdapter();
87+
const sink = { value: null as any };
88+
89+
render(
90+
<MetadataProvider adapter={adapter}>
91+
<Capture sink={sink} />
92+
</MetadataProvider>,
93+
);
94+
await flushAll();
95+
await waitFor(() => expect(sink.value.loading).toBe(false));
96+
expect(meta.getItems).toHaveBeenCalledTimes(1);
97+
98+
// Read `objects` via the legacy property (auto-triggers ensureType).
99+
let firstRead: any[] = [];
100+
await act(async () => {
101+
firstRead = sink.value.objects;
102+
});
103+
// Fetch is in-flight; the synchronous read returns the empty snapshot.
104+
expect(firstRead).toEqual([]);
105+
106+
// Wait for the lazy fetch to settle.
107+
await waitFor(() =>
108+
expect(meta.getItems).toHaveBeenCalledWith('object'),
109+
);
110+
await waitFor(() => expect(sink.value.objects).toEqual([{ name: 'object-1' }]));
111+
112+
// Still only two list fetches in total — `dashboard / report / page` untouched.
113+
expect(meta.getItems).toHaveBeenCalledTimes(2);
114+
const types = meta.getItems.mock.calls.map((c: any[]) => c[0]).sort();
115+
expect(types).toEqual(['app', 'object']);
116+
});
117+
118+
it('deduplicates concurrent ensureType calls into a single network request', async () => {
119+
const { adapter, meta } = createMockAdapter();
120+
const sink = { value: null as any };
121+
122+
render(
123+
<MetadataProvider adapter={adapter}>
124+
<Capture sink={sink} />
125+
</MetadataProvider>,
126+
);
127+
await flushAll();
128+
await waitFor(() => expect(sink.value.loading).toBe(false));
129+
130+
await act(async () => {
131+
const a = sink.value.ensureType('dashboard');
132+
const b = sink.value.ensureType('dashboard');
133+
const c = sink.value.ensureType('dashboard');
134+
await Promise.all([a, b, c]);
135+
});
136+
137+
const dashCalls = meta.getItems.mock.calls.filter((c: any[]) => c[0] === 'dashboard');
138+
expect(dashCalls.length).toBe(1);
139+
});
140+
141+
it('getItem fetches a single record without triggering a list fetch', async () => {
142+
const { adapter, meta } = createMockAdapter();
143+
const sink = { value: null as any };
144+
145+
render(
146+
<MetadataProvider adapter={adapter}>
147+
<Capture sink={sink} />
148+
</MetadataProvider>,
149+
);
150+
await flushAll();
151+
await waitFor(() => expect(sink.value.loading).toBe(false));
152+
153+
let item: any;
154+
await act(async () => {
155+
item = await sink.value.getItem('dashboard', 'sales-overview');
156+
});
157+
expect(item).toEqual({ name: 'sales-overview', type: 'dashboard' });
158+
159+
// No list fetch for `dashboard`.
160+
const dashCalls = meta.getItems.mock.calls.filter((c: any[]) => c[0] === 'dashboard');
161+
expect(dashCalls.length).toBe(0);
162+
expect(meta.getItem).toHaveBeenCalledWith('dashboard', 'sales-overview');
163+
164+
// Second call serves from the in-memory cache.
165+
meta.getItem.mockClear();
166+
await act(async () => {
167+
item = await sink.value.getItem('dashboard', 'sales-overview');
168+
});
169+
expect(item).toEqual({ name: 'sales-overview', type: 'dashboard' });
170+
expect(meta.getItem).not.toHaveBeenCalled();
171+
});
172+
173+
it('invalidate(type, name) drops a single record without dropping the bucket', async () => {
174+
const { adapter, meta } = createMockAdapter();
175+
const sink = { value: null as any };
176+
177+
render(
178+
<MetadataProvider adapter={adapter}>
179+
<Capture sink={sink} />
180+
</MetadataProvider>,
181+
);
182+
await flushAll();
183+
await waitFor(() => expect(sink.value.loading).toBe(false));
184+
185+
// Prime the dashboard bucket.
186+
await act(async () => {
187+
await sink.value.ensureType('dashboard');
188+
});
189+
expect(sink.value.dashboards).toEqual([{ name: 'dashboard-1' }]);
190+
191+
// Invalidate the single item — the bucket is preserved but the entry vanishes.
192+
await act(async () => {
193+
sink.value.invalidate('dashboard', 'dashboard-1');
194+
});
195+
expect(sink.value.dashboards).toEqual([]);
196+
197+
// Re-reading via getItem triggers a single-item fetch.
198+
meta.getItem.mockClear();
199+
await act(async () => {
200+
await sink.value.getItem('dashboard', 'dashboard-1');
201+
});
202+
expect(meta.getItem).toHaveBeenCalledWith('dashboard', 'dashboard-1');
203+
});
204+
205+
it('refresh(type) re-fetches a single bucket; refresh() refreshes only loaded buckets', async () => {
206+
const { adapter, meta } = createMockAdapter();
207+
const sink = { value: null as any };
208+
209+
render(
210+
<MetadataProvider adapter={adapter}>
211+
<Capture sink={sink} />
212+
</MetadataProvider>,
213+
);
214+
await flushAll();
215+
await waitFor(() => expect(sink.value.loading).toBe(false));
216+
217+
// Load `object` only; leave dashboard/report/page idle.
218+
await act(async () => {
219+
await sink.value.ensureType('object');
220+
});
221+
222+
meta.getItems.mockClear();
223+
await act(async () => {
224+
await sink.value.refresh();
225+
});
226+
227+
// refresh() should re-fetch `app` and `object` (loaded), not the idle types.
228+
const refreshed = meta.getItems.mock.calls.map((c: any[]) => c[0]).sort();
229+
expect(refreshed).toEqual(['app', 'object']);
230+
});
231+
});
232+
233+
describe('useMetadataItem hook', () => {
234+
it('resolves to the fetched item and reports loading transitions', async () => {
235+
const { adapter, meta } = createMockAdapter();
236+
let captured: any = null;
237+
238+
function Probe() {
239+
const state = useMetadataItem('dashboard', 'sales');
240+
captured = state;
241+
return null;
242+
}
243+
244+
render(
245+
<MetadataProvider adapter={adapter}>
246+
<Probe />
247+
</MetadataProvider>,
248+
);
249+
250+
expect(captured.loading).toBe(true);
251+
await waitFor(() => expect(captured.loading).toBe(false));
252+
expect(captured.item).toEqual({ name: 'sales', type: 'dashboard' });
253+
expect(captured.error).toBeNull();
254+
expect(meta.getItem).toHaveBeenCalledWith('dashboard', 'sales');
255+
});
256+
});

0 commit comments

Comments
 (0)