Skip to content

Commit 3e60cff

Browse files
lhawkman27lhawkinsclaudeEthan Bienstock
authored
Add lazy loading for layers in GeoSet Multi Map (#346)
* feat: Add lazy loading option for layers in GeoSet Multi Map Layers marked as "Lazy Loading" are loaded sequentially in the background after all eager layers have rendered, improving initial map load time for dashboards with many layers. Autozoom is automatically disabled for lazy-loaded layers. - Extract DeckSliceConfig, normalizeDeckSlices, and new resolveLayerAutozoom into multiUtils.ts for reuse and testability - Split layer loading into eager (parallel) and lazy (sequential) phases - Add Lazy Loading checkbox to per-layer settings popover in DeckSlicesControl - Add unit tests for resolveLayerAutozoom and normalizeDeckSlices Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: Extract layer orchestration logic and add tests - Extract loadLayersOrchestrated to multiUtils.ts for testability - Add load-bearing comment on .catch in loadSingleLayer - Deduplicate normalizeValue in DeckSlicesControl (reuse normalizeDeckSlices) - Add 8 orchestration tests: mixed eager/lazy, all-lazy, staleness cancellation, null filtering, and empty slices Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Address lazy loading bugs, harden orchestrator, and clean up imports Fix critical bug where lazy-loaded layers with initiallyHidden were not hidden on arrival (the effect only fired on the first 0→N transition). Now detects newly-added layers by comparing previous vs current slice ID sets so both eager and lazy layers are handled correctly. Add .catch() to fire-and-forget loadLayersOrchestrated call to prevent unhandled promise rejections. Wrap loadFn calls with Promise.resolve() for defensive handling of synchronous throws. Remove no-op .then(() => {}). Re-export DeckSliceConfig and normalizeDeckSlices from the plugin barrel and add tsconfig path mapping so DeckSlicesControl imports from the package entry point instead of reaching into src/. Restore autozoom to its saved value when lazy loading is toggled off. Rename test file to multiUtils.test.ts and add error rejection tests. Update wiki docs for multiUtils.ts and per-layer settings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Resolve TypeScript build error in multiUtils generic type Use type assertion instead of type predicate in Promise.all filter to avoid Awaited<TLayer> vs TLayer mismatch in generic context. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Batch lazy layer loading and preserve autozoom draft state Switch lazy layer loading from sequential to batched (batch size 2) for better throughput, simplify safeLoadFn, and fix autozoom restore to use the user's draft value via a ref instead of the saved prop. Update wiki to reflect batched loading behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Reset autozoom ref on popover reopen and improve staleness test Reset autozoomBeforeLazyRef when the layer settings popover reopens so that toggling lazy loading off after a save/reopen cycle no longer restores a stale autozoom value. Also expand the lazy-chain abort test to span multiple batch boundaries for more robust coverage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Clarify lazyLoading comment to reflect configuration intent Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Show legend entries immediately with loading spinners while layers load Build stub legend entries from slice metadata (already fetched) so the legend is fully populated when the map opens. Each stub shows an antd Spin indicator in place of checkboxes until the layer data arrives and replaces the stub with the full legend content. - Add buildStubLegendEntry() to parse legend name/icon from form_data - Add pendingLegends memo to build stubs from slicesData - Merge stubs into legendsBySlice so loaded entries replace stubs - Add loading field to LegendEntry type - Show Spin spinner for loading entries in LegendEntryContent - Show Spin spinner for group checkbox when all entries are loading Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Render map canvas immediately while layers load in background The loading gate blocked the entire map canvas (base map, legend, controls) behind a loading GIF until all eager layers finished via Promise.all. Changed the condition to only block on slice metadata fetch so the base map is visible immediately and layers appear incrementally as they load. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: Three-phase layer loading — prioritize autozoom layers Refactor two-phase loading into three phases so the map canvas renders as soon as autozoom layers finish, without waiting for all eager layers: Phase 1: Autozoom layers load in parallel (map canvas gates on this) Phase 2: Remaining eager layers load in parallel, appended incrementally Phase 3: Lazy layers load in batches, appended incrementally This prevents the viewport jump that occurred when the map rendered with a default viewport then snapped to the autozoom extent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: Update multiUtils.ts description to reflect three-phase loading Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Pre-set hidden layer visibility from config to prevent flash Layer visibility for initiallyHidden layers is now set from config at the start of loadLayers, before any network requests. This eliminates the timing dependency where the old useEffect-based approach could render a hidden+lazy layer as visible for one frame before hiding it. Category visibility is still set on load (categories aren't known until data is fetched). The layerStatesWithVisibility fallback provides an additional synchronous safety net during render. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: Memoize layerStatesWithVisibility to prevent spurious re-renders Wrap the layerStatesWithVisibility computation in useMemo so DeckGLContainer only receives new prop references when sortedLayers or layerVisibility actually change, avoiding unnecessary renders. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: Add optimistic visibility toggle tests for MultiLegend Verify that group checkboxes update immediately on click via optimistic local state, without waiting for the layerVisibility prop to change from the parent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: lhawkins <lhawkins@teamraft.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Ethan Bienstock <ethan.bienstock@acf.hhs.gov>
1 parent 2ce704d commit 3e60cff

12 files changed

Lines changed: 1498 additions & 412 deletions

File tree

‎superset-frontend/plugins/geoset-map-chart/src/GeoSetMultiMap/Multi.tsx‎

Lines changed: 437 additions & 353 deletions
Large diffs are not rendered by default.

‎superset-frontend/plugins/geoset-map-chart/src/GeoSetMultiMap/multiUtils.ts‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
/**
2+
* Licensed to the Apache Software Foundation (ASF) under one
3+
* or more contributor license agreements. See the NOTICE file
4+
* distributed with this work for additional information
5+
* regarding copyright ownership. The ASF licenses this file
6+
* to you under the Apache License, Version 2.0 (the
7+
* "License"); you may not use this file except in compliance
8+
* with the License. You may obtain a copy of the License at
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on an
14+
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
20+
export interface DeckSliceConfig {
21+
sliceId: number;
22+
autozoom: boolean;
23+
legendCollapsed: boolean;
24+
initiallyHidden: boolean;
25+
lazyLoading: boolean;
26+
}
27+
28+
/** Resolve effective autozoom for a slice: disabled when lazy loading is on */
29+
export const resolveLayerAutozoom = (
30+
config: DeckSliceConfig | undefined,
31+
): boolean => (config?.lazyLoading ? false : (config?.autozoom ?? true));
32+
33+
/** Normalize deck slices — converts legacy number[] format to DeckSliceConfig[] */
34+
export const normalizeDeckSlices = (
35+
deckSlices: (DeckSliceConfig | number)[] | undefined,
36+
): DeckSliceConfig[] =>
37+
deckSlices?.map(item =>
38+
typeof item === 'number'
39+
? {
40+
sliceId: item,
41+
autozoom: true,
42+
legendCollapsed: false,
43+
initiallyHidden: false,
44+
lazyLoading: false,
45+
}
46+
: {
47+
sliceId: item.sliceId,
48+
autozoom: item.autozoom ?? true,
49+
legendCollapsed: item.legendCollapsed ?? false,
50+
initiallyHidden: item.initiallyHidden ?? false,
51+
lazyLoading: item.lazyLoading ?? false,
52+
},
53+
) ?? [];
54+
55+
/** Callbacks for {@link loadLayersOrchestrated}. */
56+
export interface OrchestrationCallbacks<TLayer> {
57+
/** Load a single slice, returning null on failure. */
58+
loadFn: (
59+
subslice: { slice_id: number },
60+
config: DeckSliceConfig | undefined,
61+
) => Promise<TLayer | null>;
62+
/** Called once with autozoom layers after they finish loading (phase 1). */
63+
onAutozoomComplete: (layers: TLayer[]) => void;
64+
/** Called for each non-autozoom eager layer as it finishes (phase 2). */
65+
onEagerAppend: (layer: TLayer) => void;
66+
/** Called for each lazy layer as it finishes loading sequentially (phase 3). */
67+
onLazyAppend: (layer: TLayer) => void;
68+
/** Return true to abort — checked before each phase and between lazy loads. */
69+
isStale: () => boolean;
70+
}
71+
72+
/** Number of lazy layers to load concurrently in each batch. */
73+
const LAZY_BATCH_SIZE = 2;
74+
75+
/**
76+
* Orchestrates three-phase layer loading:
77+
* Phase 1 — load autozoom layers in parallel (map canvas gates on this).
78+
* Phase 2 — load remaining eager layers in parallel, appending each as it finishes.
79+
* Phase 3 — load lazy layers in small batches, appending each as it finishes.
80+
*
81+
* Returns a promise that resolves when the full chain finishes
82+
* or is aborted due to staleness.
83+
*/
84+
export function loadLayersOrchestrated<TLayer>(
85+
slices: { slice_id: number }[],
86+
deckSlicesConfig: DeckSliceConfig[],
87+
callbacks: OrchestrationCallbacks<TLayer>,
88+
): Promise<void> {
89+
if (!slices || slices.length === 0) return Promise.resolve();
90+
91+
const configById = new Map(deckSlicesConfig.map(c => [c.sliceId, c]));
92+
93+
const autozoomSlices: { slice_id: number }[] = [];
94+
const eagerSlices: { slice_id: number }[] = [];
95+
const lazySlices: { slice_id: number }[] = [];
96+
97+
slices.forEach(subslice => {
98+
const config = configById.get(subslice.slice_id);
99+
if (config?.lazyLoading) {
100+
lazySlices.push(subslice);
101+
} else if (resolveLayerAutozoom(config)) {
102+
autozoomSlices.push(subslice);
103+
} else {
104+
eagerSlices.push(subslice);
105+
}
106+
});
107+
108+
// Async wrapper ensures synchronous throws from loadFn become rejected promises
109+
const safeLoadFn = async (
110+
subslice: { slice_id: number },
111+
config: DeckSliceConfig | undefined,
112+
): Promise<TLayer | null> => callbacks.loadFn(subslice, config);
113+
114+
// Phase 1: Load autozoom layers in parallel
115+
const autozoomPromise =
116+
autozoomSlices.length > 0
117+
? Promise.all(
118+
autozoomSlices.map(subslice =>
119+
safeLoadFn(subslice, configById.get(subslice.slice_id)),
120+
),
121+
).then(results => results.filter(e => e !== null) as TLayer[])
122+
: Promise.resolve([] as TLayer[]);
123+
124+
return autozoomPromise.then(autozoomLayers => {
125+
if (callbacks.isStale()) return undefined;
126+
127+
callbacks.onAutozoomComplete(autozoomLayers);
128+
129+
// Phase 2 + 3 run sequentially after autozoom completes
130+
// eslint-disable-next-line consistent-return
131+
return (async () => {
132+
// Phase 2: Load remaining eager layers in parallel, append each as it finishes
133+
if (eagerSlices.length > 0) {
134+
const eagerResults = await Promise.all(
135+
eagerSlices.map(subslice =>
136+
safeLoadFn(subslice, configById.get(subslice.slice_id)),
137+
),
138+
);
139+
140+
for (const layerEntry of eagerResults) {
141+
if (layerEntry && !callbacks.isStale()) {
142+
callbacks.onEagerAppend(layerEntry);
143+
}
144+
}
145+
}
146+
147+
// Phase 3: Load lazy layers in batches of LAZY_BATCH_SIZE
148+
for (let i = 0; i < lazySlices.length; i += LAZY_BATCH_SIZE) {
149+
if (callbacks.isStale()) return;
150+
151+
const batch = lazySlices.slice(i, i + LAZY_BATCH_SIZE);
152+
// eslint-disable-next-line no-await-in-loop
153+
const results = await Promise.all(
154+
batch.map(subslice =>
155+
safeLoadFn(subslice, configById.get(subslice.slice_id)),
156+
),
157+
);
158+
159+
for (const layerEntry of results) {
160+
if (layerEntry && !callbacks.isStale()) {
161+
callbacks.onLazyAppend(layerEntry);
162+
}
163+
}
164+
}
165+
})();
166+
});
167+
}

‎superset-frontend/plugins/geoset-map-chart/src/components/MultiLegend.tsx‎

Lines changed: 31 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import { styled } from '@superset-ui/core';
2+
import { Spin } from '@superset-ui/core/components/Spin';
23
import { useState, useEffect, useRef } from 'react';
34
import MapIcon from '@material-ui/icons/MapTwoTone';
45
import { RGBAColor } from '../utils/colors';
@@ -222,6 +223,15 @@ const LegendEntryContent: React.FC<{
222223
onToggleVisibility,
223224
onToggleCategory,
224225
}) => {
226+
if (legendEntry.loading) {
227+
return (
228+
<CategoryRow>
229+
<Spin size="small" />
230+
<div>{legendEntry.legendName}</div>
231+
</CategoryRow>
232+
);
233+
}
234+
225235
const { fill, stroke } = getDefaultColors(legendEntry);
226236

227237
return (
@@ -482,26 +492,31 @@ export const MultiLegend: React.FC<MultiLegendProps> = ({
482492
const isIndeterminate =
483493
someVisibleSomeNot || (isVisible && hasPartialCategories);
484494

495+
const allLoading = entries.every(e => e.legendEntry.loading);
496+
485497
return (
486498
<Group key={displayTitle}>
487499
{/* Header */}
488500
<Header>
489-
{showGroupCheckboxes && (
490-
<IndeterminateCheckbox
491-
checked={isVisible}
492-
indeterminate={isIndeterminate}
493-
onChange={e => {
494-
e.stopPropagation();
495-
setOptimisticVisibility(prev => ({
496-
...prev,
497-
...Object.fromEntries(
498-
allSliceIds.map(id => [id, !isVisible]),
499-
),
500-
}));
501-
onToggleLayerVisibility?.(allSliceIds);
502-
}}
503-
/>
504-
)}
501+
{showGroupCheckboxes &&
502+
(allLoading ? (
503+
<Spin size="small" />
504+
) : (
505+
<IndeterminateCheckbox
506+
checked={isVisible}
507+
indeterminate={isIndeterminate}
508+
onChange={e => {
509+
e.stopPropagation();
510+
setOptimisticVisibility(prev => ({
511+
...prev,
512+
...Object.fromEntries(
513+
allSliceIds.map(id => [id, !isVisible]),
514+
),
515+
}));
516+
onToggleLayerVisibility?.(allSliceIds);
517+
}}
518+
/>
519+
))}
505520
<TitleRow
506521
onClick={() => toggle(displayTitle, initialCollapsed)}
507522
>

‎superset-frontend/plugins/geoset-map-chart/src/index.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,7 @@ import './utils/mapboxApi';
2222
export { default as GeoSetMapChartPreset } from './preset';
2323
export { default as GeoSetMapGeoJsonChartPlugin } from './layers/GeoSetLayer';
2424
export { default as MultiChartPlugin } from './GeoSetMultiMap';
25+
export {
26+
normalizeDeckSlices,
27+
type DeckSliceConfig,
28+
} from './GeoSetMultiMap/multiUtils';

‎superset-frontend/plugins/geoset-map-chart/src/types.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@ export type LegendEntry = {
103103
sizeEntry?: SizeLegend;
104104
isCombinedMetricSize?: boolean;
105105
initialCollapsed?: boolean; // Whether this legend entry starts collapsed
106+
loading?: boolean; // True for stub entries whose layer data is still loading
106107
};
107108

108109
export type LegendGroup = {

0 commit comments

Comments
 (0)