Resolution Search

[ngoiyaeric]

Drawings are not in context of subsequent resolution searches and relative to follow up questions. The map drawing contexts are indicative of the context that should be derived from the map view. The user should be able to continously extend context with their drawings.

[ngoiyaeric]

@coderabbitai plan Based on the codebase context, you're describing a gap where drawnFeatures from the map are not being included in resolution search context and follow-up questions. Let me trace through the current implementation and identify the issue.

Current Implementation

Drawn features are captured and stored:

• MapData context includes drawnFeatures array with geometry and measurements 1
• Mapbox drawing tools update this context when users draw polygons/lines 2
• Chat component saves drawing context to database via updateDrawingContext 3

Follow-up questions include drawn features:

• FollowupPanel appends drawnFeatures to form data when submitting 4
• The submit action in app/actions.tsx parses drawnFeatures from form data 5
• researcher agent incorporates drawn features into system prompt 6

The gap is in resolution search:

• The resolution_search action in app/actions.tsx processes the image file but does NOT pass drawnFeatures to the resolutionSearch function 7
• While resolutionSearch function accepts drawnFeatures parameter 8 , the calling code in app/actions.tsx doesn't provide it

The Fix

The resolution search action needs to:

1. Extract drawnFeatures from form data (similar to how it's done for regular submit)
2. Pass drawnFeatures to the resolutionSearch function call
3. Include camera state for full context

This would enable continuous context extension where drawings persist across resolution searches and follow-up questions.

Notes

The resolutionSearch function already has the infrastructure to use drawn features in its system prompt 9 , but the calling code in app/actions.tsx doesn't leverage this capability. The fix would involve modifying the resolution search action handler to extract and pass the drawing context.

Wiki pages you might want to explore:

• Interactive Map System (QueueLab/QCX)

Citations

File: components/map/map-data-context.tsx (L21-26)

  drawnFeatures?: Array<{ // Added to store drawn features and their measurements
    id: string;
    type: 'Polygon' | 'LineString';
    measurement: string;
    geometry: any;
  }>;

File: components/map/mapbox-map.tsx (L157-168)

      if (featureType && id && measurement && feature.geometry) {
        currentDrawnFeatures.push({
          id,
          type: featureType,
          measurement,
          geometry: feature.geometry,
        });
      }
    })

    setMapData(prevData => ({ ...prevData, drawnFeatures: currentDrawnFeatures }))
  }, [formatMeasurement, setMapData])

File: components/chat.tsx (L102-115)

  // useEffect to call the server action when drawnFeatures changes
  // OPTIMIZATION: Debounce drawing context updates
  useEffect(() => {
    if (id && mapData.drawnFeatures && mapData.cameraState) {
      const timer = setTimeout(() => {
        console.log('Chat.tsx: drawnFeatures changed, calling updateDrawingContext', mapData.drawnFeatures);
        updateDrawingContext(id, {
          drawnFeatures: mapData.drawnFeatures || [],
          cameraState: mapData.cameraState,
        });
      }, 500);
      return () => clearTimeout(timer);
    }
  }, [id, mapData.drawnFeatures, mapData.cameraState]); // OPTIMIZATION: Debounced update

File: components/followup-panel.tsx (L30-31)

    // Include drawn features in the form data
    formData.append('drawnFeatures', JSON.stringify(mapData.drawnFeatures || []))

File: app/actions.tsx (L45-51)

  const drawnFeaturesString = formData?.get('drawnFeatures') as string;
  let drawnFeatures: DrawnFeature[] = [];
  try {
    drawnFeatures = drawnFeaturesString ? JSON.parse(drawnFeaturesString) : [];
  } catch (e) {
    console.error('Failed to parse drawnFeatures:', e);
  }

File: app/actions.tsx (L67-117)

  if (action === 'resolution_search') {
    const file_mapbox = formData?.get('file_mapbox') as File;
    const file_google = formData?.get('file_google') as File;
    const file = (formData?.get('file') as File) || file_mapbox || file_google;
    const timezone = (formData?.get('timezone') as string) || 'UTC';
    const lat = formData?.get('latitude') ? parseFloat(formData.get('latitude') as string) : undefined;
    const lng = formData?.get('longitude') ? parseFloat(formData.get('longitude') as string) : undefined;
    const location = (lat !== undefined && lng !== undefined) ? { lat, lng } : undefined;

    if (!file) {
      throw new Error('No file provided for resolution search.');
    }

    const mapboxBuffer = file_mapbox ? await file_mapbox.arrayBuffer() : null;
    const mapboxDataUrl = mapboxBuffer ? `data:${file_mapbox.type};base64,${Buffer.from(mapboxBuffer).toString('base64')}` : null;

    const googleBuffer = file_google ? await file_google.arrayBuffer() : null;
    const googleDataUrl = googleBuffer ? `data:${file_google.type};base64,${Buffer.from(googleBuffer).toString('base64')}` : null;

    const buffer = await file.arrayBuffer();
    const dataUrl = `data:${file.type};base64,${Buffer.from(buffer).toString('base64')}`;

    const messages: CoreMessage[] = [...(aiState.get().messages as any[])].filter(
      (message: any) =>
        message.role !== 'tool' &&
        message.type !== 'followup' &&
        message.type !== 'related' &&
        message.type !== 'end' &&
        message.type !== 'resolution_search_result'
    );

    const userInput = 'Analyze this map view.';
    const content: CoreMessage['content'] = [
      { type: 'text', text: userInput },
      { type: 'image', image: dataUrl, mimeType: file.type }
    ];

    aiState.update({
      ...aiState.get(),
      messages: [
        ...aiState.get().messages,
        { id: nanoid(), role: 'user', content, type: 'input' }
      ]
    });
    messages.push({ role: 'user', content });

    const summaryStream = createStreamableValue<string>('Analyzing map view...');
    const groupeId = nanoid();

    async function processResolutionSearch() {
      try {

File: lib/agents/researcher.tsx (L25-27)

${drawnFeatures && drawnFeatures.length > 0 ? `The user has drawn the following features on the map for your reference:
${drawnFeatures.map(f => `- ${f.type} with measurement ${f.measurement}`).join('\n')}
Use these user-drawn areas/lines as primary areas of interest for your analysis if applicable to the query.` : ''}

File: lib/agents/resolution-search.tsx (L65-65)

export async function resolutionSearch(messages: CoreMessage[], timezone: string = 'UTC', drawnFeatures?: DrawnFeature[], location?: { lat: number, lng: number }) {

File: lib/agents/resolution-search.tsx (L117-120)

${drawnFeatures && drawnFeatures.length > 0 ? `**User-Drawn Features:**
The user has drawn the following features on the map for your reference:
${drawnFeatures.map(f => `- ${f.type} (${f.measurement}): ${JSON.stringify(f.geometry)}`).join('\n')}
Use these user-drawn areas/lines as primary areas of interest for your analysis.` : ''}

[coderabbitai]

Coding Plan

Summary

• Fix the carousel re-trigger so subsequent resolution searches carry the same drawings, coordinates, and timezone as the initial search (Phase 1).
• Persist drawing context into aiState each turn and merge it with live drawings so every subsequent resolution search and follow-up sees the accumulated set, while keeping that metadata out of the raw LLM message stream (Phase 2).
• Hydrate the map context from persisted drawing data on reload and resolve provider shadowing so accumulated drawings survive across sessions (Phase 3).

Design Choices

Design Choice 1: How should accumulated drawing context be carried across turns?

Options Considered:

1. Rely solely on the live MapDataContext and ensure every submission path forwards it (no server-side accumulation).
2. Persist drawing context into aiState as a dedicated drawing_context-typed message each turn, then merge persisted + live features when building agent prompts.

Chosen Option: 2

Rationale: It uses the already-declared-but-unused drawing_context type, makes accumulation durable across turns (and reload-recoverable), and directly satisfies "continuously extend context" without depending on transient client state.

Design Choice 2: How should overlapping/duplicate drawings be reconciled when merging?

Options Considered:

1. Union by id, with live features overriding persisted ones (keep historical features even if removed from the map).
2. Replace entirely with live features each turn (no historical retention).

Chosen Option: 1

Rationale: Union-by-id with live precedence preserves the "continuously extend" intent while preventing duplicate/stale geometry from confusing the model.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Restore Drawing & Geospatial Context in Resolution Re-Triggers

This phase fixes the most immediate, concrete gap: re-running a resolution search from the carousel currently discards all drawn features, map coordinates, and timezone, so the second analysis is "blind" relative to the first. The goal is to make the carousel's re-trigger behave consistently with the original resolution search by sourcing the same map context the other submission paths use.

Task 1: Task 1.1: Make the carousel re-trigger consume live map context

Ensure the "QCX-TERRA ANALYSIS" action carries the same context as an initial resolution search.

• Update components/resolution-carousel.tsx to consume useMapData() and read drawnFeatures, cameraState/targetPosition (for latitude/longitude), and currentTimezone.
• In handleQCXAnalysis, append drawnFeatures (serialized JSON), latitude, longitude, and timezone to the FormData alongside the existing image blob and action='resolution_search'.
• Mirror the field names and serialization used in components/header-search-button.tsx so the server action parses them identically.

Task 2: Task 1.2: Verify server-side parsing covers the re-trigger path

Confirm the resolution branch consistently consumes the newly added fields.

• Review the resolution_search branch in app/actions.tsx to ensure latitude/longitude, timezone, and drawnFeatures are read uniformly regardless of which component submitted them (no path-specific assumptions).
• Confirm resolutionSearch() receives a populated location and drawnFeatures for carousel-originated calls.

🤖 Prompt for AI agents

This phase restores drawing and geospatial context to the resolution carousel
re-trigger, ensuring it behaves identically to the initial resolution search
submission path.

**Task 1.1 — `components/resolution-carousel.tsx`:**
- Import and call `useMapData()` within the component to access `drawnFeatures`,
`cameraState`/`targetPosition` (for latitude/longitude), and `currentTimezone`.
- Inside `handleQCXAnalysis`, append the following fields to the existing
`FormData` (alongside the image blob and `action='resolution_search'`):
  - `drawnFeatures` as serialized JSON
  - `latitude` and `longitude` (derived from `cameraState`/`targetPosition`)
  - `timezone`
- Match the exact field names and serialization format used in
`components/header-search-button.tsx` so the server action in `app/actions.tsx`
can parse them without any path-specific logic.

**Task 1.2 — `app/actions.tsx`:**
- Review the `resolution_search` branch to confirm `latitude`/`longitude`,
`timezone`, and `drawnFeatures` are read from `FormData` uniformly, regardless
of whether the submission originated from the carousel or another component.
- Verify that `resolutionSearch()` receives a populated `location` object and
`drawnFeatures` for carousel-originated calls — no path-specific assumptions
should remain.

Phase 2: Persist and Accumulate Drawing Context Across Turns

This phase addresses the deeper issue: drawing context is only injected into a single agent call and never retained in conversation state, so subsequent resolution searches and follow-up questions cannot reliably "see" prior drawings. The goal is to make drawing context a durable, accumulating part of the conversation so each new query builds on the user's prior annotations.

Task 1: Task 2.1: Store drawing context in conversation state each turn

Persist the drawn features used for a query into aiState so they survive across turns.

• In app/actions.tsx, after parsing drawnFeatures for both the resolution-search branch and the standard researcher branch, append a drawing_context-typed message (the type already declared in lib/types/index.ts) to aiState capturing the serialized features.
• Treat this entry as conversation metadata, not user-visible chat content.

Task 2: Task 2.2: Build a merged (accumulated) drawing set for agent prompts

Combine historical and live drawings into a single context block.

• In app/actions.tsx, before invoking either agent, gather all prior drawing_context entries from aiState, union them with the current submission's live drawnFeatures, and de-duplicate by feature id (live features take precedence per the chosen merge strategy).
• Pass this merged set (rather than only the current live features) into both resolutionSearch() and researcher().

Task 3: Task 2.3: Keep drawing-context entries out of the LLM message stream

Ensure the new metadata messages inform prompts without polluting the conversation history.

• Extend the history-filtering logic in app/actions.tsx (the existing filters that already strip tool, followup, related, end, and resolution_search_result) to also exclude drawing_context-typed entries from the messages array sent to the LLM.
• Confirm lib/agents/resolution-search.tsx and lib/agents/researcher.tsx continue to render the merged features into their system prompts (geometry + measurement for resolution search; type + measurement for researcher), now driven by the accumulated set.

🤖 Prompt for AI agents

This phase makes drawing context a durable, accumulating part of conversation
state in `app/actions.tsx`, ensuring every subsequent agent call sees the full
history of drawn features without polluting the LLM message stream.

**Task 2.1 — Store drawing context in `aiState` (`app/actions.tsx`):**
- After parsing `drawnFeatures` from `FormData` in both the `resolution_search`
branch and the standard researcher branch, append a new message of type
`drawing_context` (already declared in `lib/types/index.ts`) to `aiState`.
- This message captures the serialized features for the current turn and should
be treated as conversation metadata (not user-visible chat content).

**Task 2.2 — Build merged drawing set before invoking agents
(`app/actions.tsx`):**
- Before calling either `resolutionSearch()` or `researcher()`, read all
existing `drawing_context`-typed entries from `aiState`.
- Union those persisted features with the current submission's live
`drawnFeatures`, de-duplicating by feature `id` — live features take precedence
over persisted ones with the same `id`.
- Pass this merged set into both `resolutionSearch()` and `researcher()` instead
of only the current live features.

**Task 2.3 — Filter `drawing_context` entries from the LLM message stream
(`app/actions.tsx`):**
- Extend the existing history-filtering logic (which already strips `tool`,
`followup`, `related`, `end`, and `resolution_search_result` typed messages) to
also exclude `drawing_context`-typed entries from the `messages` array forwarded
to the LLM.
- Confirm that `lib/agents/resolution-search.tsx` and
`lib/agents/researcher.tsx` still render the merged feature set into their
system prompts (geometry + measurement for resolution search; type + measurement
for researcher).

Phase 3: Hydrate Drawing Context on Load

This phase closes the persistence loop so accumulated drawings remain available after a page reload, supporting the "continuously extend context" expectation across sessions rather than only within a single mount. The goal is to seed the map context from previously persisted drawing data when a saved chat is reopened.

Task 1: Task 3.1: Seed MapDataContext from persisted drawing state

Read previously saved drawing context and initialize the map with it.

• In app/search/[id]/page.tsx, derive the latest persisted drawing context from the fetched messages (the role:'data' rows written by updateDrawingContext in lib/actions/chat.ts) and pass it as initial data into MapDataProvider.
• Update components/map/map-data-context.tsx to accept an optional initial MapData value so the provider can be seeded with hydrated drawnFeatures and cameraState instead of always starting empty.

Task 2: Task 3.2: Resolve duplicate provider shadowing

Ensure the hydrated context is the one actually read by consumers.

• Review the nested MapDataProvider wrappers in components/chat.tsx (mobile and desktop branches) that currently shadow the outer provider from SearchPage; ensure hydrated initial state is honored (either by removing the redundant inner providers or by threading the seed value through them) so useMapData() resolves to the hydrated context.

🤖 Prompt for AI agents

This phase closes the persistence loop by seeding the map context from
previously persisted drawing data when a saved chat is reopened, and resolving
provider shadowing so consumers actually receive the hydrated state.

**Task 3.1 — Seed `MapDataProvider` from persisted state:**
- In `app/search/[id]/page.tsx`, after fetching the chat messages, derive the
latest persisted drawing context from the `role:'data'` rows written by
`updateDrawingContext` in `lib/actions/chat.ts`.
- Pass the derived drawing data as initial data into `MapDataProvider`.
- Update `components/map/map-data-context.tsx` to accept an optional initial
`MapData` prop so the provider can be initialized with hydrated `drawnFeatures`
and `cameraState` instead of always starting with empty state.

**Task 3.2 — Resolve nested provider shadowing (`components/chat.tsx`):**
- Identify the nested `MapDataProvider` wrappers in the mobile and desktop
branches of `components/chat.tsx` that currently shadow the outer hydrated
provider from `SearchPage`.
- Resolve the shadowing by either removing the redundant inner providers, or
threading the hydrated seed value through them, so that `useMapData()` calls
throughout the component tree resolve to the hydrated context.

Research

Map drawings live in an in-memory React context (MapDataContext in components/map/map-data-context.tsx) as drawnFeatures (id, type, measurement, GeoJSON geometry). Submission components (chat-panel.tsx, followup-panel.tsx, header-search-button.tsx) serialize these into FormData, which the central submit server action in app/actions.tsx parses and forwards to either the researcher agent or the resolutionSearch agent (lib/agents/). Both agents inject drawn features into their system prompts only — the context is never stored in aiState, so it does not survive between turns except through whatever remains live on the map. The resolution carousel's re-trigger (components/resolution-carousel.tsx) builds its own FormData and omits drawings, coordinates, and timezone entirely. Drawing context is written to the DB as role:'data' messages but never read back on reload.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

This phase restores drawing and geospatial context to the resolution carousel
re-trigger, ensuring it behaves identically to the initial resolution search
submission path.

**Task 1.1 — `components/resolution-carousel.tsx`:**
- Import and call `useMapData()` within the component to access `drawnFeatures`,
`cameraState`/`targetPosition` (for latitude/longitude), and `currentTimezone`.
- Inside `handleQCXAnalysis`, append the following fields to the existing
`FormData` (alongside the image blob and `action='resolution_search'`):
  - `drawnFeatures` as serialized JSON
  - `latitude` and `longitude` (derived from `cameraState`/`targetPosition`)
  - `timezone`
- Match the exact field names and serialization format used in
`components/header-search-button.tsx` so the server action in `app/actions.tsx`
can parse them without any path-specific logic.

**Task 1.2 — `app/actions.tsx`:**
- Review the `resolution_search` branch to confirm `latitude`/`longitude`,
`timezone`, and `drawnFeatures` are read from `FormData` uniformly, regardless
of whether the submission originated from the carousel or another component.
- Verify that `resolutionSearch()` receives a populated `location` object and
`drawnFeatures` for carousel-originated calls — no path-specific assumptions
should remain.
===============================================================================

Task: 2

This phase makes drawing context a durable, accumulating part of conversation
state in `app/actions.tsx`, ensuring every subsequent agent call sees the full
history of drawn features without polluting the LLM message stream.

**Task 2.1 — Store drawing context in `aiState` (`app/actions.tsx`):**
- After parsing `drawnFeatures` from `FormData` in both the `resolution_search`
branch and the standard researcher branch, append a new message of type
`drawing_context` (already declared in `lib/types/index.ts`) to `aiState`.
- This message captures the serialized features for the current turn and should
be treated as conversation metadata (not user-visible chat content).

**Task 2.2 — Build merged drawing set before invoking agents
(`app/actions.tsx`):**
- Before calling either `resolutionSearch()` or `researcher()`, read all
existing `drawing_context`-typed entries from `aiState`.
- Union those persisted features with the current submission's live
`drawnFeatures`, de-duplicating by feature `id` — live features take precedence
over persisted ones with the same `id`.
- Pass this merged set into both `resolutionSearch()` and `researcher()` instead
of only the current live features.

**Task 2.3 — Filter `drawing_context` entries from the LLM message stream
(`app/actions.tsx`):**
- Extend the existing history-filtering logic (which already strips `tool`,
`followup`, `related`, `end`, and `resolution_search_result` typed messages) to
also exclude `drawing_context`-typed entries from the `messages` array forwarded
to the LLM.
- Confirm that `lib/agents/resolution-search.tsx` and
`lib/agents/researcher.tsx` still render the merged feature set into their
system prompts (geometry + measurement for resolution search; type + measurement
for researcher).
===============================================================================

Task: 3

This phase closes the persistence loop by seeding the map context from
previously persisted drawing data when a saved chat is reopened, and resolving
provider shadowing so consumers actually receive the hydrated state.

**Task 3.1 — Seed `MapDataProvider` from persisted state:**
- In `app/search/[id]/page.tsx`, after fetching the chat messages, derive the
latest persisted drawing context from the `role:'data'` rows written by
`updateDrawingContext` in `lib/actions/chat.ts`.
- Pass the derived drawing data as initial data into `MapDataProvider`.
- Update `components/map/map-data-context.tsx` to accept an optional initial
`MapData` prop so the provider can be initialized with hydrated `drawnFeatures`
and `cameraState` instead of always starting with empty state.

**Task 3.2 — Resolve nested provider shadowing (`components/chat.tsx`):**
- Identify the nested `MapDataProvider` wrappers in the mobile and desktop
branches of `components/chat.tsx` that currently shadow the outer hydrated
provider from `SearchPage`.
- Resolve the shadowing by either removing the redundant inner providers, or
threading the hydrated seed value through them, so that `useMapData()` calls
throughout the component tree resolve to the hydrated context.

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[coderabbitai]

Caution

CodeRabbit encountered an error during planning. Please try again later.