Warn when an encrypted search runs before the index has finished building (#34001)

* Warn when an encrypted search runs before the index has finished building

When a search runs in an encrypted room while the local Seshat index is still
crawling not-yet-indexed history, results can silently come back partial.
SearchWarning now subscribes to the event index's changedCheckpoint progress and
shows a polite (role=status) notice while the crawl is in progress, clearing
automatically the moment indexing finishes.

* Scope the partial-index warning to the room being searched

The warning was driven by `currentRoom() !== null`, which is a global signal:
it is non-null while the crawler has any outstanding checkpoint for any room.
Searching a fully-crawled room while an unrelated room was still being crawled
therefore claimed the results may be incomplete when they were not.

Drive it from `crawlingRooms()` instead, which reports the rooms with
outstanding checkpoints by id, and pass the search scope and room id in from
RoomSearchAuxPanel: a room-scoped search now asks only about that room, while
an all-rooms search still reacts to any outstanding checkpoint. Using room ids
throughout also avoids `currentRoom()` returning null, and so under-reporting a
crawl, when the js-sdk does not know the room at the head of the queue.

The `changedCheckpoint` payload only carries the globally-current room and
cannot answer a per-room question, so the handler re-reads the checkpoint set.

* Also warn when the searched room has not been indexed at all

The crawl set cannot see a room that has no checkpoint: before the initial
checkpoints have been seeded, such a room is absent from it and looks identical
to one that has been fully crawled. That is the case issue #32253 describes, so
ask isRoomIndexed() as well, and warn when the index holds no events for the
room being searched.

Only ask it while the crawler still has work outstanding. That is what the
warning claims, and the index has no event for its contents changing --
changedCheckpoint fires on checkpoint transitions only, and an idle crawler is
silent -- so a warning raised once the crawler had drained would never be
re-evaluated and would stick.

Rename the hook to useIsIndexIncomplete, as it no longer answers the narrower
question of whether a crawl is in progress. Re-seed it from the checkpoint set
on each scope or room change so that the previous search's answer is not left
on screen while the lookup is in flight, but not on each checkpoint change,
which would blink an already-earned warning off and on again.

Also reword the crawlingRooms() doc, which described the set as the rooms being
crawled when it holds every queued checkpoint too.

---------

Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>

Author: hayaksi1

apps/web/src/components/views/elements/SearchWarning.tsx

@@ -6,10 +6,12 @@ SPDX-License-Identifier: AGPL-3.0-only OR GPL-3.0-only OR LicenseRef-Element-Com
 Please see LICENSE files in the repository root for full details.
 */
 
-import React, { type JSX, type ReactNode } from "react";
+import React, { type JSX, type ReactNode, useCallback, useEffect, useState } from "react";
 import { logger } from "matrix-js-sdk/src/logger";
 
 import EventIndexPeg from "../../../indexing/EventIndexPeg";
+import type EventIndex from "../../../indexing/EventIndex";
+import { SearchScope } from "../../../Searching";
 import { _t } from "../../../languageHandler";
 import SdkConfig from "../../../SdkConfig";
 import dis from "../../../dispatcher/dispatcher";
@@ -26,11 +28,131 @@ interface IProps {
     isRoomEncrypted?: boolean;
     kind: WarningKind;
     showLogo?: boolean;
+    /** The scope of the search being warned about; only meaningful for {@link WarningKind.Search}. */
+    scope?: SearchScope;
+    /** The room being searched. Mirrors `SearchInfo.roomId`: `undefined` when searching all rooms. */
+    roomId?: string;
 }
 
-export default function SearchWarning({ isRoomEncrypted, kind, showLogo = true }: IProps): JSX.Element {
+/**
+ * Track whether the index is still missing history that is relevant to the given search.
+ *
+ * A room-scoped search is incomplete if the crawler still holds a checkpoint for the room
+ * ({@link EventIndex.crawlingRooms}, which covers both the checkpoint being crawled right now and
+ * those still queued behind it), or if the index holds no events for it at all
+ * ({@link EventIndex.isRoomIndexed}) — which is how a room looks before its checkpoint has been
+ * seeded, and is the case the checkpoint set alone cannot see.
+ *
+ * The second question is only asked while the crawler still has work outstanding, for two reasons.
+ * It is what the warning claims ("your search index is still being built"), and the index has no
+ * event for its contents changing — `changedCheckpoint` fires only on checkpoint transitions, and
+ * an idle crawler is silent — so a warning raised once the crawler has drained would never be
+ * re-evaluated and would stick.
+ *
+ * Neither signal proves completeness: `isRoomIndexed` reports only that the index holds *some*
+ * events for a room, not all of them, and a room has no checkpoint if it never had a
+ * back-pagination token to crawl from or if its checkpoint was dropped because the server rejected
+ * the request. So this under-warns rather than over-warns.
+ *
+ * An all-rooms search cannot ask the per-room question, so it uses the checkpoint set alone.
+ *
+ * The `changedCheckpoint` payload carries only the globally-current room and so cannot answer a
+ * per-room question: we re-read the index on each event rather than trust it.
+ *
+ * @param index The event index to observe, or `null` if there is no index.
+ * @param scope The scope of the search, if this warning is being rendered for one.
+ * @param roomId The room being searched, or `undefined` when searching all rooms.
+ * @returns `true` while the index is known to be missing history for the search, `false` otherwise.
+ */
+function useIsIndexIncomplete(index: EventIndex | null, scope?: SearchScope, roomId?: string): boolean {
+    const readCheckpoints = useCallback((): { relevant: boolean; anyOutstanding: boolean } => {
+        if (!index) return { relevant: false, anyOutstanding: false };
+        const { crawlingRooms } = index.crawlingRooms();
+        // Fall back to the global check when we don't know which room is being searched: the room
+        // id may still be undefined while a room alias is being resolved.
+        const roomScoped = scope === SearchScope.Room && roomId !== undefined;
+        return {
+            relevant: roomScoped ? crawlingRooms.has(roomId) : crawlingRooms.size > 0,
+            anyOutstanding: crawlingRooms.size > 0,
+        };
+    }, [index, scope, roomId]);
+
+    // The checkpoint half of the answer is known synchronously, so seed from it rather than
+    // rendering an unwarned search for a room we already know is being crawled.
+    const [incomplete, setIncomplete] = useState<boolean>(() => readCheckpoints().relevant);
+
+    useEffect(() => {
+        if (!index) {
+            setIncomplete(false);
+            return;
+        }
+
+        // Guards against a slow isRoomIndexed() response overwriting a newer one, or landing after
+        // the scope changed or the component unmounted.
+        let generation = 0;
+
+        // Answer this scope and room from the checkpoint set up front, so that the previous
+        // search's result is not left on screen while the first lookup below is in flight. Doing
+        // this per effect run rather than per event matters: a checkpoint change is not a new
+        // question, and resetting on one would blink an already-earned warning off and on again.
+        setIncomplete(readCheckpoints().relevant);
+
+        const update = async (): Promise<void> => {
+            const current = ++generation;
+            const { relevant, anyOutstanding } = readCheckpoints();
+
+            if (relevant) {
+                setIncomplete(true);
+                return;
+            }
+            if (!anyOutstanding || scope !== SearchScope.Room || roomId === undefined) {
+                setIncomplete(false);
+                return;
+            }
+
+            // Nothing is queued for this room yet, but the index may hold nothing for it at all.
+            // `undefined` means there is no index manager to ask, which is not evidence either way.
+            const indexed = await index.isRoomIndexed(roomId);
+            if (current === generation) setIncomplete(indexed === false);
+        };
+
+        const onChangedCheckpoint = (): void => {
+            void update();
+        };
+
+        // Re-sync in case the crawl state changed between the initial render and the subscription.
+        onChangedCheckpoint();
+        index.on("changedCheckpoint", onChangedCheckpoint);
+
+        return () => {
+            generation++;
+            index.removeListener("changedCheckpoint", onChangedCheckpoint);
+        };
+    }, [index, scope, roomId, readCheckpoints]);
+
+    return incomplete;
+}
+
+export default function SearchWarning({ isRoomEncrypted, kind, showLogo = true, scope, roomId }: IProps): JSX.Element {
+    const eventIndex = EventIndexPeg.get();
+    const indexIncomplete = useIsIndexIncomplete(eventIndex, scope, roomId);
+
     if (!isRoomEncrypted) return <></>;
-    if (EventIndexPeg.get()) return <></>;
+
+    if (eventIndex) {
+        // The index is still missing history for this search, so it may silently return partial
+        // results (#32253). Warn the user.
+        if (indexIncomplete && kind === WarningKind.Search) {
+            // This warning appears dynamically while a search panel is already open (the crawler
+            // finishes draining mid-session), so mark it as a polite live region for screen readers.
+            return (
+                <div className="mx_SearchWarning" role="status">
+                    <span>{_t("seshat|warning_kind_search_partial")}</span>
+                </div>
+            );
+        }
+        return <></>;
+    }
 
     if (EventIndexPeg.error) {
         return (

apps/web/src/components/views/rooms/RoomSearchAuxPanel.tsx

@@ -45,7 +45,13 @@ const RoomSearchAuxPanel: React.FC<Props> = ({ searchInfo, isRoomEncrypted, onSe
                         ) : (
                             <InlineSpinner />
                         )}
-                        <SearchWarning kind={WarningKind.Search} isRoomEncrypted={isRoomEncrypted} showLogo={false} />
+                        <SearchWarning
+                            kind={WarningKind.Search}
+                            isRoomEncrypted={isRoomEncrypted}
+                            showLogo={false}
+                            scope={scope}
+                            roomId={searchInfo?.roomId}
+                        />
                     </div>
                 </div>
                 <div className="mx_RoomSearchAuxPanel_buttons">

apps/web/src/i18n/strings/en_EN.json

@@ -2434,7 +2434,8 @@
         "warning_kind_files": "This version of %(brand)s does not support viewing some encrypted files",
         "warning_kind_files_app": "Use the <a>Desktop app</a> to see all encrypted files",
         "warning_kind_search": "This version of %(brand)s does not support searching encrypted messages",
-        "warning_kind_search_app": "Use the <a>Desktop app</a> to search encrypted messages"
+        "warning_kind_search_app": "Use the <a>Desktop app</a> to search encrypted messages",
+        "warning_kind_search_partial": "Results may be incomplete because your search index is still being built."
     },
     "setting": {
         "help_about": {

apps/web/src/indexing/EventIndex.ts

@@ -992,7 +992,10 @@ export default class EventIndex extends EventEmitter {
     }
 
     public crawlingRooms(): {
-        /** The rooms that we are currently crawling. */
+        /**
+         * The rooms with an outstanding crawler checkpoint: the one being crawled right now, and
+         * those still queued behind it.
+         */
         crawlingRooms: Set<string>;
 
         /** All the encrypted rooms known by the MatrixClient. */