Content search service — multiSearch + highlighting facade

CLAUDE.md

@@ -55,7 +55,8 @@ extraction, Meilisearch indexing with scheduled refresh, and four MCP tools
 │   ├── ContentIndexStore.java               # index read/write seam (StoredDocument)
 │   ├── MeilisearchContentIndexStore.java    # ContentIndexStore backed by MeilisearchClient
 │   ├── ContentBootstrapStep.java            # SearchIndexBootstrapStep: startup full reindex
-│   └── ContentRefreshScheduler.java         # @Scheduled incremental re-crawl (cron, guarded)
+│   ├── ContentRefreshScheduler.java         # @Scheduled incremental re-crawl (cron, guarded)
+│   └── ContentSearchService.java            # read facade: multiSearch + Highlighter (+ result records)
 ├── src/main/resources/application.yaml      # datasource, JPA, OAuth2, MCP, Meilisearch, content config
 ├── src/test/java/com/openelements/content/  # behavior tests (context, MCP enabled/disabled, search-down, jsoup)
 └── docs/
@@ -113,7 +114,11 @@ stream (over all enabled sources, via `ContentIndexer.streamAllDocuments`) in ba
 and toggles `SearchReadinessState`. `ContentRefreshScheduler` is a `@Scheduled` bean (cron
 `open-elements.content.refresh-cron`, default hourly, from `@EnableScheduling` in spec 001) that
 re-runs `ContentIndexer.indexSource` over enabled sources — guarded to skip while disabled or
-bootstrapping and to never overlap, with per-source fault isolation.
+bootstrapping and to never overlap, with per-source fault isolation. On the read side,
+`ContentSearchService` is the facade the MCP tools (spec 012) call: it builds Meilisearch
+`multi-search` bodies (AND-combined `source`/`locale`/`categories`/`since` filters, `publishedDate:desc`
+tie-breaker, `Highlighter` boundary markers) and returns `SearchHit`s with HTML-safe snippets, plus
+`listPosts`, `getByUrlOrId`, and `categoryFacets`. Read-only; the scoped key comes in spec 013.
 
 > **Key gotcha:** the library ships no Spring Boot auto-configuration and couples MCP to a JPA
 > datasource, so `@Import({ McpConfiguration, SearchConfig })` alone does **not** boot. See

docs/specs/011-content-search-service/steps.md

@@ -0,0 +1,52 @@
+# Implementation Steps: Content Search Service
+
+## Step 1: Result/filter records
+
+- [x] `ContentFilters(source, locale, category, since)` (+ `none()`)
+- [x] `SearchHit(title, url, publishedDate, snippet, score)`, `SearchHits(hits, estimatedTotal)`, `CategoryCount(category, count)`
+
+**Related behaviors:** all (return/argument shapes)
+
+---
+
+## Step 2: `ContentSearchService`
+
+- [x] `@Component` (gated on `meilisearch.enabled`) over `MeilisearchClient` + `MeilisearchProperties`
+- [x] `search`: builds a `multi-search` body — filter, `publishedDate:desc` sort, highlight tags = `Highlighter.PRE_MARK`/`POST_MARK`, crop `body`, `showRankingScore`, paging — and parses hits with `Highlighter.safeHighlight`
+- [x] `listPosts`: empty query, `publishedDate:desc`, paged
+- [x] `getByUrlOrId`: filter by `id` (preferred) or `url`; empty when both blank / not found
+- [x] `categoryFacets`: `facets:["categories"]` with source/locale filter, `limit 0`
+- [x] `buildFilter` AND-combines non-null filters (quote-escaped)
+
+**Related behaviors:** all search/filter/list/get/facet scenarios
+
+---
+
+## Step 3: Tests
+
+- [x] `ContentSearchServiceTest`: pure `buildFilter`/`parseHits`/`parseFacets` + public methods via a capturing `MeilisearchClient` subclass (request bodies + parsing + get + empty)
+
+**Acceptance criteria:**
+- [x] All tests pass (`mvn test`); build green
+
+---
+
+## Behavior Coverage
+
+| Scenario | Layer | Covered in Step |
+|----------|-------|-----------------|
+| Query returns relevant hits with snippet | Backend | Step 3 (`ResponseParsing.parsesHits` + `RequestBuilding.searchBuildsRequest`) |
+| Title matches rank above body matches | Backend | Governed by `IndexSettings` searchable order (spec 003) + Meilisearch; the service adds only the date tie-breaker (verified in `searchBuildsRequest`). Live ranking needs Meilisearch. |
+| Highlight markers become safe em tags | Backend | Step 3 (`highlightMarkersBecomeSafeEmTags`) |
+| Locale/Source/Category filter restricts results | Backend | Step 3 (`FilterBuilding.*` + `searchBuildsRequest`) |
+| since filters by published date | Backend | Step 3 (`FilterBuilding.since`) |
+| Combined filters are AND-combined | Backend | Step 3 (`FilterBuilding.combined`) |
+| listPosts sorts by published date descending | Backend | Step 3 (`listPostsBuildsRequest`) |
+| Empty query returns all (filtered) posts | Backend | Step 3 (`listPostsBuildsRequest`) |
+| Get by url / id returns full document | Backend | Step 3 (`GetByUrlOrId.returnsFullDocument`, `getById/UrlBuildsFilter`) |
+| Get for unknown url/id is empty | Backend | Step 3 (`emptyWhenNotFound`, `emptyWhenNoArguments`) |
+| Category facets return counts / respect filter | Backend | Step 3 (`parsesFacets` + `facetsBuildRequest`) |
+| No results / paging beyond the end | Backend | Step 3 (`parsesEmpty`; paging via `offset` in `searchBuildsRequest`) |
+
+All scenarios are backend; there is no frontend in this spec. Live ranking/filtering is Meilisearch's
+responsibility (the service builds the correct request); tests verify the request and the parsing.

docs/specs/INDEX.md

@@ -16,7 +16,7 @@ roadmap step, to be implemented sequentially (each builds on the previous).
 | 008 | 008-content-indexer | Content indexer | backend, search, crawler | `ContentIndexer` — orchestration, diff, upsert/delete | #15 | done |
 | 009 | 009-content-bootstrap-step | Content bootstrap step | backend, search | `ContentBootstrapStep` — initial reindex via `SearchIndexBootstrapStep` | #17 | done |
 | 010 | 010-content-refresh-scheduler | Content refresh scheduler | backend, scheduling | `ContentRefreshScheduler` — `@Scheduled` incremental re-crawl | #19 | done |
-| 011 | 011-content-search-service | Content search service | backend, search | `ContentSearchService` — `multiSearch` + highlighting facade | — | open |
+| 011 | 011-content-search-service | Content search service | backend, search | `ContentSearchService` — `multiSearch` + highlighting facade | #21 | done |
 | 012 | 012-content-mcp-tools | Content MCP tools | backend, mcp, api | `ContentMcpToolProvider` — the 4 MCP tools | — | open |
 | 013 | 013-scoped-search-key | Scoped search key | backend, search, security | Read-only scoped Meilisearch key for the content index | — | open |
 | 014 | 014-ops-robustness | Ops & robustness | backend, infrastructure, observability | robots.txt handling, fault tolerance, logging/metrics | — | open |

src/main/java/com/openelements/content/CategoryCount.java

@@ -0,0 +1,10 @@
+package com.openelements.content;
+
+/**
+ * A category facet with its document count.
+ *
+ * @param category the category value
+ * @param count    the number of matching documents
+ */
+public record CategoryCount(String category, long count) {
+}

src/main/java/com/openelements/content/ContentFilters.java

@@ -0,0 +1,18 @@
+package com.openelements.content;
+
+/**
+ * Optional filters for a content search or listing. Any {@code null} field is not applied; non-null
+ * fields are AND-combined.
+ *
+ * @param source   restrict to a source id
+ * @param locale   restrict to a locale (e.g. {@code en}, {@code de})
+ * @param category restrict to a category tag
+ * @param since    restrict to documents with {@code publishedDate >=} this ISO date
+ */
+public record ContentFilters(String source, String locale, String category, String since) {
+
+    /** @return filters with no restrictions */
+    public static ContentFilters none() {
+        return new ContentFilters(null, null, null, null);
+    }
+}

src/main/java/com/openelements/content/ContentSearchService.java

@@ -0,0 +1,245 @@
+package com.openelements.content;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.openelements.spring.base.services.search.Highlighter;
+import com.openelements.spring.base.services.search.MeilisearchClient;
+import com.openelements.spring.base.services.search.MeilisearchProperties;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
+import org.springframework.stereotype.Component;
+
+/**
+ * Read-only facade over the Meilisearch content index, backing the MCP tools (spec 012).
+ *
+ * <p>It builds {@code multi-search} request bodies with AND-combined filters, a
+ * {@code publishedDate:desc} tie-breaker, and the library {@link Highlighter}'s boundary markers as
+ * highlight tags, then turns the raw hits into {@link SearchHit}s with HTML-safe snippets. Searchable
+ * ranking ({@code title > excerpt > body}) is governed by the {@code IndexSettings} (spec 003); this
+ * service only adds the date tie-breaker and never writes.
+ *
+ * <p>Only created when the Meilisearch stack is enabled.
+ */
+@Component
+@ConditionalOnProperty(prefix = "openelements.meilisearch", name = "enabled", havingValue = "true")
+public class ContentSearchService {
+
+    /** Words of {@code body} kept around a match for the snippet preview. */
+    static final int SNIPPET_CROP_LENGTH = 50;
+
+    private final MeilisearchClient client;
+    private final String indexUid;
+
+    public ContentSearchService(MeilisearchClient client, MeilisearchProperties meilisearchProperties) {
+        this.client = client;
+        this.indexUid = meilisearchProperties.resolveIndex("content");
+    }
+
+    /**
+     * Full-text search with filters, highlighting, and a date tie-breaker.
+     *
+     * @param query   the search terms (blank matches everything)
+     * @param filters optional filters
+     * @param page    zero-based page index
+     * @param size    page size
+     * @return the matching hits
+     */
+    public SearchHits search(String query, ContentFilters filters, int page, int size) {
+        Map<String, Object> query0 = baseQuery(filters, page, size);
+        query0.put("q", query == null ? "" : query);
+        query0.put("sort", List.of("publishedDate:desc"));
+        query0.put("attributesToHighlight", List.of("title", "excerpt", "body"));
+        query0.put("attributesToCrop", List.of("body"));
+        query0.put("cropLength", SNIPPET_CROP_LENGTH);
+        query0.put("highlightPreTag", Highlighter.PRE_MARK);
+        query0.put("highlightPostTag", Highlighter.POST_MARK);
+        query0.put("showRankingScore", true);
+        return parseHits(client.multiSearch(wrap(query0)));
+    }
+
+    /**
+     * Lists documents (no query) sorted newest-first, with optional filters.
+     *
+     * @param filters optional filters
+     * @param page    zero-based page index
+     * @param size    page size
+     * @return the matching documents as hits
+     */
+    public SearchHits listPosts(ContentFilters filters, int page, int size) {
+        Map<String, Object> query0 = baseQuery(filters, page, size);
+        query0.put("q", "");
+        query0.put("sort", List.of("publishedDate:desc"));
+        return parseHits(client.multiSearch(wrap(query0)));
+    }
+
+    /**
+     * Fetches a full document by id (preferred) or url.
+     *
+     * @param url the document URL (used when {@code id} is blank)
+     * @param id  the document id
+     * @return the document, or empty if none matches (or both arguments are blank)
+     */
+    public Optional<ContentDocument> getByUrlOrId(String url, String id) {
+        String filter;
+        if (present(id)) {
+            filter = "id = \"" + escape(id) + "\"";
+        } else if (present(url)) {
+            filter = "url = \"" + escape(url) + "\"";
+        } else {
+            return Optional.empty();
+        }
+        Map<String, Object> query0 = new LinkedHashMap<>();
+        query0.put("indexUid", indexUid);
+        query0.put("q", "");
+        query0.put("filter", filter);
+        query0.put("limit", 1);
+        return firstDocument(client.multiSearch(wrap(query0)));
+    }
+
+    /**
+     * Category facet counts, optionally restricted by source/locale.
+     *
+     * @param source optional source filter
+     * @param locale optional locale filter
+     * @return the categories with their document counts
+     */
+    public List<CategoryCount> categoryFacets(String source, String locale) {
+        Map<String, Object> query0 = new LinkedHashMap<>();
+        query0.put("indexUid", indexUid);
+        query0.put("q", "");
+        String filter = buildFilter(new ContentFilters(source, locale, null, null));
+        if (filter != null) {
+            query0.put("filter", filter);
+        }
+        query0.put("facets", List.of("categories"));
+        query0.put("limit", 0);
+        return parseFacets(client.multiSearch(wrap(query0)));
+    }
+
+    // ---- request building ----
+
+    private Map<String, Object> baseQuery(ContentFilters filters, int page, int size) {
+        Map<String, Object> query = new LinkedHashMap<>();
+        query.put("indexUid", indexUid);
+        String filter = buildFilter(filters);
+        if (filter != null) {
+            query.put("filter", filter);
+        }
+        int safeSize = Math.max(0, size);
+        query.put("limit", safeSize);
+        query.put("offset", Math.max(0, page) * safeSize);
+        return query;
+    }
+
+    private static Map<String, Object> wrap(Map<String, Object> query) {
+        return Map.of("queries", List.of(query));
+    }
+
+    /** Builds the AND-combined Meilisearch filter expression, or {@code null} when no filter applies. */
+    static String buildFilter(ContentFilters filters) {
+        if (filters == null) {
+            return null;
+        }
+        List<String> clauses = new ArrayList<>();
+        if (present(filters.source())) {
+            clauses.add("source = \"" + escape(filters.source()) + "\"");
+        }
+        if (present(filters.locale())) {
+            clauses.add("locale = \"" + escape(filters.locale()) + "\"");
+        }
+        if (present(filters.category())) {
+            clauses.add("categories = \"" + escape(filters.category()) + "\"");
+        }
+        if (present(filters.since())) {
+            clauses.add("publishedDate >= \"" + escape(filters.since()) + "\"");
+        }
+        return clauses.isEmpty() ? null : String.join(" AND ", clauses);
+    }
+
+    // ---- response parsing ----
+
+    static SearchHits parseHits(JsonNode response) {
+        JsonNode result = firstResult(response);
+        if (result == null) {
+            return SearchHits.empty();
+        }
+        List<SearchHit> hits = new ArrayList<>();
+        for (JsonNode hit : result.path("hits")) {
+            hits.add(toHit(hit));
+        }
+        return new SearchHits(hits, result.path("estimatedTotalHits").asLong(0));
+    }
+
+    private static SearchHit toHit(JsonNode hit) {
+        JsonNode formatted = hit.path("_formatted");
+        String rawSnippet = firstNonBlank(text(formatted, "body"), text(formatted, "excerpt"), text(hit, "excerpt"));
+        String snippet = rawSnippet == null ? "" : Highlighter.safeHighlight(rawSnippet);
+        return new SearchHit(
+            text(hit, "title"), text(hit, "url"), text(hit, "publishedDate"),
+            snippet, hit.path("_rankingScore").asDouble(0.0));
+    }
+
+    static List<CategoryCount> parseFacets(JsonNode response) {
+        JsonNode result = firstResult(response);
+        if (result == null) {
+            return List.of();
+        }
+        List<CategoryCount> counts = new ArrayList<>();
+        result.path("facetDistribution").path("categories").fields()
+            .forEachRemaining(entry -> counts.add(new CategoryCount(entry.getKey(), entry.getValue().asLong(0))));
+        return counts;
+    }
+
+    private static Optional<ContentDocument> firstDocument(JsonNode response) {
+        JsonNode result = firstResult(response);
+        if (result == null) {
+            return Optional.empty();
+        }
+        JsonNode hits = result.path("hits");
+        if (!hits.isArray() || hits.isEmpty()) {
+            return Optional.empty();
+        }
+        return Optional.of(toDocument(hits.get(0)));
+    }
+
+    private static ContentDocument toDocument(JsonNode hit) {
+        List<String> categories = new ArrayList<>();
+        hit.path("categories").forEach(category -> categories.add(category.asText()));
+        return new ContentDocument(
+            text(hit, "id"), text(hit, "source"), text(hit, "locale"), text(hit, "url"),
+            text(hit, "title"), text(hit, "excerpt"), text(hit, "body"), text(hit, "author"),
+            categories, text(hit, "publishedDate"), text(hit, "lastmod"), text(hit, "previewImage"));
+    }
+
+    // ---- small helpers ----
+
+    private static JsonNode firstResult(JsonNode response) {
+        JsonNode results = response.path("results");
+        return results.isArray() && !results.isEmpty() ? results.get(0) : null;
+    }
+
+    private static String text(JsonNode node, String field) {
+        JsonNode value = node.path(field);
+        return value.isMissingNode() || value.isNull() ? null : value.asText();
+    }
+
+    private static String firstNonBlank(String... values) {
+        for (String value : values) {
+            if (value != null && !value.isBlank()) {
+                return value;
+            }
+        }
+        return null;
+    }
+
+    private static boolean present(String value) {
+        return value != null && !value.isBlank();
+    }
+
+    private static String escape(String value) {
+        return value.replace("\"", "\\\"");
+    }
+}

src/main/java/com/openelements/content/SearchHit.java

@@ -0,0 +1,13 @@
+package com.openelements.content;
+
+/**
+ * One search or list result: the display fields plus a safe, highlighted snippet.
+ *
+ * @param title         the document title
+ * @param url           the canonical URL
+ * @param publishedDate the published date
+ * @param snippet       an HTML-safe snippet with {@code <em>} around matches (may be empty)
+ * @param score         the ranking score (0 when not available)
+ */
+public record SearchHit(String title, String url, String publishedDate, String snippet, double score) {
+}

src/main/java/com/openelements/content/SearchHits.java

@@ -0,0 +1,20 @@
+package com.openelements.content;
+
+import java.util.List;
+
+/**
+ * A page of search/list results.
+ *
+ * @param hits           the results on this page
+ * @param estimatedTotal Meilisearch's estimate of the total matching documents
+ */
+public record SearchHits(List<SearchHit> hits, long estimatedTotal) {
+
+    public SearchHits {
+        hits = hits == null ? List.of() : List.copyOf(hits);
+    }
+
+    static SearchHits empty() {
+        return new SearchHits(List.of(), 0);
+    }
+}