Skip to content

Batch-hydrate search hits, drop stale ones and self-heal the index#165

Open
loevgaard wants to merge 1 commit into
issue-138-ordered-searchable-attributesfrom
issue-120-searchaction-batch-hydration
Open

Batch-hydrate search hits, drop stale ones and self-heal the index#165
loevgaard wants to merge 1 commit into
issue-138-ordered-searchable-attributesfrom
issue-120-searchaction-batch-hydration

Conversation

@loevgaard

Copy link
Copy Markdown
Member

What

SearchAction re-hydrated every Meilisearch hit from the database one find() at a time — with the default hits_per_page of 60, up to 60 extra queries per search page, partly negating Meilisearch's speed. On top of that a hit whose entity no longer existed was silently dropped (so the "N results" count and the rendered cards could disagree, and the page could even render empty while claiming results), and find() didn't re-check enabled, so a product disabled after its last indexing was still displayed.

  • Batch hydration: hit ids are grouped by entityClass and each group is loaded with a single findBy(['id' => $ids]), then reordered to match Meilisearch's ranking. 60 queries become one per entity class.
  • Drop stale hits + self-heal: a hit whose entity is missing from the database, or is disabled (ToggleableInterface), is not rendered and a RemoveEntity is dispatched for it so the index heals on the next request. The dispatch is failure-safe — an error is logged, never breaks rendering.
  • The reorder/drop logic lives in the pure static reorderAndFilter() for straightforward unit testing.

Testing

  • SearchActionTest: entities are reordered to match the hit order; a hit whose entity is missing is dropped and reported stale; a disabled entity is dropped and reported stale.
  • The functional SearchSortingTest/SearchPaginationTest (which render the search page) still pass; lint:container is clean.

Closes #120


Stacked on #164 (#138).

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 35.13514% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.82%. Comparing base (3d030e8) to head (df023f3).

Files with missing lines Patch % Lines
src/Controller/Action/SearchAction.php 35.13% 24 Missing ⚠️
Additional details and impacted files
@@                              Coverage Diff                              @@
##             issue-138-ordered-searchable-attributes     #165      +/-   ##
=============================================================================
- Coverage                                      48.90%   48.82%   -0.09%     
- Complexity                                       781      796      +15     
=============================================================================
  Files                                            139      139              
  Lines                                           2521     2552      +31     
=============================================================================
+ Hits                                            1233     1246      +13     
- Misses                                          1288     1306      +18     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loevgaard loevgaard force-pushed the issue-120-searchaction-batch-hydration branch from 6b4d659 to ce7643b Compare July 10, 2026 17:16
@loevgaard loevgaard force-pushed the issue-138-ordered-searchable-attributes branch 2 times, most recently from a55c4a7 to 68b8ed4 Compare July 10, 2026 17:20
@loevgaard loevgaard force-pushed the issue-120-searchaction-batch-hydration branch from ce7643b to 32be954 Compare July 10, 2026 17:20
SearchAction re-hydrated every Meilisearch hit one find() at a time (up to
hits_per_page=60 extra queries per page), silently dropped hits whose entity
was gone (so the count and rendered cards could disagree), and rendered
products disabled after their last indexing.

- Group hit ids by entity class and load each group with a single
  findBy(['id' => ]), then reorder to Meilisearch's ranking.
- Drop hits whose entity is missing from the database or is disabled
  (ToggleableInterface), and dispatch RemoveEntity for them so the index
  self-heals (failure-safe: a dispatch error is logged, never breaks
  rendering).
- The reorder/drop logic is extracted into the pure static reorderAndFilter()
  for straightforward unit testing.

Closes #120
@loevgaard loevgaard force-pushed the issue-120-searchaction-batch-hydration branch from 32be954 to df023f3 Compare July 10, 2026 18:11
@loevgaard loevgaard force-pushed the issue-138-ordered-searchable-attributes branch from 68b8ed4 to 3d030e8 Compare July 10, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant