Batch-hydrate search hits, drop stale ones and self-heal the index#165
Open
loevgaard wants to merge 1 commit into
Open
Batch-hydrate search hits, drop stale ones and self-heal the index#165loevgaard wants to merge 1 commit into
loevgaard wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## issue-138-ordered-searchable-attributes #165 +/- ##
=============================================================================
- Coverage 48.90% 48.82% -0.09%
- Complexity 781 796 +15
=============================================================================
Files 139 139
Lines 2521 2552 +31
=============================================================================
+ Hits 1233 1246 +13
- Misses 1288 1306 +18 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
6b4d659 to
ce7643b
Compare
a55c4a7 to
68b8ed4
Compare
ce7643b to
32be954
Compare
SearchAction re-hydrated every Meilisearch hit one find() at a time (up to hits_per_page=60 extra queries per page), silently dropped hits whose entity was gone (so the count and rendered cards could disagree), and rendered products disabled after their last indexing. - Group hit ids by entity class and load each group with a single findBy(['id' => ]), then reorder to Meilisearch's ranking. - Drop hits whose entity is missing from the database or is disabled (ToggleableInterface), and dispatch RemoveEntity for them so the index self-heals (failure-safe: a dispatch error is logged, never breaks rendering). - The reorder/drop logic is extracted into the pure static reorderAndFilter() for straightforward unit testing. Closes #120
32be954 to
df023f3
Compare
68b8ed4 to
3d030e8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
SearchActionre-hydrated every Meilisearch hit from the database onefind()at a time — with the defaulthits_per_pageof 60, up to 60 extra queries per search page, partly negating Meilisearch's speed. On top of that a hit whose entity no longer existed was silently dropped (so the "N results" count and the rendered cards could disagree, and the page could even render empty while claiming results), andfind()didn't re-checkenabled, so a product disabled after its last indexing was still displayed.entityClassand each group is loaded with a singlefindBy(['id' => $ids]), then reordered to match Meilisearch's ranking. 60 queries become one per entity class.ToggleableInterface), is not rendered and aRemoveEntityis dispatched for it so the index heals on the next request. The dispatch is failure-safe — an error is logged, never breaks rendering.reorderAndFilter()for straightforward unit testing.Testing
SearchActionTest: entities are reordered to match the hit order; a hit whose entity is missing is dropped and reported stale; a disabled entity is dropped and reported stale.SearchSortingTest/SearchPaginationTest(which render the search page) still pass;lint:containeris clean.Closes #120
Stacked on #164 (#138).