#2168 - proposal replace meilisearch elasticsearch with seal

[MrHDOLEK]

Resolves: #xxx

Change Log

---

Added

Fixed

Changed

Removed

Deprecated

Security

[codecov]

Codecov Report

❌ Patch coverage is 93.18182% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.85%. Comparing base (b64f56b) to head (97a4184).
⚠️ Report is 7 commits behind head on 1.x.

Additional details and impacted files

@@             Coverage Diff              @@
##                1.x    #2408      +/-   ##
============================================
- Coverage     84.87%   84.85%   -0.02%     
+ Complexity    20582    20524      -58     
============================================
  Files          1567     1561       -6     
  Lines         63194    63036     -158     
============================================
- Hits          53633    53487     -146     
+ Misses         9561     9549      -12     

Components	Coverage Δ
etl	89.34% <ø> (-0.01%)	⬇️
cli	89.40% <ø> (ø)
lib-array-dot	81.44% <ø> (ø)
lib-azure-sdk	64.44% <ø> (ø)
lib-doctrine-dbal-bulk	93.61% <ø> (ø)
lib-filesystem	85.03% <ø> (ø)
lib-types	91.98% <ø> (ø)
lib-parquet	68.89% <ø> (ø)
lib-parquet-viewer	82.26% <ø> (ø)
lib-snappy	89.82% <ø> (ø)
lib-dremel	0.00% <ø> (ø)
lib-postgresql	88.44% <ø> (ø)
lib-telemetry	84.31% <ø> (ø)
bridge-filesystem-async-aws	92.74% <ø> (ø)
bridge-filesystem-azure	90.45% <ø> (ø)
bridge-monolog-http	97.86% <ø> (ø)
bridge-monolog-telemetry	94.11% <ø> (ø)
bridge-openapi-specification	92.07% <ø> (ø)
symfony-http-foundation	78.57% <ø> (ø)
bridge-psr18-telemetry	100.00% <ø> (ø)
bridge-psr3-telemetry	97.84% <ø> (ø)
bridge-psr7-telemetry	100.00% <ø> (ø)
bridge-telemetry-otlp	90.50% <ø> (ø)
bridge-symfony-http-foundation-telemetry	89.47% <ø> (ø)
bridge-symfony-filesystem-bundle	91.54% <ø> (ø)
bridge-symfony-filesystem-cache	98.14% <ø> (ø)
bridge-symfony-postgresql-bundle	94.66% <ø> (ø)
bridge-symfony-postgresql-cache	94.41% <ø> (ø)
bridge-symfony-postgresql-messenger	98.80% <ø> (ø)
bridge-symfony-postgresql-session	93.65% <ø> (ø)
bridge-symfony-telemetry-bundle	76.00% <ø> (ø)
adapter-chartjs	84.05% <ø> (ø)
adapter-csv	91.16% <ø> (ø)
adapter-doctrine	90.79% <ø> (ø)
adapter-google-sheet	99.18% <ø> (ø)
adapter-http	73.04% <ø> (ø)
adapter-json	88.63% <ø> (ø)
adapter-logger	50.00% <ø> (ø)
adapter-parquet	81.75% <ø> (ø)
adapter-text	74.13% <ø> (ø)
adapter-xml	83.40% <ø> (ø)
adapter-avro	0.00% <ø> (ø)
adapter-excel	94.21% <ø> (ø)
adapter-postgresql	90.44% <ø> (ø)
adapter-seal	93.18% <93.18%> (∅)
bridge-phpunit-postgresql	75.30% <ø> (ø)
bridge-phpunit-telemetry	80.09% <ø> (ø)
bridge-phpstan-types	0.00% <ø> (ø)
bridge-postgresql-valinor	100.00% <ø> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[norberttech]

@MrHDOLEK as we discussed offline, please find some feedback and guidance below.

Extractor / Loader

In general I like the idea of having one extractor that accepts Seal EngineInterface which is aligned with how Doctrine Adapter works right now. Same goes for the loaders.

It would also work very well with frameworks as seal provides a bundle for symfony that literally configures engine so to use it with flow it would be a simple matter of dependency injection.

What to test

I had to think about it a bit more but I dont think we should retest every backend that seal is testing. Doctrine adapter is testing MySQL, PostgreSql nad Sqlite because flow-php/docrtine-dbal-bulk which extends doctrine default behaviors that needs to be tested.

At this time seal provides following adapters:

• Elasticsearch - cmsig/seal-elasticsearch-adapter
• Opensearch - cmsig/seal-opensearch-adapter
• Meilisearch - cmsig/seal-meilisearch-adapter
• Algolia - cmsig/seal-algolia-adapter
• Solr - cmsig/seal-solr-adapter
• RediSearch - cmsig/seal-redisearch-adapter
• Typesense - cmsig/seal-typesense-adapter
• Loupe - cmsig/seal-loupe-adapter

I would chose one of them (that requires the least configuration and is the easiest to setup in docker) and cover it with proper integration tests that would confirm that moving data from flow to seal works as expected.
Heck you can even use MemoryAdapter in tests unless other adapters have some features that requires some custom configuration.
The only thing you might want to research first is are there any differences between seal adapters. Flow needs to be flexible enough to use all of them, so if one requires something custom, flow extractor/loader needs to be able to pass it through (but I dont think thats the case).

How to test

So in general we want to cover in tests that seal adapter can handle all types of flow Entries. You can achieve that by using one of the two extractors:

• FakeStaticOrdersExtractor
• FakeRandomOrdersExtractor

They both expose static method schema() : Schema but here is where the tricky part starts. I believe you need to be able to first create an index in tests.
And that brings us to a missing piece of this PR - SchemaConverter.

SchemaConverter

Schema converters are recursive algorithms that can convert schema in both directiosn Flow to Seal and Seal to Flow. You can find some inspirations here:

• Parquet
• Doctrine
• PostgreSql

It might be the easiest to use LLM to help you create one based on those 3 examples for Seal (it's a recursive brain damaging exercise that might not be worth spending time on).

Of course schema converters would need a DSL method.

Search Engine in Tests

So in this PR you are using traits to configure backends in tests, which is fine but there is a different pattern which I found cleaner and easier to maintain.

Contexts.

Here is a good example of DatabaseContext that is used in DatabaseTableListCommandTest

If there will be more than one integration tests you can extract an abstract SealTestCase extends FlowTestCase and setup SealContext in the setUp method making it available through sealContext() : SealContext method.

In case of any questions, you know where to find me 😁