[Question] How to update an Elastic Search index doc over multiple entities

[bertoost]

Hi there,

I have multiple entities which will be persisted to Doctrine separately from each other.
At the very end of each entity I want to update de index in elastic search with data from that entity, but now it removes the previous index keys which I want to keep. I just want to update the elastic search index with new/additional data instead of replacing the index.

For example; this is my fos_elastica setup;

indexes:
    customers:
      index_name: customers
      persistence:
        driver: orm
        model: App\Entity\Customer
        identifier: number
        provider: ~
        finder: ~
      properties:
        initials:
          property_path: voorletters
        prefix:
          property_path: voorvoegsels
        lastName:
          property_path: achternaam
        street:
          property_path: straat
        houseNr:
          property_path: huisnr
        zipcode:
          property_path: postcode
        city:
          property_path: plaats
        active:
          property_path: isActive

    customer_packages:
      index_name: customers
      persistence:
        driver: orm
        model: App\Entity\Package
        identifier: customer_number
        provider: ~
        finder: ~
      properties:
        packages:
          property_path: allNumbers

I could not find any option for this? Or did I miss something?

Kind regards,
Bert

[pluseg]

@bertoost There is no built-in SYNC mechanism. You need to do it manually by:

• writing a console command to select what you want to sync to ES
• injecting FOS\ElasticaBundle\Persister\ObjectPersister into the command and sync data to ES via insertMany/updateMany.

Alternatively, you can think about building new index periodically (f.e. once per day) and switching index_alias from previous version of index to another. It depends on what task you are solving.

Keep in mind that you need to process data removal manually too. Usually, it's easier to create a subscriber:

#[AsDoctrineListener(Events::preRemove)]
#[AsDoctrineListener(Events::postPersist)]
#[AsDoctrineListener(Events::postRemove)]
#[AsDoctrineListener(Events::postUpdate)]
#[AsDoctrineListener(Events::postFlush)]
class ElasticSearchSubscriber { ... }

Which will track changes in entities and process them correspondingly (insert/update/delete).