In b06bfc2 we monkey patch reindexObjectSecurity to make sure the relevant security indexes get updated in Solr as well.
This is done by adding REINDEX operations with attributes=self._cmf_security_indexes to the collective.indexing queue for the affected objects (recursively, just like the original method does).
While collective.indexing provides the concept of the indexing queue, the actual processing of the queue happens in collective.solr (by implementing a IIndexQueueProcessor utility).
The index() method of that SolrIndexProcessor takes an attributes kwarg, the only use it makes of it is to check if the object happens to have no fields that are indexed in Solr (in which case it skips the operation).
The issue that Solr can't properly do partial updates has been recognized, according to this comment:
# unfortunately with current versions of solr we need to provide
# data for _all_ fields during an <add> -- partial updates aren't
# supported (see https://issues.apache.org/jira/browse/SOLR-139)
# however, the reindexing can be skipped if none of the given
# attributes match existing solr indexes...
But the way they chose to deal with it is to call all the indexers for all the fields on the object that are part of Solr's schema.
Until Solr properly supports partial updates (for both XML and JSON update messages) we need to supply all the fields in every update message, there's no way around that.
But instead of letting Plone index all the fields again, one could get them from Solr (assuming they are stored).
So this is the fix I would propose:
- Override the
SolrIndexProcessor utility in ftw.solr, inheriting from the original
- Override the
index() method on the subclass, making sure the attributes get passed through to the self.getData() call
- Override the
getData() method and change it do to the following:
- If
attributes is not None AND attributes doesn't contain all the fields in the Solr schema, get the remaining fields by querying Solr.
- If a Solr query fails (e.g. because the field isn't stored), fall back on calling the indexer in Plone for those fields
In b06bfc2 we monkey patch
reindexObjectSecurityto make sure the relevant security indexes get updated in Solr as well.This is done by adding REINDEX operations with
attributes=self._cmf_security_indexesto thecollective.indexingqueue for the affected objects (recursively, just like the original method does).While
collective.indexingprovides the concept of the indexing queue, the actual processing of the queue happens incollective.solr(by implementing aIIndexQueueProcessorutility).The
index()method of thatSolrIndexProcessortakes anattributeskwarg, the only use it makes of it is to check if the object happens to have no fields that are indexed in Solr (in which case it skips the operation).The issue that Solr can't properly do partial updates has been recognized, according to this comment:
But the way they chose to deal with it is to call all the indexers for all the fields on the object that are part of Solr's schema.
Until Solr properly supports partial updates (for both XML and JSON update messages) we need to supply all the fields in every update message, there's no way around that.
But instead of letting Plone index all the fields again, one could get them from Solr (assuming they are stored).
So this is the fix I would propose:
SolrIndexProcessorutility inftw.solr, inheriting from the originalindex()method on the subclass, making sure theattributesget passed through to theself.getData()callgetData()method and change it do to the following:attributesis not None ANDattributesdoesn't contain all the fields in the Solr schema, get the remaining fields by querying Solr.