Add annotation similar to @JsonUnwrapped on document fields

[AlcipPopa]

This is a feature request.

I'm migrating a platform from Apache Solr to ElasticSearch and I'm struggling in creating dynamic fields like Apache Solr used to have. I need facets (aka "aggregations" in ElasticSearch) on these dynamic fields. On Solr that was easily done with an annotation on a Map:

@Dynamic("*_str")
@Field("attributes")
private Map<String, String> attributes;

Instead, with ElasticSearch, I need to create a dynamic template for the index like this:

{
    "mappings": {
        "dynamic_templates": [
            {
                "_str": {
                    "match": "*_str",
                    "match_mapping_type": "string",
                    "mapping": {
                        "type": "keyword"
                    }
                }
            }
        ]
    }
}

The main issue is that when I now save a document having

@Field("attributes")
private Map<String, String> attributes;

, Spring is inputting a payload like

{
  // other document fields above
  "attributes": {
    "code1_str":"value1",
    "code2_str":"other-value"
  },
  // other document fields below
}

and I can't do any aggregation query to get a result like

{
    "aggregations": {
        "code1_str": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "value1",
                    "doc_count": 1
                },
                {
                    "key": "value2",
                    "doc_count": 1
                }
            ]
        },
        "code2_str": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "other-value",
                    "doc_count": 2
                }
            ]
        }
    }
}

The workaround for this issue is atm a custom Spring Converter<S, T> for the entire document: immagine having to convert 20 or 30 fields.. this would take a lot of work just because of 1 field serialization/deserialization issue.
A solution could be found Instead in having something like @JsonUnwrapped annotation from Jackson library to remove the attributes level from the payload, leading to Spring inputting a payload like:

{
  // other document fields above
  "code1_str":"value1",
  "code2_str":"other-value"
  // other document fields below
}

Could this be feasible in an upcoming release?

[sothawo]

When pulling out the properties from a map into the entity that contains the map, how should this work when reading entities from Elasticsearch?

This works when writing data, but on reading it is not possible to determine which of the fields returned from Elasticsearch should go into properties and which into the map, there is no general way to determine where code1_str should be put into. What if the entity has two maps as properties? In which one should this be read?

What would happen on writing, when an entity has a property prop1, and the map that should be unwrapped as well has a key prop1? Which one should be written and which one overwritten?

[AlcipPopa]

Hmm yes you're right: the issue is on serialization from JSON to POJO.
Would a wildcard ease the mapping of fields between POJOs and ElasticSearch documents? This would also require a validation of the POJO field names before saving data to ElasticSearch, to avoid having multiple matching fields.

So this should be possible to save on ElasticSearch:

{
  // other document fields above
  "code1_str":"value1",
  "code2_str":"other-value"
  // other document fields below
}

While this should throw a RuntimeException before even saving into ElasticSearch, because of ambiguous mapping:

P.S.: Don't mind the @dynamic annotation since it's Solr related.

[sothawo]

We'd need to use some new annotation to prevent a mappping to be written for these fields. That idea probably will work.

I'm not yet sure if this can be done only for primitive types in the map like Map<String, String> or if this can work for something like Map<String,E> , might be possible.

[AlcipPopa]

I wouldn't use a generic value for the map.. at least not for a 1st version :)

[youssef3wi]

You can use the @DynamicTemplates annotation can generate the mapping definition you need.

After that the converter should handle the de/serialization process between the Elasticsearch request/response and the Document (Java object) to manage this definition.