DbJsonB immediately dirty on load if multiple enum as property

[AntoineDuComptoirDesPharmacies]

Hi everyone,

We are having a strange behavior with DbJsonB feature on POJO where multiple properties are Enum.
Ebean Version : 13.20.1

Expected behavior

Having an entity A with a DbJsonB property composed of a POJO B with multiple fields (each fields is an enum).
After DB/find.byId of entity A , DB.beanState should answer that the entity is not dirty.

Actual behavior

Having an entity A with a DbJsonB property composed of a POJO B with multiple fields (each fields is an enum).
After DB/find.byId of entity A , DB.beanState answer that the entity is dirty on the DbJsonB property even without change.

Steps to reproduce

@Entity
public class A{
    [...]
    @DbJsonB
    @Column(nullable = false)
    private A.B sources;
    [...]

    @JsonInclude(JsonInclude.Include.NON_NULL) // Do not encode null to "null" to gain space in DB
    static public class B {
        private DataProviderEnum field1;
        private DataProviderEnum field2;

        public B() {
           // NOTHING TO DO
        }

        @JsonCreator
        public B(
                @JsonProperty("field1") DataProviderEnum field1,
                @JsonProperty("field2") DataProviderEnum field2
                                   ) {
            this.field1= field1;
            this.field2 = field2;
        }


        public DataProviderEnum getField1() {
            return field1;
        }

        public void setField1(DataProviderEnum field1) {
            this.field1= field1;
        }

        public DataProviderEnum getField2() {
            return field2;
        }

        public void setField2(DataProviderEnum field2) {
            this.field2= field2;
        }

        @Override
        public String toString() {
            return "ProductFieldSources{" +
                    "field1=" + field1+
                    ", field2=" + field2+
                    '}';
        }
    }
}

public enum DataProviderEnum {
    @EnumValue("VALUE1")
    VALUE1,
    @EnumValue("VALUE2")
    VALUE2,
    @EnumValue("VALUE3")
    VALUE3
}

Here is the value for this property in database :

{"field1": "VALUE1", "field2": "VALUE2"}

When executing :
DB.beanState(A.find.byId(1L)).dirtyValues()
If mutationDetection = SOURCE :
=> Return the property sources with Old being null and New being the value from database.

If mutationDetection = DEFAULT :
=> Return the property sources with Old and New value being the same (value from database) but with different object reference.

Important note :

• If I replace DataProviderEnum by String type, the problem disappear
• If I delete one of the field from database, (eg. : {"field1": "VALUE1"}), then the problem disappear
• Even if the second field have no meaning with the schema, the problem stay (eg. : {"field1": "VALUE1", "any": "blue"})

Thanks in advance for your help.
Yours faithfully,
LCDP

[AntoineDuComptoirDesPharmacies]

We did further analysis on this problem and we figure out the exact cause of this wrong dirtiness detection.

The problem come from PostgreSQL not respecting order of fields while using JSONB column type because the data is stored optimized.
See : https://www.postgresql.org/docs/10/datatype-json.html

It seems that PostgreSQL is storing the keys as following :

• At first, order keys by their length
• If same length, then alphabetically

In class ScalarJsonJacksonMapper.java, call to class RsetDataReader -> reader.getString(); this get the raw BD value and so return a JSON which have a fields ordering done by PostgreSQL.
When values are compared in isEqualToObject of SourceMutableValue and ChecksumMutableValue, it will not match and be considered dirty at the value of obj is serialized using Jackson, resulting in different fields ordering.

This dirtiness will never be resolved and these bean will always be considered dirty on load because when Ebean will persist the new value (with its own ordering) in the database, PostgreSQL will reorder the fields to optimize the way it is stored.

We successfully avoid these wrong dirtiness status adding JsonPropertyOrder annotation on the class using complete and explicit field ordering following the exact order as PostgreSQL but it is near to impossible to maintain when adding/removing fields.

We were wondering if it could be interesting to modify the method isEqualToObject in a way that both side will have their fields ordered before comparing. Or maybe we could reorder the result of RsetDataReader according to the order of the JsonB POJO. Or maybe you will come up with a better solution ?

Maybe you will have an idea @rbygrave or @rPraml ?

Thanks in advance for your help.
Yours faithfully,
LCDP

[AntoineDuComptoirDesPharmacies]

Another Pull request (better performance) to solve the problem here : #3746