Merge pull request #259 from commercetools/3-assets

3 assets

Author: heshamMassoud

.gitignore

@@ -6,3 +6,6 @@
 # gradle specific
 .gradle/
 build/
+
+# Benchmarks
+benchmarks.json

docs/RELEASE_NOTES.md

@@ -33,6 +33,7 @@
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
 
+- [v1.0.0-M11 -  Mar 08, 2018](#v100-m11----mar-08-2018)
 - [v1.0.0-M10 -  Feb 13, 2018](#v100-m10----feb-13-2018)
 - [v1.0.0-M9 -  Jan 22, 2018](#v100-m9----jan-22-2018)
 - [v1.0.0-M8 -  Dec 29, 2017](#v100-m8----dec-29-2017)
@@ -48,33 +49,55 @@
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
 <!--
-### v1.0.0-M11 -  Mar 01, 2018
+### v1.0.0-M12 -  Mar 30, 2018
+[Commits](https://github.com/commercetools/commercetools-sync-java/compare/v1.0.0-M11...v1.0.0-M12) |
+[Javadoc](https://commercetools.github.io/commercetools-sync-java/v/v1.0.0-M12/) | 
+[Jar](https://bintray.com/commercetools/maven/commercetools-sync-java/v1.0.0-M12)
+-->
+
+### v1.0.0-M11 -  Mar 08, 2018
 [Commits](https://github.com/commercetools/commercetools-sync-java/compare/v1.0.0-M10...v1.0.0-M11) |
 [Javadoc](https://commercetools.github.io/commercetools-sync-java/v/v1.0.0-M11/) | 
 [Jar](https://bintray.com/commercetools/maven/commercetools-sync-java/v1.0.0-M11)
 
-**New Features** (4)
+**New Features** (19)
+- **Category Sync** - Support of categories' asset syncing. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - Support of product variants' asset syncing. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Category Sync** - CategorySyncUtils#buildActions now also calculates all asset update actions needed. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - ProductSyncUtils#buildActions now also calculates variants' all asset update actions needed. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - ProductUpdateActionUtils#buildVariantsUpdateActions now also calculates variants' all asset update actions needed. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - Introduced the new ActionGroup: ASSETS which can be used in blacklisting/whitelisting assets syncing during the product sync. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Category Sync** - Introduced new update action build utility for building all neeeded update actions between two categories' asstes ProductVariantUpdateActionUtils#buildProductVariantAssetsUpdateActions. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - Introduced new update action build utility for building all neeeded update actions between two variants' asstes ProductVariantUpdateActionUtils#buildProductVariantAssetsUpdateActions. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Category Sync** - Introduced new update action granular build utils for category asset fields in CategoryAssetUpdateActionUtils. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - Introduced new update action granular build utils for product variant assets fields in ProductVariantAssetUpdateActionUtils. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
 - **Commons** - Introduced `AssetReferenceResolver` which is a helper that can resolve all the references of an AssetDraft. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
 - **Commons** - `VariantReferenceResolver` and `CategoryReferenceResolver` now also resolve all the containing AssetDrafts references. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
 - **Commons** - Support for custom update actions calculation for secondary resources (e.g. prices, product assets and category assets). [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - ProductReferenceReplacementUtils#replaceProductsReferenceIdsWithKeys and VariantReferenceReplacementUtils#replaceVariantsReferenceIdsWithKeys now support replacing asset custom reference ids with keys. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Category Sync** - CategoryReferenceReplacementUtils#replaceCategoriesReferenceIdsWithKeys now supports replacing asset custom reference ids with keys. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Commons** - Introduced new SyncUtils#replaceAssetsReferenceIdsWithKeys which is a util that replaces the custom type ids with keys in a list of assets. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - ProductReferenceReplacementUtils#buildProductQuery now expands custom types on assets. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Category Sync** - CategoryReferenceReplacementUtils#buildCategoryQuery now expands custom types on assets. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
 - **Commons** - Introduced new ResourceIdentifierUtils#toResourceIdentifierIfNotNull. [#262](https://github.com/commercetools/commercetools-sync-java/issues/262) 
 
-**Changes** (3)
+**Changes** (5)
 - **Commons** - `CustomUpdateActionUtils#buildCustomUpdateActions` is now 
 `CustomUpdateActionUtils#buildPrimaryResourceCustomUpdateActions`. It now takes a new third parameter `customActionBuilder` 
 which represents the concrete builder of custom update actions. For a list of concrete builder options check the 
 implementors of the `GenericCustomActionBuilder` interface. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
 - **Commons** - `CustomUpdateActionUtils#buildCustomUpdateActions` can now be used to build custom update actions
 for secondary resources (e.g. assets and prices). [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
 - **Commons** - New Custom Type Id is now validated against being empty/null. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Product Sync** - ProductSyncUtils#buildCoreActions is now removed. ProductSyncUtils#buildActions should be used instead. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
+- **Category Sync** - CategorySyncUtils#buildCoreActions is now removed. CategorySyncUtils#buildActions should be used instead. [#3](https://github.com/commercetools/commercetools-sync-java/issues/3)
 
 **Enhancements** (1)
-- **Build Tools** - Bumped commercetools-jvm-sdk to version [1.29.0](http://commercetools.github.io/commercetools-jvm-sdk/apidocs/io/sphere/sdk/meta/ReleaseNotes.html#v1_29_0). [#262](https://github.com/commercetools/commercetools-sync-java/issues/262)
+- **Build Tools** - Bumped commercetools-jvm-sdk to version [1.30.0](http://commercetools.github.io/commercetools-jvm-sdk/apidocs/io/sphere/sdk/meta/ReleaseNotes.html#v1_30_0). [#262](https://github.com/commercetools/commercetools-sync-java/issues/262)
 
 **Bug Fixes** (1)
 - **Build Tools** - Fixed bug where jar and Codecov were triggered on benchmark stages of the build when they should 
 only be triggered on the full build. [#249](https://github.com/commercetools/commercetools-sync-java/issues/249)
--->
 
 
 ### v1.0.0-M10 -  Feb 13, 2018

docs/usage/CATEGORY_SYNC.md

@@ -143,5 +143,4 @@ More examples of those utils for different fields can be found [here](/src/integ
 ## Caveats
 
 1. Categories are either created or updated. Currently the tool does not support category deletion.
-2. The library doesn't sync category assets yet [#3](https://github.com/commercetools/commercetools-sync-java/issues/3), but it will not delete them.
-3. The library will sync all field types of custom fields, except `ReferenceType`. [#87](https://github.com/commercetools/commercetools-sync-java/issues/3). 
+2. The library will sync all field types of custom fields, except `ReferenceType`. [#87](https://github.com/commercetools/commercetools-sync-java/issues/3). 

docs/usage/PRODUCT_SYNC.md

@@ -159,7 +159,6 @@ More examples of those utils for different fields can be found [here](/src/integ
 
 ## Caveats
 1. Products are either created or updated. Currently the tool does not support product deletion.
-2. The library doesn't sync product variant assets yet [#3](https://github.com/commercetools/commercetools-sync-java/issues/3), but it will not delete them.
-3. The library will not sync attribute field types with `ReferenceType` and `SetType` field definitions, except 
+2. The library will not sync attribute field types with `ReferenceType` and `SetType` field definitions, except 
 for Product references. (See more: [#87](https://github.com/commercetools/commercetools-sync-java/issues/87) [#160](https://github.com/commercetools/commercetools-sync-java/issues/87))
 

src/integration-test/java/com/commercetools/sync/integration/commons/utils/CategoryITUtils.java

@@ -2,8 +2,6 @@
 
 import com.commercetools.sync.categories.CategorySync;
 import com.commercetools.sync.categories.helpers.CategorySyncStatistics;
-import com.fasterxml.jackson.databind.JsonNode;
-import com.fasterxml.jackson.databind.node.JsonNodeFactory;
 import io.sphere.sdk.categories.Category;
 import io.sphere.sdk.categories.CategoryDraft;
 import io.sphere.sdk.categories.CategoryDraftBuilder;
@@ -16,17 +14,9 @@
 import io.sphere.sdk.models.Reference;
 import io.sphere.sdk.products.CategoryOrderHints;
 import io.sphere.sdk.queries.QueryExecutionUtils;
-import io.sphere.sdk.queries.QueryPredicate;
-import io.sphere.sdk.types.BooleanFieldType;
 import io.sphere.sdk.types.CustomFieldsDraft;
-import io.sphere.sdk.types.FieldDefinition;
-import io.sphere.sdk.types.LocalizedStringFieldType;
 import io.sphere.sdk.types.ResourceTypeIdsSetBuilder;
 import io.sphere.sdk.types.Type;
-import io.sphere.sdk.types.TypeDraft;
-import io.sphere.sdk.types.TypeDraftBuilder;
-import io.sphere.sdk.types.commands.TypeCreateCommand;
-import io.sphere.sdk.types.queries.TypeQueryBuilder;
 
 import javax.annotation.Nonnull;
 import javax.annotation.Nullable;
@@ -38,22 +28,21 @@
 import java.util.Locale;
 import java.util.Map;
 import java.util.Objects;
-import java.util.Optional;
 import java.util.Set;
 import java.util.concurrent.CompletableFuture;
 import java.util.concurrent.CompletionStage;
 import java.util.stream.Collectors;
 
+import static com.commercetools.sync.integration.commons.utils.ITUtils.createCustomFieldsJsonMap;
+import static com.commercetools.sync.integration.commons.utils.ITUtils.createTypeIfNotAlreadyExisting;
 import static com.commercetools.sync.commons.utils.ResourceIdentifierUtils.toResourceIdentifierIfNotNull;
+
 import static io.sphere.sdk.utils.CompletableFutureUtils.listOfFuturesToFutureOfList;
 import static java.lang.String.format;
-import static java.util.Arrays.asList;
 
 public final class CategoryITUtils {
     public static final String OLD_CATEGORY_CUSTOM_TYPE_KEY = "oldCategoryCustomTypeKey";
     public static final String OLD_CATEGORY_CUSTOM_TYPE_NAME = "old_type_name";
-    public static final String LOCALISED_STRING_CUSTOM_FIELD_NAME = "backgroundColor";
-    public static final String BOOLEAN_CUSTOM_FIELD_NAME = "invisibleInShop";
 
     /**
      * Builds a list of the supplied number ({@code numberOfCategories}) of CategoryDraft objects that can be used for
@@ -145,7 +134,7 @@ public static List<Category> createChildren(final int numberOfChildren,
                     LocalizedString.of(Locale.ENGLISH, categoryName))
                 .key(categoryName)
                 .parent(toResourceIdentifierIfNotNull(parent))
-                .custom(CustomFieldsDraft.ofTypeKeyAndJson(OLD_CATEGORY_CUSTOM_TYPE_KEY, getCustomFieldsJsons()))
+                .custom(CustomFieldsDraft.ofTypeKeyAndJson(OLD_CATEGORY_CUSTOM_TYPE_KEY, createCustomFieldsJsonMap()))
                 .orderHint("sameOrderHint")
                 .build();
             final CompletableFuture<Category> future = ctpClient
@@ -171,24 +160,7 @@ public static List<Category> createChildren(final int numberOfChildren,
      * @return a dummy instance of {@link CustomFieldsDraft} with some hardcoded custom fields and key.
      */
     public static CustomFieldsDraft getCustomFieldsDraft() {
-        return CustomFieldsDraft.ofTypeKeyAndJson(OLD_CATEGORY_CUSTOM_TYPE_KEY, getCustomFieldsJsons());
-    }
-
-    /**
-     * Builds a {@link Map} for the custom fields to their {@link JsonNode} values that looks as follows in JSON
-     * format:
-     *
-     * <p>"fields": {"invisibleInShop": false, "backgroundColor": { "en": "red", "de": "rot"}}
-     *
-     * @return a Map of the custom fields to their JSON values with dummy data.
-     */
-    public static Map<String, JsonNode> getCustomFieldsJsons() {
-        final Map<String, JsonNode> customFieldsJsons = new HashMap<>();
-        customFieldsJsons.put(BOOLEAN_CUSTOM_FIELD_NAME, JsonNodeFactory.instance.booleanNode(false));
-        customFieldsJsons
-            .put(LOCALISED_STRING_CUSTOM_FIELD_NAME, JsonNodeFactory.instance.objectNode()
-                                                                             .put("de", "rot").put("en", "red"));
-        return customFieldsJsons;
+        return CustomFieldsDraft.ofTypeKeyAndJson(OLD_CATEGORY_CUSTOM_TYPE_KEY, createCustomFieldsJsonMap());
     }
 
     /**
@@ -218,74 +190,18 @@ public static List<Category> createCategories(@Nonnull final SphereClient ctpCli
     /**
      * This method blocks to create a category custom Type on the CTP project defined by the supplied
      * {@code ctpClient}, with the supplied data.
-     *
-     * @param typeKey   the type key
+     *  @param typeKey   the type key
      * @param locale    the locale to be used for specifying the type name and field definitions names.
      * @param name      the name of the custom type.
      * @param ctpClient defines the CTP project to create the type on.
      */
-    public static void createCategoriesCustomType(@Nonnull final String typeKey,
+    public static Type createCategoriesCustomType(@Nonnull final String typeKey,
                                                   @Nonnull final Locale locale,
                                                   @Nonnull final String name,
                                                   @Nonnull final SphereClient ctpClient) {
-        if (!typeExists(typeKey, ctpClient)) {
-            final TypeDraft typeDraft = TypeDraftBuilder
-                .of(typeKey, LocalizedString.of(locale, name), ResourceTypeIdsSetBuilder.of().addCategories())
-                .fieldDefinitions(buildCategoryCustomTypeFieldDefinitions(locale))
-                .build();
-            ctpClient.execute(TypeCreateCommand.of(typeDraft)).toCompletableFuture().join();
-        }
-    }
-
-    private static boolean typeExists(@Nonnull final String typeKey, @Nonnull final SphereClient ctpClient) {
-        final Optional<Type> typeOptional = ctpClient
-            .execute(TypeQueryBuilder.of().predicates(QueryPredicate.of(format("key=\"%s\"", typeKey))).build())
-            .toCompletableFuture()
-            .join().head();
-        return typeOptional.isPresent();
-    }
-
-    /**
-     * Builds a list of two field definitions; one for a {@link LocalizedStringFieldType} and one for a
-     * {@link BooleanFieldType}. The JSON of the created field definition list looks as follows:
-     *
-     * <p>"fieldDefinitions": [
-     * {
-     * "name": "backgroundColor",
-     * "label": {
-     * "en": "backgroundColor"
-     * },
-     * "required": false,
-     * "type": {
-     * "name": "LocalizedString"
-     * },
-     * "inputHint": "SingleLine"
-     * },
-     * {
-     * "name": "invisibleInShop",
-     * "label": {
-     * "en": "invisibleInShop"
-     * },
-     * "required": false,
-     * "type": {
-     * "name": "Boolean"
-     * },
-     * "inputHint": "SingleLine"
-     * }
-     * ]
-     *
-     * @param locale defines the locale for which the field definition names are going to be bound to.
-     * @return the list of field definitions.
-     */
-    private static List<FieldDefinition> buildCategoryCustomTypeFieldDefinitions(@Nonnull final Locale locale) {
-        return asList(
-            FieldDefinition
-                .of(LocalizedStringFieldType.of(), LOCALISED_STRING_CUSTOM_FIELD_NAME,
-                    LocalizedString.of(locale, LOCALISED_STRING_CUSTOM_FIELD_NAME), false),
-            FieldDefinition
-                .of(BooleanFieldType.of(), BOOLEAN_CUSTOM_FIELD_NAME,
-                    LocalizedString.of(locale, BOOLEAN_CUSTOM_FIELD_NAME), false));
 
+        return createTypeIfNotAlreadyExisting(typeKey, locale, name, ResourceTypeIdsSetBuilder.of().addCategories(),
+            ctpClient);
     }