TPC Improvement (Additional of #1635) (#1650)

Remaining work on PR #1635
* TPC Improvement
  * Add TPC v5.0 for SDSP converter v3.18 (#1640)
  * Change get_target_platform_capabilities API in tutorials. (#1637, #1638)
  * Modify links in README for TPC (#1641)
    * Since there was a link to a previously unpublished page, we temporarily used a relative path in #1622. It has now been reverted to the official link.
* Fix for missing parameter specifications in MCTWrapper. (#1642)
* Change implementation of wrapper for TPC Improvement.
  * Maintenance Tutorials of MCTWrapper (#1639)
  * Update and Maintenance API Documents of MCTWrapper (#1644)

Author: ueno-hiroshi002

docs/api/api_docs/classes/Wrapper.html

@@ -51,8 +51,8 @@ <h3>Navigation</h3>
 <dt class="field-even">Parameters<span class="colon">:</span></dt>
 <dd class="field-even"><ul class="simple">
 <li><p><strong>n_epochs</strong> (<em>int</em>) – Number of epochs for running the representative dataset for fine-tuning.</p></li>
-<li><p><strong>optimizer</strong> (<em>OptimizerV2</em>) – Keras optimizer to use for fine-tuning for auxiliry variable with a default learning rate set to 0.2.</p></li>
-<li><p><strong>optimizer_rest</strong> (<em>OptimizerV2</em>) – Keras optimizer to use for fine-tuning of the bias variable.</p></li>
+<li><p><strong>optimizer</strong> (<em>OptimizerV2</em>) – Keras optimizer to use for fine-tuning for auxiliary variable. Default: Adam(learning rate set to 3e-2).</p></li>
+<li><p><strong>optimizer_rest</strong> (<em>OptimizerV2</em>) – Keras optimizer to use for fine-tuning of the bias variable. Default: Adam(learning rate set to 1e-4).</p></li>
 <li><p><strong>loss</strong> (<em>Callable</em>) – loss to use during fine-tuning. should accept 4 lists of tensors. 1st list of quantized tensors, the 2nd list is the float tensors, the 3rd is a list of quantized weights and the 4th is a list of float weights.</p></li>
 <li><p><strong>log_function</strong> (<em>Callable</em>) – Function to log information about the gptq process.</p></li>
 <li><p><strong>use_hessian_based_weights</strong> (<em>bool</em>) – Whether to use Hessian-based weights for weighted average loss.</p></li>

docs/api/api_docs/methods/get_keras_gptq_config.html

@@ -51,8 +51,8 @@ <h3>Navigation</h3>
 <dt class="field-even">Parameters<span class="colon">:</span></dt>
 <dd class="field-even"><ul class="simple">
 <li><p><strong>n_epochs</strong> (<em>int</em>) – Number of epochs for running the representative dataset for fine-tuning.</p></li>
-<li><p><strong>optimizer</strong> (<em>Optimizer</em>) – Pytorch optimizer to use for fine-tuning for auxiliry variable.</p></li>
-<li><p><strong>optimizer_rest</strong> (<em>Optimizer</em>) – Pytorch optimizer to use for fine-tuning of the bias variable.</p></li>
+<li><p><strong>optimizer</strong> (<em>Optimizer</em>) – Pytorch optimizer to use for fine-tuning for auxiliary variable. Default: Adam(learning rate set to 3e-2).</p></li>
+<li><p><strong>optimizer_rest</strong> (<em>Optimizer</em>) – Pytorch optimizer to use for fine-tuning of the bias variable. Default: Adam(learning rate set to 1e-4).</p></li>
 <li><p><strong>loss</strong> (<em>Callable</em>) – loss to use during fine-tuning. See the default loss function for the exact interface.</p></li>
 <li><p><strong>log_function</strong> (<em>Callable</em>) – Function to log information about the gptq process.</p></li>
 <li><p><strong>use_hessian_based_weights</strong> (<em>bool</em>) – Whether to use Hessian-based weights for weighted average loss.</p></li>

docs/api/api_docs/methods/get_pytroch_gptq_config.html

@@ -75,8 +75,8 @@ def get_keras_gptq_config(n_epochs: int,
 
         args:
             n_epochs (int): Number of epochs for running the representative dataset for fine-tuning.
-            optimizer (OptimizerV2): Keras optimizer to use for fine-tuning for auxiliry variable with a default learning rate set to 0.2.
-            optimizer_rest (OptimizerV2): Keras optimizer to use for fine-tuning of the bias variable.
+            optimizer (OptimizerV2): Keras optimizer to use for fine-tuning for auxiliary variable. Default: Adam(learning rate set to 3e-2).
+            optimizer_rest (OptimizerV2): Keras optimizer to use for fine-tuning of the bias variable. Default: Adam(learning rate set to 1e-4).
             loss (Callable): loss to use during fine-tuning. should accept 4 lists of tensors. 1st list of quantized tensors, the 2nd list is the float tensors, the 3rd is a list of quantized weights and the 4th is a list of float weights.
             log_function (Callable): Function to log information about the gptq process.
             use_hessian_based_weights (bool): Whether to use Hessian-based weights for weighted average loss.

docs/searchindex.js

@@ -69,8 +69,8 @@ def get_pytorch_gptq_config(n_epochs: int,
 
         args:
             n_epochs (int): Number of epochs for running the representative dataset for fine-tuning.
-            optimizer (Optimizer): Pytorch optimizer to use for fine-tuning for auxiliry variable.
-            optimizer_rest (Optimizer): Pytorch optimizer to use for fine-tuning of the bias variable.
+            optimizer (Optimizer): Pytorch optimizer to use for fine-tuning for auxiliary variable. Default: Adam(learning rate set to 3e-2).
+            optimizer_rest (Optimizer): Pytorch optimizer to use for fine-tuning of the bias variable. Default: Adam(learning rate set to 1e-4).
             loss (Callable): loss to use during fine-tuning. See the default loss function for the exact interface.
             log_function (Callable): Function to log information about the gptq process.
             use_hessian_based_weights (bool): Whether to use Hessian-based weights for weighted average loss.

model_compression_toolkit/gptq/keras/quantization_facade.py

@@ -13,7 +13,7 @@ in some operator for its weights/activations, fusing patterns, etc.)
 ## Supported Target Platform Models 
 
 Currently, MCT contains three target-platform models
-(new models can be created and used by users as demonstrated [here](./tpc_models/imx500_tpc/v1_0/tpc.py)):
+(new models can be created and used by users as demonstrated [here](https://github.com/SonySemiconductorSolutions/mct-model-optimization/blob/main/model_compression_toolkit/target_platform_capabilities/tpc_models/imx500_tpc/v1_0/tpc.py)):
 - [IMX500](https://developer.sony.com/develop/imx500/)
 - [TFLite](https://www.tensorflow.org/lite/performance/quantization_spec)
 - [QNNPACK](https://github.com/pytorch/QNNPACK)
@@ -29,7 +29,7 @@ One may view the full default target-platform model and its parameters [here](./
 
 The simplest way to initiate a TPC and use it in MCT is by using the function [get_target_platform_capabilities](https://sonysemiconductorsolutions.github.io/mct-model-optimization/api/api_docs/methods/get_target_platform_capabilities.html#ug-get-target-platform-capabilities).
 
-This function gets a TPC object matching the tpc version and device type. Please check [here](../../README.md#supported-versions) for supported versions.
+This function gets a TPC object matching the tpc version and device type. Please check [here](https://github.com/SonySemiconductorSolutions/mct-model-optimization/blob/main/README.md#supported-versions) for supported versions.
 
 For example:
 
@@ -45,7 +45,7 @@ quantized_model, quantization_info = mct.ptq.keras_post_training_quantization(in
                                                                               target_platform_capabilities=tpc)
 ```
 
-You can also get a TPC for IMX500 using the function [get_target_platform_capabilities_sdsp](../../docs/api/api_docs/methods/get_target_platform_capabilities_sdsp.html#ug-get-target-platform-capabilities_sdsp) that specifies the sdsp converter version. Please check [here](../../README.md#supported-versions) for supported versions.
+You can also get a TPC for IMX500 using the function [get_target_platform_capabilities_sdsp](https://sonysemiconductorsolutions.github.io/mct-model-optimization/api/api_docs/methods/get_target_platform_capabilities_sdsp.html#ug-get-target-platform-capabilities_sdsp) that specifies the sdsp converter version. Please check [here](https://github.com/SonySemiconductorSolutions/mct-model-optimization/blob/main/README.md#supported-versions) for supported versions.
 
 For example:
 

model_compression_toolkit/gptq/pytorch/quantization_facade.py

@@ -31,6 +31,7 @@
 SDSP_V3_14 = '3.14'
 SDSP_V3_16 = '3.16'
 SDSP_V3_17 = '3.17'
+SDSP_V3_18 = '3.18'
 
 # TP Attributes
 KERNEL_ATTR = "kernel_attr"

model_compression_toolkit/target_platform_capabilities/README.md

@@ -13,7 +13,7 @@
 # limitations under the License.
 # ==============================================================================
 from model_compression_toolkit.target_platform_capabilities.constants import IMX500_TP_MODEL, TPC_V1_0, TPC_V4_0, TPC_V5_0, \
-    SDSP_V3_14, SDSP_V3_16, SDSP_V3_17
+    SDSP_V3_14, SDSP_V3_16, SDSP_V3_17, SDSP_V3_18
 from model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema import TargetPlatformCapabilities
 from model_compression_toolkit.target_platform_capabilities.tpc_models import generate_tpc_func
 
@@ -56,6 +56,7 @@ def get_target_platform_capabilities_sdsp(sdsp_version: str = SDSP_V3_14) -> Tar
         SDSP_V3_14: TPC_V1_0,
         SDSP_V3_16: TPC_V4_0,
         SDSP_V3_17: TPC_V5_0,
+        SDSP_V3_18: TPC_V5_0,
     }
 
     msg = (f"Error: The specified sdsp converter version '{sdsp_version}' is not valid. "

model_compression_toolkit/target_platform_capabilities/constants.py

@@ -17,9 +17,19 @@
 FW_NAME = 'fw_name'
 SDSP_VERSION = 'sdsp_version'
 
+# QuantizationConfig parameters
+ACTIVATION_ERROR_METHOD = 'activation_error_method'
+WEIGHTS_BIAS_CORRECTION = 'weights_bias_correction'
+Z_THRESHOLD = 'z_threshold'
+LINEAR_COLLAPSING = 'linear_collapsing'
+RESIDUAL_COLLAPSING = 'residual_collapsing'
+
 # MixedPrecisionQuantizationConfig parameters
+DISTANCE_WEIGHTING_METHOD = 'distance_weighting_method'
 NUM_OF_IMAGES = 'num_of_images'
 USE_HESSIAN_BASED_SCORES = 'use_hessian_based_scores'
+
+# ResourceUtilization parameters
 WEIGHTS_COMPRESSION_RATIO = 'weights_compression_ratio'
 
 # Resource utilization data parameters
@@ -32,14 +42,6 @@
 TARGET_RESOURCE_UTILIZATION = 'target_resource_utilization'
 IN_MODULE = 'in_module'
 
-# QuantizationConfig parameters
-ACTIVATION_ERROR_METHOD = 'activation_error_method'
-WEIGHTS_ERROR_METHOD = 'weights_error_method'
-WEIGHTS_BIAS_CORRECTION = 'weights_bias_correction'
-Z_THRESHOLD = 'z_threshold'
-LINEAR_COLLAPSING = 'linear_collapsing'
-RESIDUAL_COLLAPSING = 'residual_collapsing'
-
 # GPTQ specific parameters
 GPTQ_CONFIG = 'gptq_config'
 MODEL = 'model'
@@ -48,7 +50,12 @@
 N_EPOCHS = 'n_epochs'
 OPTIMIZER = 'optimizer'
 
-# Export parameters
+# low_bit_quantizer_ptq
 CONVERTER_VER = 'converter_ver'
 LEARNING_RATE = 'learning_rate'
+
+# Export parameters
 SAVE_MODEL_PATH = 'save_model_path'
+
+# default compression ratio
+DEFAULT_COMPRESSION_RATIO = 0.75