<div align="center">Getting Started</div>
Quick Installation
Pip install the model compression toolkit package in a Python>=3.9 environment with PyTorch>=2.1 or Tensorflow>=2.12.
pip install model-compression-toolkit
For installing the nightly version or installing from source, refer to the installation guide.
Important note: In order to use MCT, you’ll need to provide a floating point .pt or .keras model as an input.
Tutorials and Examples
Our tutorials section will walk you through the basics of the MCT tool, covering various compression techniques for both Keras and PyTorch models. Access interactive notebooks for hands-on learning with popular models/tasks or move on to Resources section.
Supported Quantization Methods</div>
MCT supports various quantization methods as appears below.
Quantization Method | Complexity | Computational Cost | API | Tutorial
-------------------- | -----------|--------------------|---------|--------
PTQ (Post Training Quantization) | Low | Low (~1-10 CPU minutes) | [PyTorch API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/pytorch_post_training_quantization.html) / [Keras API](https://sony.github.io/model_optimization/docs/api/api_docs/methods/keras_post_training_quantization.html) | 
For each flow, Quantization core utilizes various algorithms and hyper-parameters for optimal hardware-aware quantization results. For further details, please see Supported features and algorithms.
Required input:
- Floating point model - 32bit model in either .pt or .keras format
- Representative dataset - can be either provided by the user, or generated utilizing the Data Generation capability
<div align="center">High level features and techniques</div>
MCT offers a range of powerful features to optimize models for efficient edge deployment. These supported features include:
Quantization Core Features
🏆 Mixed-precision search 
. Assigning optimal quantization bit-width per layer (for weights/activations)
📈 Graph optimizations. Transforming the model to be best fitted for quantization process.
🔎 Quantization parameter search . Minimizing expected quantization-noise during thresholds search using methods such as MSE, No-Clipping and MAE.
🧮 Advanced quantization algorithms . Enhancing quantization performance for advanced cases is available with some algorithms that can be applied, such as Shift negative correction, Outliers filtering and clustering.
____________________________________
Hardware-aware optimization
🎯 TPC (Target Platform Capabilities). Describes the target hardware’s constrains, for which the model optimization is targeted. See TPC Readme for more information. ____________________________________
Data-free quantization (Data Generation)
Generates synthetic images based on the statistics stored in the model’s batch normalization layers, according to your specific needs, for when image data isn’t available. See Data Generation Library for more. ____________________________________
Structured Pruning
Reduces model size/complexity and ensures better channels utilization by removing redundant input channels from layers and reconstruction of layer weights. Read more (Pytorch API / Keras API). ____________________________________
Debugging and Visualization
🎛️ Network Editor (Modify Quantization Configurations) .
Modify your model’s quantization configuration for specific layers or apply a custom edit rule (e.g adjust layer’s bit-width) using MCT’s network editor
🖥️ Visualization. Observe useful information for troubleshooting the quantized model’s performance using TensorBoard. Read more.
🔑 XQuant (Explainable Quantization) . Get valuable insights regarding the quality and success of the quantization process of your model. The report includes histograms and similarity metrics between the original float model and the quantized model in key points of the model. The report can be visualized using TensorBoard.
____________________________________
Enhanced Post-Training Quantization (EPTQ)
As part of the GPTQ capability, we provide an advanced optimization algorithm called EPTQ. The specifications of the algorithm are detailed in the paper: “EPTQ: Enhanced Post-Training Quantization via Hessian-guided Network-wise Optimization“ [4]. More details on how to use EPTQ via MCT can be found in the GPTQ guidelines.
<div align="center">Resources</div>
-
User Guide contains detailed information about MCT and guides you from installation through optimizing models for your edge AI applications.
-
MCT’s API Docs is seperated per quantization methods:
-
Post-training quantization PTQ API docs -
Gradient-based post-training quantization GPTQ API docs -
Quantization-aware training QAT API docs
-
-
Debug – modify optimization process or generate explainable report
<div align="center">Supported Versions</div>
Currently, MCT is being tested on various Python, Pytorch and TensorFlow versions:
Supported Versions Table
| | PyTorch 2.2 | PyTorch 2.3 | PyTorch 2.4 | PyTorch 2.5 | |-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Python 3.9 | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch22.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch23.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch24.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_pytorch25.yml) | | Python 3.10 | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch22.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch23.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch24.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_pytorch25.yml) | | Python 3.11 | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch22.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch23.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch24.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_pytorch25.yml) | | Python 3.12 | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch22.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch23.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch24.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python312_pytorch25.yml) | | | TensorFlow 2.12 | TensorFlow 2.13 | TensorFlow 2.14 | TensorFlow 2.15 | |-------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Python 3.9 | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras212.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras213.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras214.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python39_keras215.yml) | | Python 3.10 | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml) | | Python 3.11 | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml) | [](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml) |<div align="center">Results</div>
### Pytorch
We quantized classification networks from the torchvision library.
In the following table we present the ImageNet validation results for these models:
| Network Name | Float Accuracy | 8Bit Accuracy | Data-Free 8Bit Accuracy |
|---------------------------|-----------------|-----------------|-------------------------|
| MobileNet V2 [3] | 71.886 | 71.444 |71.29|
| ResNet-18 [3] | 69.86 | 69.63 |69.53|
| SqueezeNet 1.1 [3] | 58.128 | 57.678 ||
### Keras
MCT can quantize an existing 32-bit floating-point model to an 8-bit fixed-point (or less) model without compromising accuracy.
Below is a graph of [MobileNetV2](https://keras.io/api/applications/mobilenet/) accuracy on ImageNet vs average bit-width of weights (X-axis), using
single-precision quantization, mixed-precision quantization, and mixed-precision quantization with GPTQ.
For more results, please see [1]
### Pruning Results
Results for applying pruning to reduce the parameters of the following models by 50%:
| Model | Dense Model Accuracy | Pruned Model Accuracy |
|-----------------|----------------------|-----------------------|
| ResNet50 [2] | 75.1 | 72.4 |
| DenseNet121 [3] | 74.44 | 71.71 |
##
##