Get GradientPTQConfig for Keras Models

model_compression_toolkit.gptq.get_keras_gptq_config(n_epochs, optimizer=tf.keras.optimizers.Adam(learning_rate=LR_DEFAULT), optimizer_rest=tf.keras.optimizers.Adam(learning_rate=LR_REST_DEFAULT), loss=GPTQMultipleTensorsLoss(), log_function=None, use_hessian_based_weights=True, regularization_factor=REG_DEFAULT, hessian_batch_size=ACT_HESSIAN_DEFAULT_BATCH_SIZE)

Create a GradientPTQConfigV2 instance for Keras models.

Parameters:

  • n_epochs (int) – Number of epochs for running the representative dataset for fine-tuning.

  • optimizer (OptimizerV2) – Keras optimizer to use for fine-tuning for auxiliry variable with a default learning rate set to 0.2.

  • optimizer_rest (OptimizerV2) – Keras optimizer to use for fine-tuning of the bias variable.

  • loss (Callable) – loss to use during fine-tuning. should accept 4 lists of tensors. 1st list of quantized tensors, the 2nd list is the float tensors, the 3rd is a list of quantized weights and the 4th is a list of float weights.

  • log_function (Callable) – Function to log information about the gptq process.

  • use_hessian_based_weights (bool) – Whether to use Hessian-based weights for weighted average loss.

  • regularization_factor (float) – A floating point number that defines the regularization factor.

  • hessian_batch_size (int) – Batch size for Hessian computation in Hessian-based weights GPTQ.

Returns:

a GradientPTQConfigV2 object to use when fine-tuning the quantized model using gptq.

Examples

Import MCT and TensorFlow:

>>> import model_compression_toolkit as mct
>>> import tensorflow as tf

Create a GradientPTQConfigV2 to run for 5 epochs:

>>> gptq_conf = mct.gptq.get_keras_gptq_config(n_epochs=5)

Other Tensorflow optimizers can be passed:

>>> gptq_conf = mct.gptq.get_keras_gptq_config(n_epochs=3, optimizer=tf.keras.optimizers.Nadam())

The configuration can be passed to keras_post_training_quantization() in order to quantize a keras model using gptq.

Return type:

GradientPTQConfig