Get GradientPTQConfig for Pytorch Models

model_compression_toolkit.gptq.get_pytorch_gptq_config(n_epochs, optimizer=Adam([torch.Tensor([])], lr=LR_DEFAULT), optimizer_rest=Adam([torch.Tensor([])], lr=LR_REST_DEFAULT), loss=multiple_tensors_mse_loss, log_function=None, use_hessian_based_weights=True, regularization_factor=REG_DEFAULT, hessian_batch_size=ACT_HESSIAN_DEFAULT_BATCH_SIZE)

Create a GradientPTQConfigV2 instance for Pytorch models.

Parameters:

  • n_epochs (int) – Number of epochs for running the representative dataset for fine-tuning.

  • optimizer (Optimizer) – Pytorch optimizer to use for fine-tuning for auxiliry variable.

  • optimizer_rest (Optimizer) – Pytorch optimizer to use for fine-tuning of the bias variable.

  • loss (Callable) – loss to use during fine-tuning. should accept 4 lists of tensors. 1st list of quantized tensors, the 2nd list is the float tensors, the 3rd is a list of quantized weights and the 4th is a list of float weights.

  • log_function (Callable) – Function to log information about the gptq process.

  • use_hessian_based_weights (bool) – Whether to use Hessian-based weights for weighted average loss.

  • regularization_factor (float) – A floating point number that defines the regularization factor.

  • hessian_batch_size (int) – Batch size for Hessian computation in Hessian-based weights GPTQ.

Returns:

a GradientPTQConfigV2 object to use when fine-tuning the quantized model using gptq.

Examples

Import MCT and Create a GradientPTQConfigV2 to run for 5 epochs:

>>> import model_compression_toolkit as mct
>>> gptq_conf = mct.gptq.get_pytorch_gptq_config(n_epochs=5)

Other PyTorch optimizers can be passed with dummy params:

>>> import torch
>>> gptq_conf = mct.gptq.get_pytorch_gptq_config(n_epochs=3, optimizer=torch.optim.Adam([torch.Tensor(1)]))

The configuration can be passed to pytorch_post_training_quantization() in order to quantize a pytorch model using gptq.

Return type:

GradientPTQConfig