Quantization aware training in onnx

[Nedae]

Hello,
The input to the quantize_qat function
https://github.com/microsoft/onnxruntime/blob/0b9f7bb1b0d56d7fffccd0557525d6c03229d137/onnxruntime/python/tools/quantization/quantize.py#L248
is supposed to be quantize-aware traning onnx model, based on the function description Given a quantize-aware traning onnx model, create a quantized onnx model and save it into a file.

1. Is this quantize-aware traning onnx model a QAT model trained and fine-tuned in other platforms (e.g. pytorch), and converted to onnx?
2. Is it currently possible to convert such a qat-trained model from pytorch to onnx?
3. Is there a way to finetune a model in onnx for QAT purpose?

[snnn]

@yufenglee Can you answer it?

[yufenglee]

Yes, the QAT model is fine-tuned with either PyTorch or Tensorflow. And then use the PyTorch exporter and tf2onnx to convert the QAT model to onnx model.

We don't have plan to finetune ONNX model directly with QAT for now. @mrry could provide more on the timeline.

[Nedae]

Thanks @yufenglee!
The pytorch to onnx export for quantized models was not supported for pytorch versions before 1.7, and in the new release of pytorch it is mentioned for exporting to onnx, support for export of torch.quantization.FakeQuantize has also been added to help enable some QAT workflows.. I am hoping that is all we need to such an export!