python (65.1k questions)

javascript (44.2k questions)

reactjs (22.7k questions)

java (20.8k questions)

c# (17.4k questions)

html (16.3k questions)

r (13.7k questions)

android (12.9k questions)

Questions - quantization-aware-training

Cannot create the calibration cache for the QAT model in tensorRT

I've trained a quantized model (with help of quantized-aware-training method in pytorch). I want to create the calibration cache to do inference in INT8 mode by TensorRT. When create calib cache, I ge...
test-img

Mahsa

python-3.x

pytorch

onnx

tensorrt

quantization-aware-training

Votes: 0

Answers: 1

Latest Answer

If the ONNX model has Q/DQ nodes in it, you may not need calibration cache because quantization parameters such as scale and zero point are included in the Q/DQ nodes. You can run the Q/DQ ONNX model ...
test-img

cacup

Quantized model gives negative accuracy after conversion from pytorch to ONNX

I'm trying to train a quantize model in pytorch and convert it to ONNX. I employ the quantized-aware-training technique with help of pytorch_quantization package. I used the below code to convert my m...
test-img

Mahsa

pytorch

onnx

quantization-aware-training

Votes: 0

Answers: 1

Latest Answer

After some tries, I found that there is a version conflict. I changed the versions accordingly: onnx == 1.9.0 onnxruntime == 1.8.1 pytorch == 1.9.0+cu111 torchvision == 0.10.0+cu111
test-img

Mahsa

How to continue Quantization Aware Training of saved model in PyTorch?

I have a DL model that is trained in two phases: Pretraining using synthetic data Finetuning using real world data Model is saved after phase 1. At phase 2 model is created and loaded from .pth file...
test-img

albert828

python

deep-learning

pytorch

quantization-aware-training

Votes: 0

Answers: 0

Posts

Questions

Blogs

Jobs