Comparing quantized model performance
Understanding performance improvements isn't just about accuracy. Quantized models often offer faster inference times—a key benefit in deployment scenarios. You'll measure how long it takes for both the original and quantized models to process the test set.
The measure_time() function has been predefined. It sets the model to evaluation mode, runs a forward pass on all batches in the dataloader, and returns the elapsed time.
Both model (the original model) and model_quantized (the quantized version) are preloaded along with test_loader.
本练习是课程的一部分
Scalable AI Models with PyTorch Lightning
练习说明
- Compute inference time for the original and quantized models.
- Print both times rounded to two decimal points.
交互式实操练习
通过完成这段示例代码来试试这个练习。
# Measure inference time of the original model
original_time = measure_time(____)
# Measure inference time of the quantized model
quant_time = measure_time(____)
# Print results
print(f"Original Model Time: {____}s")
print(f"Quantized Model Time: {____}s")