1. Learn
    2. /
  2. Kurser
    3. /
  3. Efficient AI Model Training with PyTorch
Connected

övning

Compute the optimizer size

You're exploring different optimizers for training a model, and you need to quantify an optimizer's memory usage for an objective comparison. As a test, you've loaded a DistilBERT model and AdamW optimizer so that you quantify memory usage. Write the compute_optimizer_size function to compute the size of an optimizer.

The AdamW optimizer has been defined directly (without Trainer), and training has completed.

Instruktioner

100 XP
  • Compute number of elements and size of each tensor in the for loop.
  • Compute the total size of the optimizer in megabytes.
  • Access the optimizer state dictionary using the appropriate method on optimizer.state.