1. Uczyć się
    2. /
  2. Courses
    3. /
  3. Efficient AI Model Training with PyTorch
Connected

Exercise

Gradient checkpointing with Accelerator

You're continuing to optimize memory usage so you can train your language translation model on your device. Gradient accumulation has helped you to effectively train on larger batch sizes. Build on this work to add gradient checkpointing to reduce the memory footprint of your model.

The model, train_dataloader, and accelerator have been pre-defined.

Instrukcje

100 XP
  • Enable gradient checkpointing on the model.
  • Set up an Accelerator context manager to enable gradient accumulation on the model.