1. Learn
    2. /
  2. Courses
    3. /
  3. Efficient AI Model Training with PyTorch
Connected

Exercise

Training loops before and after Accelerator

You want to modify a PyTorch training loop to use Accelerator for your language model to simplify translations using the MPRC dataset of sentence paraphrases. Update the training loop to prepare your model for distributed training.

Some data has been pre-loaded:

  • accelerator is an instance of Accelerator
  • train_dataloader, optimizer, model, and lr_scheduler have been defined and prepared with Accelerator

Instructions

100 XP
  • Update the .to(device) lines so that Accelerator handles device placement.
  • Modify the gradient computation to use Accelerator.