1. 학습
    2. /
  2. 강의
    3. /
  3. Efficient AI Model Training with PyTorch
Connected

연습 문제

8-bit Adam with Accelerator

You would like to customize your training loop with 8-bit Adam to reduce memory requirements of your model. Prepare the loop with 8-bit Adam for training.

Assume that an 8-bit Adam optimizer has been defined as adam_bnb_optim. Other training objects have been defined: model, train_dataloader, lr_scheduler, and accelerator.

지침

100 XP
  • Prepare the 8-bit Adam optimizer for distributed training.
  • Update the model parameters with the optimizer.
  • Zero the gradients with the optimizer.