AdamW with Accelerator
You want to customize your training loop using Accelerator and use AdamW as a benchmark optimizer for your language translation model. Build the training loop to use AdamW.
Some training objects have been pre-loaded and defined, including model, train_dataloader, and accelerator.
Latihan ini adalah bagian dari kursus
Efficient AI Model Training with PyTorch
Petunjuk latihan
- Prepare training objects for distributed training before the loop.
- Update the model parameters in the training loop.
Latihan interaktif praktis
Cobalah latihan ini dengan menyelesaikan kode contoh berikut.
optimizer = AdamW(params=model.parameters())
# Prepare training objects for distributed training
model, optimizer, train_dataloader = ____.____(model, optimizer, train_dataloader)
for batch in train_dataloader:
inputs, targets = batch["input_ids"], batch["labels"]
outputs = model(inputs, labels=targets)
loss = outputs.loss
accelerator.backward(loss)
# Update the model parameters
____.____()
optimizer.zero_grad()
print(f"Loss = {loss}")