Specify the TrainingArguments

You're configuring the training process for your language model. TrainingArguments specifies input parameters for Trainer. This exercise provides values for these parameters; generally, you'll have to tune the parameters for a model. Prepare the arguments for your model to use Trainer!

Some data has been pre-loaded:

output_dir is a pre-defined directory
The TrainingArguments class has been imported

This exercise is part of the course

Efficient AI Model Training with PyTorch

View Course

Exercise instructions

Define training_args using the TrainingArguments class.
Set the learning_rate to 2e-5 to fine-tune pre-trained weights of your model.
Set the per device train batch size on each device to 16.
Set evaluation_strategy to create evaluation checkpoints every epoch.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Define training_args using a transformers class
training_args = TrainingArguments(
    output_dir=output_dir,
    # Set the learning rate to 2e-5
    learning_rate=____,
    # Set train batch size on each device to 16
    per_device_train_batch_size=____,
    per_device_eval_batch_size=16,
    num_train_epochs=2,
    weight_decay=0.01,
    save_strategy="epoch",
    # Set evaluation checkpoints every epoch
    evaluation_strategy=____,
)

Edit and Run Code