Specify the TrainingArguments

You're configuring the training process for your language model. TrainingArguments specifies input parameters for Trainer. This exercise provides values for these parameters; generally, you'll have to tune the parameters for a model. Prepare the arguments for your model to use Trainer!

Some data has been pre-loaded:

output_dir is a pre-defined directory
The TrainingArguments class has been imported

This exercise is part of the course

Efficient AI Model Training with PyTorch

View Course

Exercise instructions

Define training_args using the TrainingArguments class.
Set the learning_rate to 2e-5 to fine-tune pre-trained weights of your model.
Set the per device train batch size on each device to 16.
Set the evaluation checkpoints for every "epoch".

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Define training_args using a transformers class
training_args = ____(
    output_dir=output_dir,
    # Set the learning rate to 2e-5
    ____=____,
    # Set train batch size on each device to 16
    ____=16,
    per_device_eval_batch_size=16,
    num_train_epochs=2,
    weight_decay=0.01,
    save_strategy="epoch",
    # Set evaluation checkpoints every epoch
    ____="____",
)

Edit and Run Code