1. Learn
    2. /
  2. Courses
    3. /
  3. Fine-Tuning with Llama 3
Connected

Exercise

Filtering datasets for evaluation

You are building a training and evaluation pipeline for your company's health care chatbot, which is used by hospitals to onboard new patients.

Your task is to create a pipeline to load the MedQuad-MedicalQnADataset to evaluate an LLM on its ability to answer medical questions. You are asked to load the dataset in the ds variable, and only include the first 500 samples of the train split of the dataset stored in dataset_name as your evaluation set.

Instructions

100 XP
  • Import necessary functions and classes from datasets.
  • Load the dataset in the ds variable.
  • Manipulate ds to include the first 500 samples of the train split of the dataset stored in dataset_name as your evaluation set.