Session Ready
Exercise

Custom scorers in pipelines

You are proud of the improvement in your code quality, but just remembered that previously you had to use a custom scoring metric in order to account for the fact that false positives are costlier to your startup than false negatives. You hence want to equip your pipeline with scorers other than accuracy, including roc_auc_score(), f1_score(), and you own custom scoring function. The pipeline from the previous lesson is available as pipe, as is the parameter grid as params and the training data as X_train, y_train. You also have confusion_matrix() for the purpose of writing your own metric.

Instructions 1/3
undefined XP
  • 1

    Convert the metric roc_auc_score() into a scorer, and feed it into GridSearchCV(). Then fit that to the data.

    • 2

      Now repeat for the F1 score, instead, given by f1_score().

    • 3

      Now repeat with a custom metric which is available to you as as simple Python function called my_metric().