Evaluating a model on test and train

The function auc_train_test calculates the AUC of model that is built on a train set and evaluated on a test set:

auc_train, auc_test = auc_train_test(variables, target, train, test)

with variables a list of the names of the variables that is used in the model.

In this exercise, you will apply this function, and check whether the train and test AUC are similar.

This exercise is part of the course

Introduction to Predictive Analytics in Python

View Course

Exercise instructions

The basetable is loaded. Partition the basetable such that the train set contains 70% of the data, and make sure that train and test set have equal target incidence.
Calculate the train and test AUC of the model using "age" and "gender_F" as predictors using the auc_train_test function.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Load the partitioning module
from sklearn.model_selection import train_test_split

# Create DataFrames with variables and target
X = basetable.drop('target', 1)
y = basetable["target"]

# Carry out 70-30 partititioning with stratification
X_train, X_test, y_train, y_test = ____(X, y, test_size = ____, stratify = ____)

# Create the final train and test basetables
train = pd.concat([X_train, y_train], axis=1)
test = pd.concat([X_test, y_test], axis=1)

 # Apply the auc_train_test function
auc_train, auc_test = ____([____, ____], ["target"], ____, ____)
print(round(auc_train,2))
print(round(auc_test,2))

Edit and Run Code