Using outlier probabilities
An alternative to isolating outliers with contamination is using outlier probabilities. The best thing about this method is that you can choose an arbitrary probability threshold, which means you can be as confident as you want in the predictions.
IForest and big_mart are already loaded.
This exercise is part of the course
Anomaly Detection in Python
Exercise instructions
- Calculate probabilities for both inliers and outliers.
- Extract the probabilities for outliers into
outlier_probs. - Filter the outliers into
outliersby using a 70% threshold onoutlier_probs.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
iforest = IForest(random_state=10).fit(big_mart)
# Calculate probabilities
probs = iforest.____
# Extract the probabilities for outliers
outlier_probs = ____[____]
# Filter for when the probability is higher than 70%
outliers = ____[____]
print(len(outliers))