Concluding remarks

1. Concluding remarks

Congratulations for completing this challenging and broad course. Let us review the incredible progress you have made.

2. Concluding remarks

You started by refreshing your knowledge of the standard supervised learning workflow, including feature engineering, model fitting and model selection, all with an eye to the ever-present danger of overfitting. You then took a step back to better understand the process by which a problem gets translated to a supervised learning task. You considered data fusion as a way to produce training examples out of disparate data sources, and understood how to make the best of noisy labels and heuristic expert knowledge. You gained a solid hands-on understanding of the critical importance of loss functions, and making sure that your choice of loss reflects the real-world costs of false positives and false negatives.

3. Concluding remarks

You then stepped outside the comfort zone of supervised learning and grappled with anomaly and novelty detection. A lack of labels will stop most data scientists - but not you! You now know from first-hand experience that it is possible to get fantastic accuracy using detectors that are trained without any actual labels, as long as you pick your distance metric carefully, even with entirely unstructured data. This course has given you machine learning superpowers that will help you stand out, and exposed you to a number of fascinating real-world problems and real-world data ranging from cybersecurity to retail banking and healthcare.

4. Congratulations!

In your next job interview, try mentioning that you now know how to extract value from data even with noisy labels, or without any labels at all; or explaining that you are familiar with agile model development - and see how this battle-tested knowledge will help you get ahead of other candidates. Best of luck, and congratulations again on this achievement!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Designing Machine Learning Workflows in Python

AdvancedSkill Level

4.8+

11 reviews

In the previous chapters you established a solid foundation in supervised learning, complete with knowledge of deploying models in production but always assumed you a labeled dataset would be available for your analysis. In this chapter, you take on the challenge of modeling data without any, or with very few, labels. This takes you into a journey into anomaly detection, a kind of unsupervised modeling, as well as distance-based learning, where beliefs about what constitutes similarity between two examples can be used in place of labels to help you achieve levels of accuracy comparable to a supervised workflow. Upon completing this chapter, you will clearly stand out from the crowd of data scientists in confidently knowing what tools to use to modify your workflow in order to overcome common real-world challenges.

Exercise 1: Anomaly detection Exercise 2: A simple outlier Exercise 3: LoF contamination Exercise 4: Novelty detection Exercise 5: A simple novelty Exercise 6: Three novelty detectors Exercise 7: Contamination revisited Exercise 8: Distance-based learning Exercise 9: Find the neighbor Exercise 10: Not all metrics agree Exercise 11: Unstructured data Exercise 12: Restricted Levenshtein Exercise 13: Bringing it all together Exercise 14: Concluding remarks

Current Exercise