In this chapter, you'll learn how numerical and graphical summaries can be used to informally assess whether data contain unusual points.  You'll use a statistical procedure called Grubbs' test to check whether a point is an outlier, and learn about the Seasonal-Hybrid ESD algorithm, which can help identify outliers when the data are a time series.

Cosa intendiamo quando parliamo di anomalie?

Riconoscere i tipi di anomalie

Esplorare i dati sui nitrati del fiume

Mettere alla prova gli estremi con il test di Grubbs

Verifica visiva di normalità

Test di Grubbs

Caccia a più outlier con il test di Grubbs

Anomalie nelle serie temporali

Valutazione visiva della stagionalità

Algoritmo Seasonal Hybrid ESD

Interpretare l'output di Seasonal-Hybrid ESD

Seasonal-Hybrid ESD contro il test di Grubbs

Statistical outlier detection

In this chapter, you'll learn how to calculate the  k-nearest neighbors distance and the local outlier factor, which are used to construct continuous anomaly scores for each data point when the data have multiple features.   You'll learn the difference between local and global anomalies and how the two algorithms can help in each case.

k-nearest neighbors distance score

Exploring wine

kNN distance matrix

kNN distance score

Visualizing kNN distance

Standardizing features

Appending the kNN score

Visualizing kNN distance score

Local outlier factor

LOF calculation

LOF visualization

LOF vs kNN

Distance and density based anomaly detection

k-nearest neighbors distance and local outlier factor use the distance or relative density of the nearest neighbors to score each point.  In this chapter, you'll explore an alternative tree-based approach called an isolation forest, which is a fast and robust method of detecting anomalies that measures how easily points can be separated by randomly splitting the data into smaller and smaller regions.

Isolation trees

Fit and predict with an isolation tree

Score interpretation

Isolation forest

Fit an isolation forest

Checking convergence

Visualizing the isolation score

A grid of points

Prediction over a grid

Anomaly contours

 You've now been introduced to a few different algorithms for anomaly scoring. In this final chapter, you'll learn to compare the detection performance of the algorithms in instances where labeled anomalies are available.  You'll learn to calculate and interpret the precision and recall statistics for an anomaly score, and how to adapt the algorithms so they can accommodate data with categorical features.

Labeled anomalies

Thyroid data

Visualizing thyroid disease

Anomaly score

Measuring performance

Binarized scores

Cross-tabulate binary scores

Thyroid precision and recall

Working with categorical features

Converting character to factor

Isolation forest with factors

LOF with factors

Wrap-up

Comparing performance

Furniture

Wine

Thyroid

Ti preoccupano record imprecisi o sospetti nei tuoi dati, ma non sai da dove iniziare? Un algoritmo di anomaly detection può aiutarti! L'anomaly detection è un insieme di tecniche pensate per identificare punti dati insoliti ed è fondamentale per rilevare frodi e proteggere le reti informatiche da attività dannose. In questo corso esplorerai test statistici per individuare gli outlier e imparerai a usare sofisticati algoritmi di punteggio delle anomalie come il local outlier factor e l'isolation forest. Metterai in pratica questi algoritmi per riconoscere vini insoliti nel dataset UCI Wine Quality e per rilevare casi di patologie tiroidee da misurazioni ormonali anomale.

Intermediate R

Impara i test statistici per identificare gli outlier e usa algoritmi avanzati di punteggio anomalie.

Introduzione all'Anomaly Detection in R

Impara i test statistici per trovare i valori anomali e come usare algoritmi avanzati per calcolare il punteggio delle anomalie. 

Data Scientist

Testing the extremes with Grubbs' test

Visual assessment is not always reliable!

Grubbs' test

Checking normality with a histogram

Running Grubbs' test

Interpreting the p-value

Get the row index of an outlier

Let's practice!

Statistical outlier detection - Mettere alla prova gli estremi con il test di Grubbs

Mettere alla prova gli estremi con il test di Grubbs

Create Your Free Account