LOF met factoren

De functie lof() accepteert als invoer zowel een numeriek data frame als een afstandsmatrix om LOF-scores te berekenen. In deze oefening ga je een afstandsmatrix berekenen met Gowers afstand, die je daarna kunt doorgeven aan de functie lof() voor het scoren.

Net als in de vorige oefening is de thyroid-data, waarbij tekenkolommen zijn omgezet naar factoren, alvast voor je geladen.

Deze oefening maakt deel uit van de cursus

Introductie tot anomaliedetectie in R

Oefeninstructies

Bereken de Gower-afstandsmatrix voor de thyroid-data en sla het resultaat op in het nieuwe object thyroid_dist.
Gebruik thyroid_dist om een LOF voor elke patiënt te genereren met k = 10.
Print het bereik van de afstanden in de matrix thyroid_dist.

Interactieve oefening met praktijkervaring

Probeer deze oefening door deze voorbeeldcode aan te vullen.

# Calculate Gower's distance matrix
thyroid_dist <- daisy(___, metric = ___)

# Generate LOF scores for thyroid data
thyroid_lof <- lof(thyroid_dist, k = 10)

# Range of values in the distance matrix
___(as.matrix(___))

Code bewerken en uitvoeren

Deze oefening maakt deel uit van de cursus

Introductie tot anomaliedetectie in R

SkillTag.level.intermediateSkillTag.label

4.8+

Begin gratis met de cursus

In this chapter, you'll learn how numerical and graphical summaries can be used to informally assess whether data contain unusual points. You'll use a statistical procedure called Grubbs' test to check whether a point is an outlier, and learn about the Seasonal-Hybrid ESD algorithm, which can help identify outliers when the data are a time series.

Exercise 1: What do we mean when we talk about anomalies?Exercise 2: Recognizing anomaly types Exercise 3: Exploring the river nitrate data Exercise 4: Testing the extremes with Grubbs' test Exercise 5: Visual check of normality Exercise 6: Grubbs' test Exercise 7: Hunting multiple outliers using Grubbs' test Exercise 8: Anomalies in time series Exercise 9: Visual assessment of seasonality Exercise 10: Seasonal Hybrid ESD algorithm Exercise 11: Interpreting Seasonal-Hybrid ESD output Exercise 12: Seasonal-Hybrid ESD versus Grubbs' test

In this chapter, you'll learn how to calculate the k-nearest neighbors distance and the local outlier factor, which are used to construct continuous anomaly scores for each data point when the data have multiple features. You'll learn the difference between local and global anomalies and how the two algorithms can help in each case.

Exercise 1: k-nearest neighbors distance score Exercise 2: Exploring wine Exercise 3: kNN distance matrix Exercise 4: kNN distance score Exercise 5: Visualizing kNN distance Exercise 6: Standardizing features Exercise 7: Appending the kNN score Exercise 8: Visualizing kNN distance score Exercise 9: Local outlier factor Exercise 10: LOF calculation Exercise 11: LOF visualization Exercise 12: LOF vs kNN

k-nearest neighbors distance and local outlier factor use the distance or relative density of the nearest neighbors to score each point. In this chapter, you'll explore an alternative tree-based approach called an isolation forest, which is a fast and robust method of detecting anomalies that measures how easily points can be separated by randomly splitting the data into smaller and smaller regions.

Exercise 1: Isolation trees Exercise 2: Fit and predict with an isolation tree Exercise 3: Score interpretation Exercise 4: Isolation forest Exercise 5: Fit an isolation forest Exercise 6: Checking convergence Exercise 7: Visualizing the isolation score Exercise 8: A grid of points Exercise 9: Prediction over a grid Exercise 10: Anomaly contours

You've now been introduced to a few different algorithms for anomaly scoring. In this final chapter, you'll learn to compare the detection performance of the algorithms in instances where labeled anomalies are available. You'll learn to calculate and interpret the precision and recall statistics for an anomaly score, and how to adapt the algorithms so they can accommodate data with categorical features.

Exercise 1: Gelabelde anomalieën Exercise 2: Schildkliergegevens Exercise 3: Schildklieraandoening visualiseren Exercise 4: Anomaliescore Exercise 5: Prestaties meten Exercise 6: Gebinariseerde scores Exercise 7: Kruistabelle van binaire scores Exercise 8: Precisie en recall voor schildklier Exercise 9: Werken met categorische kenmerken Exercise 10: Tekst naar factor omzetten Exercise 11: Isolation forest met factoren Exercise 12: LOF met factoren

Huidige oefening

Exercise 13: Afronding