KL divergence

There are more distance metrics that can be used to compute how similar two feature vectors are. For instance, the philentropy package has the function distance(), which implements 46 different distance metrics. For more information, use ?distance in the console.

In this exercise, you will compute the KL divergence and check if the results differ from the previous metrics. Since the KL divergence is a measure of the difference between probability distributions you need to rescale the input data by dividing each input feature by the total pixel intensities of that digit.

The philentropy package and mnist_sample data have been loaded.

Store the first 10 records of mnist_sample without the label column in an object called mnist_10.
Compute the rowSums().
Compute the KL of mnist_10 by dividing each value over the total sum per row, using the function distance() and generate a heatmap.

Exercise

KL divergence

Instructions