Visualizing Linear SVMs
1. Visualizing linear SVMs
In this lesson, we'll visualize the linear decision boundary we built in the previous lesson. This will help build an intuition for what the algorithm does and also clarify terminology. Specifically, we'll learn the significance of the term "support" in support vector machines.2. Visualizing support vectors
The first step is to plot the training data, distinguishing classes using colors. This is identical to what we did in the previous chapter for the entire dataset. After this is done, we extract the support vectors from the training data using the indices from the model and visually distinguish them from other points by overlaying them with semi-transparent blobs. The transparency is controlled by the ggplot() parameter alpha.3. Visualizing support vectors
Here is the plot. Note that the support vectors are all close to the decision boundary. One could thus say that they "hold" or "support" the boundary, which is the origin of the term "support vector". This point will become clearer when we add in the decision boundary and margins.4. Slope and intercept of the decision boundary
The first step in plotting the boundary is to extract the slope and intercept from the model. This requires a bit of work as the model object does not store the slope and intercept explicitly. We first use coefs and SV elements of the svm model object to build the weight vector, w, which is the product of the coefs matrix with the matrix containing the SVs. Recall that matrix multiplication in R is represented by an asterisk between two percent symbols. The slope is given by the negative ratio of the first and second components of the weight vector, and the intercept by the ratio of the rho element of the svm object and the y-component of the weight vector. Whew! That done we can plot the decision boundary using the calculated slope and intercept.5. Visualizing the decision and margin boundaries
We add the decision boundary to our earlier plot using the slope and intercept that we calculated and stored in the variables slope underscore1 and intercept underscore1. The decision boundary is easily plotted using the geom_abline() function in ggplot. The margin boundaries are parallel to the decision boundary with intercepts offset by an amount equal to the reciprocal y-component of the weight vector. Let's take a look at the plot with the boundary and margins added in.6. Visualizing the decision and margin boundaries
The plot has some interesting features. First, the boundary is "supported" - so to speak - by roughly the same number of support vectors on either side. Second, the margin is soft, that is, wide, and there are a number of points that violate the margin.7. Soft margin classifiers
Soft margin classifiers are useful because they allow for a degree of uncertainty in the exact location and shape of the boundary, which is usually neither perfectly linear nor known in real life. In this example, however, we do know that the decision boundary is linear so we're better off reducing the number of boundary violations. We'll do this in the next lesson.8. Visualizing the decision boundary using the svm plot() function
We can also visualize the boundary using the svm plot() function in e1071. As this is a two dimensional problem, we need specify only the parameters x and data, which are the model name and dataframe to be plotted, respectively.9. Plot of decision boundary using the svm plot() function
Here's the output of svm plot() function. Although not as clear as the ggplot() outputs, it is a good "rough and ready" visualization. Distinct classes are distinguished by color and support vectors are marked x. Note that the axes have been flipped, so be careful when comparing this to the earlier plots. This completes the lesson. In the next lesson we'll learn how to tune linear SVMs.10. Time to practice!
But first, let's do some exercises.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.