Tuning SVMs
1. Tuning SVMs
Up until now, when building SVMs, we have mostly used default values for cost and other parameters. Sure, we have varied the cost manually when building linear SVMs in Chapter 2, but we have not seen how to systematically choose the best value for the cost.2. Objective of tuning
This issue becomes even more critical for complex kernels with multiple parameters as it becomes difficult to find the best combination of parameters manually. In this chapter we'll see how to arrive at an optimal set of parameters for an SVM model using the tune-dot-svm() function.3. Tuning in a nutshell
The basic idea is simple. The first step is to decide on a search range for each parameter. This is typically specified as a sequence of values. For example, for a polynomial kernel, we might specify that the cost varies from 0.1 to 1000 in multiples of 10 and gamma and coef0 might take on values of 0-point-1, 1, or 10. For each possible combination of parameters, tune-dot-svm() builds an SVM model and evaluates its accuracy. The parameter combination that results in the highest accuracy is returned as the optimal set of tuning parameters. Depending on the number of parameter combinations, this procedure can become quite computationally intensive.4. Introducing tune.svm()
Let's how this works in practice. We'll use the radially separable dataset we created in the first lesson of this chapter. As you may recall, we built a polynomial kernel SVM for this model in the previous lesson and got a decent accuracy of a little less than 94%. Let's see if we can do better by tuning the cost, gamma and coef0 parameters.The arguments of tune-dot-svm() are as follows: x is the training data excluding the class column and y is the class column. The type, kernel and degree are the same as before, and the values of cost, gamma and coef0 are specified as ranges as discussed in the previous slide. Depending on the ranges you specify, tune-dot-svm() can take a while to run. When it's done, we can view the best values of the parameters as shown. That done, let's see how good these parameters are.5. Build and examine optimal model
Although tune-dot-svm() returns the optimal model, let's build it explicitly using the parameter values tune-dot-svm() and calculate the training and test accuracies. We get a test accuracy of 97%, an approximately 3% improvement on the default parameter accuracy that we obtained in the previous lesson. This is not bad. One can repeat the calculation for different training/test partitions to check that the improvement is indeed robust. I won't do that here, but if you don't want to take my word for it, you can try it out using the dataset you built in the first exercise of this chapter. That done, let's take a look at the plot of the optimal SVM against the training data. To do that we use the usual plot-dot-svm() function.6. Plot of optimal quadratic SVM
This model looks really good. If you compare it carefully with the default parameter quadratic model that we obtained in the last lesson, you will see that a few of the points that were on the wrong side of the boundary have now shifted to the correct decision regions. And that's an excellent note to close this chapter on. In the next chapter, we'll look at even more complex decision boundaries and introduce a general purpose kernel that you may end up using quite often in your own work.7. Time to practice!
But before that, let's do a few exercises.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.