1. Linear decision boundaries
In this video, we'll discuss what it means for a classifier to be linear.
2. Linear decision boundaries
A decision boundary tells us what class our classifier will predict for any value of x. In this image, the classifier predicts
the blue class in the blue shaded area, where feature 2 is small, and the red
class in the red shaded area, where feature 2 is large. The dividing line between
the two regions is called the decision boundary.
This decision boundary is considered linear because it
looks like a line. The line doesn't have to be horizontal; it could be in any orientation. This definition extends to more than 2 features as well. With 5 features, the space of possible x-values is 5-dimensional, which is hard for me to draw on a slide! In that case, the boundary would be a higher-dimensional "hyperplane" cutting the space into two halves.
A nonlinear boundary is any other type of boundary. Sometimes this leads to non-contiguous regions of a certain prediction, like in the figure.
In their basic forms, logistic regression and SVMs are linear
classifiers, which means they learn linear decision boundaries. In Chapter 4 we'll discuss nonlinear versions that produce boundaries like the one on the right.
3. Definitions
Here's a list of some important vocabulary we'll be using in the course. Classification is supervised learning when the y-values are categories; this is in contrast with regression, where we're trying to predict a continuous value. We've defined decision boundaries and linear classifiers on the previous slide.
A data set is called linearly separable if it can be perfectly explained by a linear classifier.
4. Linearly separable data
Here's an example of a data set that is linearly separable and one that is not. In the left figure, there's no single line that separates the red and blue examples. Note that for now we're just talking about binary, or 2-class, classification. Later in the course we'll discuss multi-class classification (with 3 or more categories).
On the other hand, in the right-hand figure we could divide the two classes with a straight line, so it's called linearly separable.
5. Let's practice!
So if logistic regression, and linear SVMs are both linear classifiers, how are they different? That's what Chapter 2 is all about.
Let's look at some more examples of linear and nonlinear decision boundaries.