1. Eigenvalues and Eigenvectors
In this chapter, we will focus on one of the more important tools in all of linear algebra, which are eigenvalues and eigenvectors.
2. Motivation - Face Recognition
Eigenvalues and eigenvectors are an important aspect of the field of computerized face recognition, as eigenvectors take collections of objects that can be high dimensional and select a subset of vectors that closely approximates the collection.
For example, even though an image can be created from very complex datasets, carefully-chosen subsets of digitized facial information can yield an approximation that is good enough, allowing us to get the gist of the image with less storage cost.
3. Matrix-Vector Multiplication
As we've learned in the previous two chapters, matrix multiplication can have multiple different, complex, effects on vectors.
Geometrically, these can be things like rotations, taking vectors and rotating them via an angle, or reflections, taking vectors and reflecting them about lines like the x or y-axis. Dilations or contractions, growing or shrinking of vectors. Projections, extracting certain components from a vector. Or any permutation of any number of these operations.
In any event, most matrices transform vectors into other vectors in a complicated way.
Eigenvalue/eigenvector analyses allow us to decompose these complex operations into the sum of much simpler ones.
4. Scalar Multiplication
Scalar multiplication is multiplication of a vector or a matrix by a scalar, or an element that is of the same type as those that make up the collection of matrices or vectors of interest.
The scalar c times the vector x is denoted by c times x.
which simply multiplies each element of x by c.
5. Scalar Multiplication
Scalar multiplication, as we've discussed earlier, uses the normal multiplication symbol in R. Here we multiply the vector x equal to 3, 2, 3 by the scalar c = 4 to get the vector 12, 8, 12.
6. Scalar Multiplication Achieved Through Matrix Multiplication
If a matrix has zeros everywhere except for the main diagonal, and this main diagonal consists entirely of the same number, then multiplication by this matrix has the same effect as scalar multiplication. In other words, c times I times x is the same thing as c times x, even though they are two different types of multiplication.
7. Scalar Multiplication Achieved Through Matrix Multiplication
We can confirm the truism from the previous slide through an example. Notice that, if we take the 3-by-3 identity matrix and multiply it by 3, we get the matrix A here, with 3s down the main diagonal. Multiplying this matrix by x has yields the vector 9, 6, 9, which is the same vector you'd get if you simply multiplying x by 3 in the first place.
Our goal is to be able to decompose a matrix into a few matrices whose individual effect is the same as scalar multiplication. This is the essence of eigenvalue/eigenvector analyses.
8. Let's practice!
Time to explore some of these ideas in the exercises.