Matrix-Vector Equations - Some Theory

1. Matrix-Vector Equations - Some Theory

Before we can use matrix-vector equations to create ratings and use those ratings as features in our machine learning models, for example, we have to know under what conditions does a matrix-vector equation a) has a solution and b) has a unique solution. This will require some theory.

2. A Matrix-Vector Equation Without a Solution

Here we have a simple example of a matrix-vector equation that cannot have a solution. The first row of the matrix multiplied by the unknown vector can equal -1 in an infinite number of ways. However, 0 can never equal 2, so this system is what is called inconsistent.

3. A Matrix-Vector Equation with Infinitely-Many Solutions

On the other hand, if the right-hand side of the equation is changed (even slightly), the same matrix can yield a matrix-vector equation that has infinitely many solutions. Matrix-vector equations that have at least one solution are called consistent. While this is an improvement over having no solutions, many of the problems we want to solve in the real world (for example the rating of athletic teams or websites) are such that one and only one solution is the desirable outcome.

4. A Matrix-Vector Equation with a Unique Solution

Changing one element of the matrix further yields a matrix-vector equation with exactly one solution, as x1, in this case, must equal zero, forcing x2 to equal 1.

5. Properties of Solutions to Matrix-Vector Equations - Exactly One Solution

When a matrix-vector equation has a solution, but not infinitely-many it must only have one solution. This is a fundamental result in linear algebra, and opens the door for the solution to many real-world problems, and can be visualized by the idea of two lines in the x-y plane intersecting once - unless they are the same line, in which case they intersect infinitely many times.

6. Properties of Solutions to Matrix-Vector Equations - No Solutions

When there are no solutions, you can think of the two equations representing two parallel lines (line-ar algebra) in the x-y plane. For more equations (i.e. bigger matrices) the same idea holds - planes that don't mutually intersect.

7. Properties of Solutions to Matrix-Vector Equations - Infinitely-Many Solutions

For the infinitely-many solutions case, the idea is that the equations represent parallel lines that are the same line - with their intersection including infinitely-many points in the x-y plane. For more equations and bigger matrices, the idea again holds true, with planes such that their intersections are things like lines or other planes, which consist of infinitely many points.

8. Properties to Ensure A Unique Solution to $A\vec{x} = \vec{b}$

There are many ways that we can determine if a matrix-vector equation has a unique solution. And all of these conditions are equivalent. For an n by n matrix, i.e. a system of equations with as many equations as unknowns, here are some of the highlights. First, the matrix A must have an inverse. We discussed inverses briefly in the last chapter, but essentially they are the matrix that "undoes" multiplication by A. Another condition is that the determinant of A, which is a crude measure of the size of the matrix, is nonzero. This number can be easily computed in R. A basis of a vector space is a set of vectors that can create all of the other vectors in the space via linear combinations, and this creation is unique. We need the columns of A to be a basis for the vector space of all n-dimensional vectors, again to have a unique solution to Ax equals b in this case.

9. Properties to Ensure A Unique Solution to $A\vec{x} = \vec{b}$

It's easy in R to determine if a matrix is invertible or not, using the "solve()" function. If a matrix is returned, then the matrix is invertible. If not, an error shows. The same thing is true about the determinant of a matrix. "det()" is the function here.

10. Let's practice!

Let's explore these ideas in the exercises!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.