Other Considerations for Matrix-Vector Equations
1. Other Considerations for Matrix-Vector Equations
Now that we've studied the properties of n equations and n unknowns, equations that elicit matrix-vector equations with square matrices, we're going to consider some other situations that can arise when using linear algebra for data science.2. More Equations than Unknowns
When a linear system of equations has more equations than unknowns (which would occur if you're requiring data to satisfy too many requirements), one or more of the equations has to be redundant (i.e. be some combination of the other equations in the system) for a solution to exist. In this particular case, where there are three equations but just two unknowns, we might be in luck, since the third equation appears to be the sum of the first two. In data science, this would occur if a collection of the variables were correlated in some way. While the theory discussed in the previous section depend only on the square matrix in question (i.e. they are independent of b), when square matrices are no longer in play, the existence of a solution often comes down to b.3. More Equations than Unknowns
However, if the right-hand side changes even slightly, in this case from 8 to 0, the system of equations goes from consistent (i.e. has a solution) to inconsistent (i.e. does not have a solution). This is saying that the data simultaneously satisfies two conditions that are mutually exclusive, which cannot occur. As with systems that give rise to square matrices, there are only three outcomes: zero, one or infinitely many solutions.4. Fewer Equations than Unknowns
In the opposite case, where there are fewer equations than unknowns, it's often difficult to have a unique solution, since there are not enough conditions (equations) to rule out collections of solutions. For example, in this problem, where there are three unknowns but just two equations, if x_1 and x_2 can multiply by the first two vectors, respectively to create (0,1), then x_3 can be zero. Similarly for x_1, x_3 and x_2, and so on. While rare, particular systems like this can have a unique solution, or fail to have a solution if the columns of the matrix are redundant in any way and/or b is inconsistent with the combinations of these redundant columns.5. Some Options for Non-Square Matrices
While a full approach to solving non-square systems is out of the scope of this course. Some examples of approaches that people take in this setting are row reduction (this is the first thing taught in a university-level matrix theory course for mathematics, engineering and physics students. Least squares is actually the method used most often in the rating problems we've explored during this chapter, while singular value decomposition has many uses in both applied mathematics and data science. The former is extremely important in linear regression, while the latter is crucial for principal component analysis. The generalized or pseudo-inverse is an attempt to get back at least some of the properties of the matrix inverse when a matrix is not invertible. We will discuss this option in the next slide and exercises.6. Moore-Penrose Generalized Inverse
Here we use the MASS package in R to compute the Moore-Penrose general inverse for A, the first matrix in this lecture, using the "ginv" function. This command serves the purpose of "solve" in this case. Notice that its efficacy is only one-sided, producing the identity matrix only when applied to A from the left. In other words ginv(A)%*%A is the 2 by 2 identity matrix, but A%*%ginv(A) is not.7. Moore-Penrose Generalized Inverse
Let A and b be the given matrix and vector here. Notice that if the system Ax equals b has a solution, we can find at least one of them with this generalized inverse, using the command ginv(A)%*%b. It's beyond the scope of this course to show you how to find more (if there are more), but this is a good introduction. If a solution doesn't exist, this will give you an erroneous result, so it is always good to check by multiplying x by A using A%*%x, as in the previous lecture.8. Let's Practice
We're ready to explore some more questions about matrix-vector equations!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.