Relation of null space to linear independence of columns

So I have the matrix A over here, and A has M rows and N columns. So we could call this an M by N matrix. What I want to do in this video is relate the linear independence or linear dependence of the column vectors of A to the null space of A.

First of all, what am I talking about column vectors? Well, as you can see, there are columns here, and we can view each of those as an M-dimensional vector. Let me do it this way. You could view this one right over here; we could write that as V1. This next one over here would be V2, and you would have N of these because we have N columns. So this one right over here would be VN, V sub N.

We could rewrite A; we could rewrite the matrix A, the M by N matrix A. I'm bolding it to show that that's a matrix. We could rewrite it as—let me do it the same way; I’ll draw my little brackets there. We can express it in terms of its column vectors, and we could just say, well, this is going to be V1 for that column, V2 for this column, all the way—we're going to have N columns—so you're going to have VN for the Nth column.

Remember, each of these is going to have M terms, or I should say M components in them. These are M-dimensional column vectors.

Now, what I want to do—I said I want to relate the linear independence of these vectors to the null space of A. So let's remind ourselves what the null space of A even is. The null space of A is equal to—or I could say it's equal to the set. It's the set of all vectors X that are members of R^N, and I'm going to double down on why I'm saying R^N in a second, such that if I take my matrix A and multiply it by one of those X's, I am going to get the zero vector.

So why does X have to be a member of R^N? Well, just for the matrix multiplication to work. For this to be—if this is M by N, let me write this down. If this is M by N, well, in order to just make the matrix multiplication work—or you could say the matrix-vector multiplication, this has to be an N by one vector. So it's going to have N components.

So it's going to be a member of R^N. If this was M by—let me use a different letter; if this was M by I don’t know 7, then this would be R^7 that we would be dealing with. So that is the null space.

Another way of thinking about it is, well, if I take my matrix A and I multiply it by some vector X that’s a member of this null space, I’m going to get the zero vector.

So if I take my matrix A, which I've expressed here in terms of its column vectors, multiply it by some vector X—so some vector X—and actually, let me make it clear that it doesn't have to have the same—so let me just denote vector X right over here. Draw the other bracket.

So this is the vector X, and so it’s going to have—it’s a member of R^N. So it’s going to have N components; you’re going to have X1 as the first component, X2, and go all the way to XN. If you multiply—so if we say that this X is a member of the null space of A, then this whole thing is going to be equal to the zero vector.

It's going to be equal to the zero vector, and once again the zero vector—this is going to be an M by one vector. So it’s going to look—let me write it like this. It’s going to have the same number of rows as A, so I'll try to make it the brackets roughly the same length.

There we go. Try and draw my brackets neatly. So you’re going to have M of these: one, two, and then go all the way to M0o.

So let's actually just multiply this out using what we know of matrix multiplication, and by the definition of matrix multiplication. One way to view this—if you were to multiply our matrix A times our vector X here, you are going to get the first column vector V1 times the first component here X1 plus the second component times the second column vector V2, and we’re going to do that N times.

So plus dot dot dot X sub N times V sub N, and these—all when you add them together are going to be equal to the zero vector. Now this should be—this is going to be equal to the zero vector, and now this should start ringing a bell to you.

When we looked at linear independence, we saw something like this. In fact, we saw that these vectors V1, V2—these N vectors are linearly independent if and only if any linear—if and only if the solution to this—or I guess you could say the weights on these vectors—the only way to get this to be true is if X1, X2, XN are all equal to zero.

So let me write this down. So V sub 1, V sub 2, all the way to V sub N are linearly independent if and only if the only solution—or you could say weights on these vectors to this equation—the only solution is X1, X2, all the way to XN are equal to zero.

So if the only solution here is X1, X2, and XN are equal to zero, well that means that our vectors V1, V2, all the way through to VN are linearly independent. Or vice versa—if they are linearly independent, then the only solution to this—if we’re solving for the weights on those vectors—is for X1, X2, and XN to be equal to zero.

Remember linear independence; if you want to say it—a bit more common language is—if these vectors are linearly independent, that means none of these vectors can be constructed by linear combinations of the other vectors. Looking at it this way, this right over here is a—you could view this as a linear combination of all of the vectors.

The only way to get this linear combination of all the vectors to be equal to zero is if X1, X2, all the way through XN are equal to zero, and we prove that in other videos on linear independence.

Well, if the only solution to this is all of the X1's through XNs are equal to zero, that means that the null space—this is only going to be true, you could say if and only if the null space of A—let me make sure it looks like a matrix; I'm going to bold it—the null space of A only contains one vector. It only contains the zero vector.

Remember, this is if all of these are going to be zero, well then the only solution here is going to be the zero vector.

So the result that we’re showing here is if the column vectors of a matrix are linearly independent, then the null space of that matrix is only going to consist of the zero vector. Or you could go the other way—if the null space of a matrix only contains the zero vector, well that means that the columns of that matrix are linearly independent.