Jacobian prerequisite knowledge

Hello everyone.

So, in these next few videos, I'm going to be talking about something called the Jacobian and more specifically, it's the Jacobian matrix or sometimes the associated determinant. Here, I just want to talk about some of the background knowledge that I'm assuming, because to understand the Jacobian, you do have to have a little bit of a background in linear algebra.

In particular, I want to make sure that everyone here understands how to think about matrices as transformations of space. When I say transformations here, let me just get kind of a matrix on here. I'll call it 2, 1, and negative 3, 1. You'll see why I'm coloring it like this in just a moment.

When I say how to think about this as a transformation of space, I mean you can multiply a matrix by some kind of two-dimensional vector, some kind of two-dimensional x-y, and this is going to give us a new two-dimensional vector. This is going to bring us to, let's see, in this case, it'll be [Music].

I'll write kind of 2, 1, negative 3, 1, where what it gives us is 2x plus negative 3 times y and then 1x plus 1 times y. Right? This is a new two-dimensional vector somewhere else in space, and even if you know how to compute it, there's still room for a deeper geometric understanding of what it actually means to take a vector x, y to the vector 2x plus negative 3y and 1x plus 1y.

There's also still a deeper understanding in what we mean when we call this a linear transformation. A linear transformation. So, what I'm going to do is just show you what this particular transformation looks like on the left here, where every single point on this blue grid, I'm going to tell the computer, “Hey, if that point was x, y, I want you to take it to 2x plus negative 3y, 1x plus 1y.”

And here's what it looks like. So let me just kind of play it out here. All of the points in space move, and you end up in some final state here. There are a couple important things to note.

First of all, all of the grid lines remain parallel and evenly spaced and they're still lines; they didn't get curved in some way. And that's very, very special. That is the geometric way that you can think about this term, this idea of a linear transformation. I kind of like to think about it: the lines stay lines.

In particular, the grid lines here, the ones that started off as, you know, kind of vertical and horizontal, they still remain parallel and they still remain evenly spaced. The other thing to notice here is I have these two vectors highlighted, the green vector and the red vector. These are the ones that started off; if we kind of back things up, these are the ones that started off as the basis vectors, right?

Let me kind of make a little bit more room here. The green vector is (1, 0) — one in the x-direction, zero in the y-direction — and then that red vertical vector here is (0, 1), right? So, zero, 1. If we notice where they land under this transformation, when the matrix is multiplied by every single vector in space, the place where the green vector lands, the one that started off as (1, 0), has coordinates (2, 1), and that corresponds very directly with the fact that the first column of our matrix is (2, 1).

Then, similarly, over here, the second vector, the one that started off at (0, 1), ends up at the coordinates (-3, 1), and that's what corresponds with the fact that the next column is (-3, 1). It's actually relatively simple to see why that's going to be true here.

I'll go ahead and multiply this matrix that we had, that was, see, now it's kind of easy to remember what the matrix is, right? I can just kind of read it off here as (2, 1), (-3, 1). But just to see why it's actually taking the basis vectors to the columns like this, when you do the multiplication by (1, 0), notice how it's going to take us: 2 times 1; that'll be 2. Negative 3 times 0; so that'll just be zero.

And over here, it's one times one, so that's one, and then one times zero, so again we're adding zero. So the only terms that actually mattered because of this zero down here was everything in that first column. Similarly, if we take that same matrix (2, 1), (-3, 1) and we multiply it by (0, 1), over here by the second basis vector, what you're going to get is 2 times 0, so 0, plus that element in that second column, and then 1 times 0, so another 0, plus 1 times 1; plus that 1.

So again, it's kind of like that 0 knocks out all of the terms in other columns. And then, like I said, geometrically, the meaning of a linear transformation is the grid lines remain parallel and evenly spaced.

When you start to think about it a little bit, if you can know where this green vector lands and where this red vector lands, that's going to lock into place where the entire grid has to go. Let me show you what I mean and how this corresponds with maybe a different definition that you've heard for what linear transformation means.

If we have some kind of function L and it's going to take in a vector and spit out a vector, it's said to be linear if it satisfies the property that when you take a constant times a vector, what it produces is that same constant times whatever would have happened if you applied that transformation to the vector not scaled.

Right? So here, you're applying the transformation to a scaled vector, and evidently, that's the same as scaling the transformation of the vector. Similarly, the second property of linearity is that if you add two vectors, it doesn't really matter if you add them before or after the transformation.

If you take the sum of the vectors, then apply the transformation, that's the same as first applying the transformation to each one separately and then adding up the results. One of the most important consequences of this, this formal definition of linearity, is that it means if you take your function and apply it to some vector (x, y), well, I can split up that vector as x times the first basis factor (x times (1, 0)) plus y times that second basis vector (0, 1).

Because of these two properties of linearity, if I can split it up like this, it doesn't matter if I do the scaling and adding before the transformation or if I do that scaling and adding after the transformation, and say that it's x times whatever the transformed version of (1, 0) is.

I'll show you geometrically what this means in just a moment, but I kind of want to get all the algebra on the screen, plus y times the transformed version of (0, 1). So, to be concrete, let's actually put in a value for x and y here and try to think about that specific vector geometrically.

So maybe I'll put in something like the vector (2, 1). So if we look over on the grid, we're going to be focusing on the point that's over here at (2, 1), and this particular point, and I'm going to play the transformation, and I want you to follow this point to see where it lands.

It's going to end up over here, okay? So in terms of the old grid, right, the original one that we started with, it's now at the point (1, 3). This is where we've ended up, but importantly, I want you to notice how it's still two times that green vector plus 1 times that red vector.

So it's satisfying that property that it's still x times whatever the transformed version of that first basis vector is plus y times the transformed version of that second basis vector. So that's all just a little overview, and the upshot, the main thing I want you to remember from all of this is when you have some kind of matrix, you can think of it as a transformation of space that keeps grid lines parallel and evenly spaced.

That's a very special kind of transformation. That is a very restrictive property to have on a function from 2D points to other 2D points. The convenient way to encode that is that the landing spot for that first basis vector, the one that started off one unit to the right, is represented with the first column of the matrix.

And the landing spot for the second basis vector, the one that was pointing one unit up, is encoded with that second column. If this feels totally unfamiliar or you want to learn more about this, it's something that I've made other videos on in the past.

But in terms of understanding the Jacobian matrix, where we're going with this and kind of getting a geometric feel for it, that short overview that I gave should be enough to get us going. So, with that, I will see you next video.