Saddle points

In the last video, I talked about how if you're trying to maximize or minimize a multivariable function, you can imagine its graph. In this case, this is just a two-variable function, and we're looking at its graph. You want to find the spots where the tangent plane is completely flat.

So, one way to visualize this is to imagine a flat plane that just represents a constant Z value, so a constant output value for the function. If you kind of move it up and down, you're looking for the spot where it only barely intersects with the graph at the top before it's not intersecting it anymore. This means that there's no values of the function that get above that point. You're looking just for where it's tangent, where you can find a tangent plane that's flat.

But this will give you some other points, like the little local minima here, the bumps where the value of the function at that point is higher than all of the neighbor points. You know, if you walk in any direction, you're going downhill. So that's another thing you're going to, you know, incidentally pick up by looking for places where this tangent plane is flat.

There's also a really interesting new possibility that comes up in the context of multivariable functions, and this is the idea of a saddle point. So, let me pull up another graph here. This guy and the function that you're looking at here—I'll write it down—the function that you're looking at is f of XY is equal to x^2 minus y^2.

Okay, so now let's think about what the tangent plane at the origin of this entire graph would be. Now, the tangent plane to this graph at the origin is actually flat. Here's what it looks like. To convince yourself of this, let's go ahead and actually compute the partial derivatives of this function and evaluate each one at the origin.

So, the partial derivative with respect to X—we look here, and x^2 is the only spot where an X shows up—so it's 2X. And the other partial derivative, the partial with respect to Y, we take the derivative of this negative y^2, and we ignore the X because it looks like a constant as far as Y is concerned, and we get -2y.

Now, if we plug in the point, the origin, into any one of these, you know, we plug in the point XY is equal to 0, 0, then what do each of these go to? Well, the top one, x equals 0, so this guy goes to zero. Similarly, over here, Y is zero, so this goes to zero. So, both partial derivatives are zero, and what that means is if you are standing at the origin and you move in any direction, there's no slope to your movement.

One way of seeing this is to chop the graph. If we imagine chopping it with a plane that represents a constant x value and we kind of chop off the graph there, what you'll see here—I’ll get rid of the tangent plane—what you'll see is that the curve where this intersects the graph, so let me trace that out, the curve where it intersects the graph basically has a local maximum at that origin point. The tangent line of the curve at that point in the y-direction is flat, and it's because it looks like a local maximum from that perspective.

But now let's imagine chopping it in a different direction. So, if instead we have the full graph and, instead of chopping it with a constant x value, we chop it with a constant y value, then in that case we look at the curve. If we trace out the curve where this constant y value intersects the graph, let's see what it would look like. It's also kind of this parabolic shape; again, the tangent line is flat because it looks like a local minimum of that curve.

So, because it's flat in one direction and it's flat in the other direction, the tangent plane of the graph as a whole is indeed going to be flat. But notice this is neither a local maximum nor a local minimum because from one direction it looked like it was a local maximum; here, I'll get rid of that guy. It looks like a local maximum when you look on the curve there, but from another direction, if you chop it in another way, it looks like a local minimum.

If we look at the equations, this kind of makes sense because if you're just thinking about movements in the X direction, the entire function looks like x^2 plus some kind of constant. So, the graph of that would look like an x^2 parabola shape that has a local minimum. But if you're thinking of pure movements in the y direction and you're just focused on that negative y^2 component, the graph that you get for -y^2 is going to look like an upside-down parabola.

Here, I'll draw that again—it’s going to look like an upside-down parabola, and that's got a local maximum. So, it's kind of like the X and Y directions disagree over whether this point, whether this point where you have a flat tangent plane should be a local maximum or a local minimum.

This is new to multivariable calculus. This is something that doesn't come up in single-variable calculus because when you're looking at the graph of a function, you know, you're looking at some kind of graph. If the tangent line is zero, you know, if the tangent line is completely flat at some point, either it's a local maximum or it's a local minimum; it can't disagree because there's only one input variable. There's only one, you know, X as the input variable for your graph.

But once we have two, it's possible that they disagree. This kind of point has a special name, and the name is kind of after this graph that you're looking at: it's called a saddle point. Saddle point.

This is one of those rare times where I actually kind of like the terminology that mathematicians have given something because this looks like a saddle, the sort of thing that you would put on a horseback before riding it.

So, one thing that this means for us is we're going to try to figure out ways to find the absolute maximum or the minimum of a function as we're trying to optimize a function that might represent, you know, represent, like, the profits of your company or a cost function in a machine-learning setting or something like that.

We're going to have to be able to recognize if a point is a saddle point, and if you're just looking at the graph, it's fine; you can recognize it visually. But oftentimes, if you're just given the formula of a function, it's some long thing without looking at the graph. How would you be able to tell just by doing certain computations to the formula whether or not it's a saddle point?

That comes down to something called the second partial derivative test, which I will talk about in the next few videos. See you then!