Multivariable maxima and minima

When you have a multi-variable function, something that takes in multiple different input values, and let's say it's just outputting a single number. A very common thing you want to do with an animal like this is maximize it. Maximize it, and what this means is you're looking for the input points, the values of x and y, and all of its other inputs such that the output f is as great as it possibly can be.

Now this actually comes up all the time in practice because usually when you're dealing with a multivariable function, it's not just for fun and for dealing with abstract symbols, it's because it actually represents something. So maybe it represents like profits of a company. Maybe this is a function where you're considering all the choices you can make, like the wages you give your employees, or the prices of your goods, or the, you know, amount of debt that you raise for capital. All sorts of choices that you might make, and you want to know what values should you give to those choices such that you maximize profits. You maximize a thing.

If you have a function that models these relationships, there are techniques which I'm about to teach you that you can use to maximize this. Another very common setting, more and more important these days, is that of machine learning and artificial intelligence, where often what you do is you assign something called a cost function to a task. So maybe you're trying to teach a computer how to understand audio, or how to read handwritten text. What you do is you find a function that basically tells it how wrong it is when it makes a guess.

If you do a good job designing that function, you just need to tell the computer to minimize. So that's kind of the flip side, right? Instead of finding the maximum, to minimize a certain function. If it minimizes this cost function, that means that it's doing a really good job at whatever task you've assigned it. So a lot of the art and science of machine learning and artificial intelligence comes down to, well, one, finding this cost function and actually describing difficult tasks in terms of a function, but then applying the techniques that I'm about to teach you to have the computer minimize that.

A lot of time and research has gone into figuring out ways to basically apply these techniques but really quickly and efficiently. So first of all, on a conceptual level, let's just think about what it means to be finding the maximum of a multivariable function. So I have here the graph of a two-variable function; that's something that has a two-variable input that we're thinking of as the x-y plane, and then its output is the height of this graph.

If you're looking to maximize it, basically what you're finding is this peak, kind of the tallest mountain in the entire area, and you're looking for the input value, the point on the x-y plane directly below that peak, because that tells you the values of the inputs that you should put in to maximize your function. So how do you go about finding that? Well, this is perhaps the core observation in calculus, not just multivariable calculus. This is similar in the single-variable world, and there's similarities in other settings.

But the core observation is that if you take a tangent plane at that peak—so let's just draw in a tangent plane at that peak—it's going to be completely flat. But let's say you did this at a different point, right? Because if you tried to find the tangent plane not at that point, but you kind of moved it about a bit to somewhere that's not quite a maximum, if the tangent plane has any kind of slope to it, what that's telling you is that if you take very small directions, kind of in the direction of that upward slope, you can increase the value of your function.

So if there's any slope to the tangent plane, you know that you can walk in some direction to increase it. But if there's no slope to it, if it's flat, then that's a sign that no matter which direction you walk, you're not going to be significantly increasing the value of your function. So what does this mean in terms of formulas? Well, if you kind of think back to how we compute tangent planes, and if you're not very comfortable with that, now would be a good time to take another look at those videos about tangent planes.

The slope of the plane in each direction—so you know this would be the slope in the x direction, and then if you look at it from another perspective, this would be the slope in the y direction—each one of those has to be zero. And that, in terms of partial derivatives, means the partial derivative of your function at whatever point you're dealing with, right? So I'll call it (x₀, y₀) as the point where you're inputting this, has to be zero.

Then similarly, the partial derivative with respect to the other variable, with respect to y at that same point, has to be zero. Both of these have to be true because let's just take a look— I don't know, let's slide it over a little bit here. This tangent plane, if you look at the slope, you imagine walking in the y direction, you're not increasing your value at all. The slope in the y direction would actually be zero, so that would mean the partial derivative with respect to y would be zero.

But with respect to x, when you're moving in the x direction here, the slope is clearly negative because as you take positive steps in the x direction, the height of your tangent plane is decreasing, which corresponds to if you take tiny steps on your graph, then the height will decrease in a manner proportional to the size of those tiny steps.

So what this gives you here is going to be a system of equations where you're solving for the value of (x₀, y₀) that satisfies both of these equations. In future videos, I'll go through specific examples of this. For now, I just want to give a good conceptual understanding. But one very important thing to notice is that just because this condition is satisfied—meaning your tangent plane is flat—just because that's satisfied doesn't necessarily mean that you've found the maximum.

That's just one requirement that it has to satisfy. But for one thing, if you found the tangent plane at other little peaks, like this guy here or this guy here or all of the little bumps that go up, those tangent planes would also be flat. Those little bumps actually have a name because this comes up a lot—they're called, um, they're called local minima or local maxima, sorry.

So those guys are called local maxima. Maxima is just the plural of maximum, and local means that it's relative to a single point. So it's basically if you walk in any direction when you're on that little peak, you'll go downhill. So relative to the neighbors of that little point, it is a maximum. But relative to the entire function, you know, these guys are the shorter mountains next to Mount Everest.

But there's also another circumstance where you might find a flat tangent plane, and that's at the minimum points, right? If you have the global minimum, the absolute smallest, or also just the local minima, these inverted peaks, you'll also find flat tangent planes. So what that means, first of all, is that when you're minimizing a function, you also have to look for this requirement where all the partial derivatives are zero.

But it mainly just means that your job isn't done once you've done this. You have to do more tests to check whether or not what you found is a local maximum or a local minimum or a global maximum. These requirements, by the way, often you'll see them written in a more succinct form where instead of saying all the partial derivatives have to be zero—which is what you need to find—they'll write it in a different form where you say that the gradient of your function f, which of course is just the vector that contains all those partial derivatives, its first component is the partial derivative with respect to the first variable, its second component is the partial derivative with respect to the second variable, and if there were more variables, you would keep going.

You'd say that this whole thing has to equal the zero vector, the vector that has nothing but zeros as its components. As kind of a common abuse of notation, people will just call that zero vector zero, and maybe they'll emphasize it by making it bold because the number zero is not a vector, and often making things bold emphasizes that you want to be referring to a vector. But this gives a very succinct way of describing the requirement—you're just looking for where the gradient of your function is equal to the zero vector, and that way you can just write it on one line.

But in practice, every time that you're expanding that out, what that means is you find all of the different partial derivatives. So this is really just a matter of notational convenience and using less space on a blackboard. But whenever you see this—that the gradient equals zero—what you should be thinking of is the idea that the tangent plane is completely flat.

And as I just said, that's not enough because you might also be finding local maxima or minima points. But in multivariable calculus, there's also another possibility—a place where the tangent plane is flat but what you're looking at is neither a local maximum nor a local minimum. And this is the idea of a saddle point, which is new to multivariable calculus, and that's what I'll be talking about in the next video. So I will see you then.