yego.me
💡 Stop wasting time. Read Youtube instead of watch. Download Chrome Extension

Constrained optimization introduction


5m read
·Nov 11, 2024

Hey everyone! So, in the next couple videos, I'm going to be talking about a different sort of optimization problem: something called a constrained optimization problem.

An example of this is something where you might see — you might be asked to maximize some kind of multivariable function. And let's just say it was the function f of x, y is equal to x squared times y. But that's not all you're asked to do; you're subject to a certain constraint where you're only allowed values of x and y on a certain set. I'm just going to say the set of all values of x and y such that x squared plus y squared equals 1. This is something you might recognize as the unit circle. This particular constraint that I've put on here, this is the unit circle.

One way that you might think about a problem like this, you know, you're maximizing a certain two-variable function, is to first think of the graph of that function. That's what I have pictured here: the graph of f of x, y equals x squared times y. Now, this constraint x squared plus y squared is basically just a subset of the x, y plane. So if we look at it head-on here and we look at the x, y plane, this circle represents all of the points x, y such that this holds.

What I've actually drawn here isn't the circle on the xy-plane, but I've projected it up onto the graph. So, this is showing you basically the values where this constraint holds and also what they look like when graphed. A way you can think about a problem like this is that you're looking on this circle, this kind of projected circle onto the graph, and looking for the highest points.

You might notice kind of here there's sort of a peak on that wiggly circle, and over here there's another one. The low points would be, you know, around that point and around over here. Now, this is good, and I think this is a nice way to sort of wrap your head around what this problem is asking. But there's actually a better way to visualize it in terms of finding the actual solution, and that's to look only in the x, y plane rather than trying to graph things and just limit our perspective to the input space.

What I have here are the contour lines for f of x, y equals x squared plus y squared. If you're unfamiliar with contour lines or contour maps, I have a video on that. You can go back and take a look; it's going to be pretty crucial for the next couple videos to have a feel for that. But basically, each one of these lines represents a certain constant value for f.

So, for example, one of them might represent all of the values of x and y where f of x, y is equal to, you know, 2. Right? So if you looked at all the values of x and y where this is true, you'd find yourself on one of these lines, and each line represents a different possible value for what this constant here actually is. So, what I'm going to do here is I'm actually going to just zoom in on one particular contour line.

Right? So this here is something that I'm going to vary, where I'm going to be able to change what the constant we're setting f equal to is and look at how the contour line changes as a result. So for example, if I put it around here-ish, what you're looking at is the contour line for f of x, y equals 0.1. So, all of the values on these two blue lines here tell you what values of x and y satisfy 0.1.

But on the other hand, I could also shift this guy up, and maybe I'll shift it up. I'm going to set it to where that constant is actually equal to 1. So that would be kind of an alternative. I'll just kind of separate over here; that would be the line where f of x, y is set equal to 1 itself. The main thing I want to highlight here is that at some values, like 0.1, this contour line intersects with the circle. It intersects with our constraint.

And let's just think about what that means. If there's a point x and y on that intersection there, that basically gives us a pair of numbers x and y such that this is true: that fact that f of x, y equals 0.1 and also that x squared plus y squared equals 1. So, it means this is something that actually exists and is possible. In fact, we can see that there's four different pairs of numbers where that's true: where they intersect here, where they intersect over here, and then the other two kind of symmetrically on that side.

But on the other hand, if we look at this other world where we shift up to the line f of x, y equals 1, this never intersects with the constraint. So, what that means is x, y — the pairs of numbers that satisfy this guy are off the constraint; they're off of that circle x squared plus y squared equals 1. So what that tells us as we try to maximize this function subject to this constraint is that we can never get as high as 1.

0.1 would be achievable, and in fact, you know, if we kind of go back to that and we look at 0.1, if I upped that value and, you know, changed it to the line where instead what you're looking at is 0.2, that's also possible because it intersects with the circle. In fact, you know, you could play around with it and increase it a little bit more. If I go to 0.3 instead, and I go over here and I say 0.3, that's also possible.

What we're basically trying to do is find the maximum value that we can put here — the maximum value so that if we look at the line that represents f of x, y equals that value, it still intersects with the circle. The key here, the key observation is that that maximum value happens when these guys are tangent.

In the next video, I'll start going into the details of how we can use that observation — this notion of tangency — to solve the problem, to find the actual value of x and y that maximizes this subject to the constraint. But in the interim, I kind of want you to mull on that and think a little bit about how you might use that. What does tangency mean here? How can you take advantage of certain other notions that we've learned about in multivariable calculus, like — hint, the gradient — to actually solve something like this?

So with that, I will see you next video!

More Articles

View All
Don’t forget the “viable” part.
If you can’t get anyone to use your MVP, it’s probably not an MVP. Well, it’s not the V; we’re missing the V, which is viable. Like, basically, if it doesn’t work for anyone, yeah, how hard to argue that it’s viable? No, and like, shouldn’t an MVP— it see…
Product rule example
So let’s see if we can find the derivative with respect to ( x ) of ( F = e^x \cdot \cos(x) ). And like always, pause this video and give it a go on your own before we work through it. So when you look at this, you might say, “Well, I know how to find th…
Overpopulation – The Human Explosion Explained
Never before in history, have there been so many people on Earth as right now. Our numbers have skyrocketed, from 1 billion in 1800, to 2.3 billion in 1940, 3.7 billion in 1970, and 7.4 billion in 2016. The world population increased fourfold in the last …
NYE Reflection on 2019 and ahead in 2020
Hello everyone! I want to take a moment to say Happy New Year to you all and reflect on what has happened in 2019 and what’s gonna happen in 2020. So first of all, it has been a very slow year. I have not done nearly as much as I wanted to. There were so…
Creating scale drawings | Geometry | 7th grade | Khan Academy
Sue is a software engineer. She wants to create a large-scale drawing of a processor inside a cell phone. The processor is a square chip, nine millimeters on each side. Draw the processor such that one unit on the grid below represents one half of a milli…
Daylight Saving Time 101 | National Geographic
Daylight saving time adds extra hours of daylight during the summer season while making the day shorter during the winter months. But who came up with the concept of daylight saving time, and how does it work? The concept of shifting our clocks to adjust…