Lagrange Multipliers

Turning a constrained problem into an unconstrained one

The basic idea behind the method of Lagrange multipliers is to convert a constrained optimization problem into a form where the usual derivative-based tests for maxima and minima can still be applied.

Suppose we want to maximize or minimize a function $f(x)$ subject to one or more equality constraints:

$$g(x)=0$$ (1)

Instead of solving this directly, we define a new function—called the **Lagrangian**:

$$ℒ(x,\lambda )=f(x)+⟨\lambda ,g(x)⟩$$ (2)

where $⟨\cdot ,\cdot ⟩$ is the inner product. In simpler cases, this reduces to a dot product:

$$ℒ(x,\lambda )=f(x)+\lambda \cdot g(x)$$ (3)

To find optimal points, we look for stationary points of $ℒ(x,\lambda )$. That means we solve:

${\displaystyle \frac{\partial ℒ}{\partial x}}$ $=0$ (4)

${\displaystyle \frac{\partial ℒ}{\partial \lambda }}$ $=0$ (5)

Equation (5) is equivalent to enforcing the original constraint:

$$g(x)=0$$ (6)

while (4) ensures the gradients of $f$ and $g$ are aligned:

$$\nabla f(x)+\lambda \cdot \nabla g(x)=0$$ (7)

This approach avoids needing to parameterize the constraint explicitly, and the solution to the original problem turns out to be a saddle point of the Lagrangian function. In more advanced settings, checking definiteness via the bordered Hessian can confirm the nature of the point.

This method also extends beyond equality constraints using the more general Karush–Kuhn–Tucker (KKT) conditions, which can handle inequalities like $h(x)\le c$.

Example

Let’s find the maximum and minimum values of:

$$f(x,y)=xy$$

subject to:

$$g(x,y)=x^2+y^2-1=0$$

This keeps us on the unit circle.

Setting up the Lagrangian and equations

We define the Lagrangian:

$$ℒ(x,y,\lambda )=xy-\lambda (x^2+y^2-1)$$ (8)

Take partial derivatives and set them to zero:

${\displaystyle \frac{\partial ℒ}{\partial x}}$ $=y-2\lambda x=0$ (9)

${\displaystyle \frac{\partial ℒ}{\partial y}}$ $=x-2\lambda y=0$ (10)

${\displaystyle \frac{\partial ℒ}{\partial \lambda }}$ $=-(x^2+y^2-1)=0$ (11)

Solving the system

From (9) and (10):

$$y=2\lambda x,x=2\lambda y\Rightarrow x=4{\lambda }^{2}x\Rightarrow (1-4{\lambda }^2)x=0$$

This gives two cases:

• $x=0\Rightarrow y=\pm 1$

• $\lambda =\pm \frac{1}{2}\Rightarrow x=\pm \frac{1}{\sqrt{2}},y=\pm \frac{1}{\sqrt{2}}$

Checking which ones are max or min

Evaluate $f(x,y)=xy$ at each critical point:

$f(0,\pm 1)$ $=0$

$f({\displaystyle \frac{1}{\sqrt{2}}},{\displaystyle \frac{1}{\sqrt{2}}})$ $={\displaystyle \frac{1}{2}}$

$f(-{\displaystyle \frac{1}{\sqrt{2}}},-{\displaystyle \frac{1}{\sqrt{2}}})$ $={\displaystyle \frac{1}{2}}$

$f({\displaystyle \frac{1}{\sqrt{2}}},-{\displaystyle \frac{1}{\sqrt{2}}})$ $=-{\displaystyle \frac{1}{2}}$

$f(-{\displaystyle \frac{1}{\sqrt{2}}},{\displaystyle \frac{1}{\sqrt{2}}})$ $=-{\displaystyle \frac{1}{2}}$

Therefore: Maximum value: $\boxed{\frac{1}{2}}$ Minimum value: $\boxed{-\frac{1}{2}}$