Vector Calculus

Johan Krouthen (1858-1932) was a Swedish painter.

We have sections

Limits and Derivative of Vector-Valued Functions
Partial Derivatives
Equality of Mixed Partials
Gradient and Directional Derivatives
Maximum and Minimum Values

\begin{equation}{\label{a}}\tag{A}\mbox{}\end{equation}

Limits and Derivative of Vector-Valued Functions

The norm of a vector ${\bf a}=(a_{1},a_{2},a_{3})$, denoted by $\parallel {\bf a}\parallel$, is the number
\[\parallel {\bf a}\parallel =\sqrt{a_{1}^{2}+a_{2}^{2}+a_{3}^{2}}\]
Let ${\bf a}=(a_{1},a_{2},a_{3})$ and ${\bf b}=(b_{1},b_{2},b_{3})$. Then, the vector addition ${\bf a}+{\bf b}$ is defined by
\[{\bf a}+{\bf b}=(a_{1}+b_{1},a_{2}+b_{2},a_{3}+b_{3}).\]

For $\alpha\in\mathbb{R}$, the scalar multiplication $\alpha {\bf a}$ is defined by
\[\alpha {\bf a}=(\alpha a_{1},\alpha a_{2},\alpha a_{3}).\]
The inner produce (or dot product) is defined by
\[{\bf a}\bullet {\bf b}=a_{1}b_{1}+a_{2}b_{2}+a_{3}b_{3}.\]

Definition. (Limit of Vector Function). Let the vector function ${\bf f}(t)=(f_{1}(t),f_{2}(t),f_{3}(t))$ be defined on some interval $I$ containing the point $t_{0}$, except possibly at $t_{0}$ itself, and let ${\bf L}$ be a vector. We define
\[\lim_{t\rightarrow t_{0}} {\bf f}(t)={\bf L}\mbox{ if and only if }\lim_{t\rightarrow t_{0}}\parallel {\bf f}(t)-{\bf L}\parallel =0.\]

We can show that
\[\lim_{t\rightarrow t_{0}} {\bf f}(t)={\bf L}\mbox{ implies }\lim_{t\rightarrow t_{0}}\parallel {\bf f}(t)\parallel =\parallel {\bf L}\parallel .\]
However the converse is false. In other words,

\[\lim_{t\rightarrow t_{0}}\parallel {\bf f}(t)\parallel =\parallel {\bf L}\parallel\mbox{ does not necessarily imply }\lim_{t\rightarrow t_{0}} {\bf f}(t).\]

For example, given ${\bf a}$ be a nonzero vector, set ${\bf f}(t)={\bf a}$ for all $t$ and set ${\bf L}=-{\bf a}$. Then, for any $t_{0}\in\mathbb{R}$, we have
\[\lim_{t\rightarrow t_{0}}\parallel {\bf f}(t)\parallel =\lim_{t\rightarrow t_{0}} \parallel {\bf a}\parallel =\parallel -{\bf a}\parallel =\parallel {\bf L}\parallel\]

but
\[\lim_{t\rightarrow t_{0}} {\bf f}(t)=\lim_{t\rightarrow t_{0}} {\bf a}={\bf a}\neq -{\bf a}={\bf L}.\]

\begin{equation}{\label{t5}}\tag{1}\mbox{}\end{equation}

Theorem \ref{t5}. (Rules of Limits). Let ${\bf f}$ and ${\bf g}$ be two vector functions, and let $u$ be a real-valued function. Suppose that
\[\lim_{t\rightarrow t_{0}}{\bf (t)}={\bf L},\quad \lim_{t\rightarrow t_{0}}{\bf g}(t)={\bf M}\mbox{ and }\lim_{t\rightarrow t_{0}}u(t)= A.\]
Then, we have
\begin{align*} & \lim_{t\rightarrow t_{0}}\left ({\bf f}(t)+{\bf g}(t)\right )={\bf L}+{\bf M}\\
& \lim_{t\rightarrow t_{0}}\alpha {\bf f}(t)=\alpha {\bf L}\\
& \lim_{t\rightarrow t_{0}}u(t){\bf f}(t)\rightarrow A{\bf L}\\
& \lim_{t\rightarrow t_{0}}\left ({\bf f}(t)\bullet {\bf g}(t)\right ){\bf L}\bullet {\bf M}.\end{align*}

Let ${\bf f}(t)=(f_{1}(t),f_{2}(t),f_{3}(t))$ and ${\bf L}=(L_{1},L_{2},L_{3})$. Then, we have
\[\lim_{t\rightarrow t_{0}}{\bf f}(t)={\bf L}\mbox{ if and only if }\lim_{t\rightarrow t_{0}} f_{1}(t)=L_{1},
\lim_{t\rightarrow t_{0}} f_{2}(t)=L_{2},\lim_{t\rightarrow t_{0}} f_{3}(t)=L_{3}.\]

Example. Let ${\bf f}(t)=(\cos (t+\pi ),\sin (t+\pi ), e^{-t^{2}})$. Then, we have
\[\lim_{t\rightarrow 0} {\bf f}(t)=(\lim_{t\rightarrow 0} \cos (t+\pi ),\lim_{t\rightarrow 0} \sin (t+\pi ),\lim_{t\rightarrow 0} e^{-t^{2}})=(-1,0,1).\]

Definition. We say that the vector function ${\bf f}$ is continuous at $t_{0}$ when
\[\lim_{t\rightarrow t_{0}} {\bf f}(t)={\bf f}(t_{0}).\]

Theorem. ${\bf f}(t)$ is continuous at $t_{0}$ if and only if each component of ${\bf f}(t)$ is continuous at $t_{0}$; that is, $f_{1}(t)$, $f_{2}(t)$ and $f_{3}(t)$ are continuous at $t_{0}$.

Proof. It is clear from Theorem \ref{t5}. $\blacksquare$

Definition. (Derivative of Vector Function) We say that the vector function ${\bf f}$ is differentiable at $t$ when the limit
\[\lim_{h\rightarrow 0}\frac{{\bf f}(t+h)-{\bf f}(t)}{h}\]
exists. In this case, it is called the derivative of ${\bf f}$ at $t$, and is denoted by ${\bf f}'(t)$. We say that the vector function ${\bf f}$ is differentiable when it is differentiable at each point of its domain. $\sharp$

Theorem. Suppose that ${\bf f}(t)$ is differentiable at $t$. Then ${\bf f}'(t)=(f’_{1}(t),f’_{2}(t),f’_{3}(t))$. If ${\bf f}$ is differentiable at $t$, then ${\bf f}$ is continuous at $t$. $\sharp$

Example. Let ${\bf f}(t)=(t\sin t,e^{-t},t)$. Then, we have

\[{\bf f}'(t)=(=\cos t+\sin t,-e^{-t},1)\mbox{ and }{\bf f}’^{\prime\prime}(t)=(2\cos t-t\sin t,e^{-t},0).\]

Definition. Let ${\bf f}(t)=(f_{1}(t),f_{2}(t),f_{3}(t))$ be continuous on $[a,b]$. We define
\[\int_{a}^{b} {\bf f}(t)dt=\left (\int_{a}^{b} f_{1}(t)dt,\int_{a}^{b} f_{2}(t)dt,\int_{a}^{b} f_{3}(t)dt\right ).\]

Example. We have some interesting examples.

(i) Let ${\bf f}(t)=(t,\sqrt{1+t},-e^{t})$. Then, we have
\begin{align*} \int_{0}^{1} {\bf f}(t)dt & =\left (\int_{0}^{1} tdt,\int_{0}^{1} \sqrt{1+t}dt,
\int_{0}^{1} (-e^{t})dt\right )\\ & =(\frac{1}{2},\frac{2}{3}(2\sqrt{2}-1),1-e).\end{align*}

(ii) Given ${\bf f}'(t)=(2\cos t,-t\sin t,2t)$ and ${\bf f}(0)=(1,0,3)$, find ${\bf f}(t)$. By integrating ${\bf f}'(t)$, we obtain
\[{\bf f}(t)=(2\sin t+C_{1},\frac{1}{2}\cos t^{2}+C_{2},t^{2}+C_{3}).\]
Since
\[(1,0,3)={\bf f}(0)=(C_{1},\frac{1}{2}+C_{2},C_{3}),\]
it follows $C_{1}=1$, $C_{2}=-\frac{1}{2}$ and $C_{3}=3$. Therefore, we obtain
\[{\bf f}(t)=(2\sin t+1,\frac{1}{2}\cos t^{2}-\frac{1}{2},t^{2}+3).\]

Theorem. Let ${\bf f}$ and ${\bf g}$ be two vector functions.

(i) We have \[\int_{a}^{b} [{\bf f}(t)+{\bf g}(t)]dt=\int_{a}^{b}{\bf f}(t)dt+\int_{a}^{b} {\bf g}(t)dt.\]

(ii) We have \[\int_{a}^{b} [\alpha {\bf f}(t)]dt=\alpha\int_{a}^{b} {\bf f}(t)dt\] for every $\alpha\in\mathbb{R}$.

(iii) We have \[\int_{a}^{b} [{\bf c}\bullet {\bf f}(t)]dt={\bf c}\bullet\left (\int_{a}^{b} {\bf f}(t)dt\right )\] for every constant vector ${\bf c}$.

(iv) We have \[\left |\left |\int_{a}^{b} {\bf f}(t)dt\right |\right |\leq\int_{a}^{b}\parallel {\bf f}(t)\parallel dt.\]

There are two ways of bringing scalar function into play. If a scalar function $u$ has the same domain as ${\bf f}$, we can form the scalar product $(u{\bf f})(t)=u(t){\bf f}(t)$. If $u(t)$ is in the domain of ${\bf f}$ for each $t$ in some interval, then we can form the composition ${\bf f}\circ u(t)={\bf f}(u(t))$.

Theorem. Let ${\bf f}$ and ${\bf g}$ be two vector functions.
(i) We have \[({\bf f}+{\bf g})'(t)={\bf f}'(t)+{\bf g}'(t).\]

(ii) We have \[(\alpha {\bf f})'(t)=\alpha {\bf f}'(t)\] for $\alpha\in\mathbb{R}$.

(iii) We have \[(u{\bf f})'(t)=u(t){\bf f}'(t)+u'(t){\bf f}(t).\]

(iv) We have \[({\bf f}\bullet {\bf g})'(t)={\bf f}(t)\bullet {\bf g}'(t)+{\bf f}'(t)\bullet {\bf g}(t).\]

(v) We have the chain rule given by \[({\bf f}\circ u)'(t)={\bf f}'(u(t))u'(t).\]

\begin{equation}{\label{b}}\tag{B}\mbox{}\end{equation}

Partial Derivatives.

Let $f(x,y)$ be a function of $x$ and $y$. The partial derivative of $f$ with respect to $x$ is the function $f_{x}$ obtained by differentiating $f$ with respect to $x$ and treating $y$ as a constant. The partial derivative of $f$ with respect to $y$ is the function $f_{y}$ obtained by differentiating $f$ with respect to $y$ and treating $x$ as a constant. The partial derivatives are formally defined as limits

Definition. (Partial Derivatives). Let $f$ be a real-valued function of two variables. The partial derivatives of $f$ with respect to $x$ and $y$ are the functions $f_{x}$ and $f_{y}$ defined by
\[f_{x}(x,y)=\lim_{h\rightarrow 0}\frac{f(x+h,y)-f(x,y)}{h}\mbox{ and }f_{y}(x,y)=\lim_{h\rightarrow 0}\frac{f(x,y+h)-f(x,y)}{h}\]
provided these limits exist. $\sharp$

Example. We have some interesting examples.

(i) Given $f(x,y)=x\tan^{-1}xy$, we have
\[f_{x}(x,y)=\frac{xy}{1+(xy)^{2}}+\tan^{-1}xy\mbox{ and }f_{y}(x,y)=\frac{x^{2}}{1+x^{2}y^{2}}.\]

(ii) Given $f(x,y)=e^{xy}+\ln (x^{2}+y)$, we have
\[f_{x}(x,y)=ye^{xy}+\frac{2x}{x^{2}+y}\mbox{ and }f_{y}(x,y)=xe^{xy}+\frac{1}{x^{2}+y^{2}}.\]

Definition. (Partial Derivatives). Let $f$ be a real-valued function of three variables. The partial derivatives of $f$ with respect to $x$, $y$ and $z$ are the functions $f_{x}$, $f_{y}$ and $f_{z}$ defined by
\begin{align*}
f_{x}(x,y,z) & =\lim_{h\rightarrow 0}\frac{f(x+h,y,z)-f(x,y,z)}{h}\\
f_{y}(x,y,z) & =\lim_{h\rightarrow 0}\frac{f(x,y+h,z)-f(x,y,z)}{h}\\
f_{z}(x,y,z) & =\lim_{h\rightarrow 0}\frac{f(x,y,z+h)-f(x,y,z)}{h}
\end{align*}
provided these limits exist. $\sharp$

Example. We have some interesting examples.

(i) Given the function $f(x,y,z)=xy^{2}z^{3}$, the partial derivatives are
\[f_{x}(x,y,z)=y^{2}z^{3},\quad f_{y}(x,y,z)=2xyz^{3}\mbox{ and }f_{z}(x,y,z)=3xy^{2}z^{2}.\]

(ii) Given the function $g(x,y,z)=x^{2}e^{y/z}$, the partial derivatives are
\[g_{x}(x,y,z)=2xe^{y/z}, \quad g_{y}(x,y,z)=\frac{x^{2}}{2}e^{y/z}\mbox{ and }g_{z}(x,y,z)=-\frac{x^{2}y}{z^{2}}e^{y/z}.\]

(iii) Given a function of the form $f(x,y,z)=F(x,y)G(y,z)$, we have
\[f_{x}(x,y,z)=F_{x}(x,y)G(y,z),f_{y}(x,y,z)=F(x,y)G_{y}(y,z)+F_{y}(x,y)G(y,z), f_{z}(x,y,z)=F(x,y)G_{z}(y,z).\]

We usually write the partial derivatives $f_{x}$, $f_{y}$ and $f_{z}$ as
\[\frac{\partial f}{\partial x}=f_{x},\quad\frac{\partial f}{\partial y}=f_{y}\mbox{ and }\frac{\partial f}{\partial z}=f_{z}.\]

\begin{equation}{\label{c}}\tag{C}\mbox{}\end{equation}

Equality of Mixed Partials.

Definition. (The Limit of Function of Several Variables) Let $f$ be a function defined at least on some neighborhood of ${\bf x}_{0}$ except possibly for ${\bf x}_{0}$. We define
\[\lim_{{\bf x}\rightarrow {\bf x}_{0}} f({\bf x})=L\]

when, given any $\epsilon >0$, there exists $\delta >0$ such that

\[0<\parallel {\bf x}-{\bf x}_{0}\parallel <\delta\mbox{ implies }|f({\bf x})-L|<\epsilon.\]

Example. We will show that the function
\[f(x,y)=\frac{xy+y^{3}}{x^{2}+y^{2}}\]
does not have a limit at $(0,0)$. We first note that $f$ is not defined at $(0,0)$, but is defined for all $(x,y)\neq (0,0)$. Along the $x$-axis, i.e., $y=0$, we have $f(x,y)=f(x,0)=0$ and ${displaystyle \lim_{x\rightarrow 0} f(x,0)=0}$. Along the $y$-axis, i.e., $x=0$. We have $f(x,y)=f(0,y)=y$ and $latex {\displaystyle
\lim_{y\rightarrow 0}f(0,y)=0}$. Along the line $y=2x$, we have
\[f(x,y)=f(x,2x)=\frac{2x^{2}+8x^{3}}{x^{2}+4x^{2}}=\frac{2}{5}+\frac{8}{5}x\rightarrow\frac{2}{5}\mbox{ as }x\rightarrow 0.\]
We also can verify that $f(x,y)\rightarrow -\frac{1}{2}$ as $(x,y)\rightarrow (0,0)$ along the line $y=-x$. We have shown that not all paths to $(0,0)$ yield the same limiting value. It follows that $f$ does not have a limit at $(0,0)$. $\sharp$

Example. Show that the function
\[g(x,y)=\frac{x^{2}y}{x^{4}+y^{2}}\]
has limiting value $0$ as $(x,y)\rightarrow (0,0)$ along any line through the origin, but ${\displaystyle \lim_{(x,y)\rightarrow (0,0)} g(x,y)}$ still does not exist. We first note that the domain of $g$ is all $(x,y)\neq (0,0)$. It is easy to verify $g(x,y)\rightarrow 0$ when $(x,y)\rightarrow (0,0)$ along the coordinate axes. Let $y=mx$. Then, we have
\[g(x,y)=g(x,mx)=\frac{mx^{3}}{x^{4}+m^{2}x^{2}}=\frac{mx}{x^{2}+m^{2}}\]

and

\[\lim_{x\rightarrow 0} g(x,mx)=\lim_{x\rightarrow 0}\frac{mx}{x^{2}+m^{2}}=0.\]
Therefore $g(x,y)\rightarrow 0$ as $(x,y)\rightarrow 0$ along any line through the origin. Now suppose that $(x,y)\rightarrow (0,0)$ along the parabola $y=x^{2}$. Then, we have
\[g(x,y)=g(x,x^{2})=\frac{x^{4}}{x^{4}+x^{4}}=\frac{1}{2}\mbox{ and }\lim_{x\rightarrow 0} g(x,x^{2})=\frac{1}{2}.\]
Since not all paths to $(0,0)$ yield the same limiting value, we conclude that $g$ does not have a limit at $(0,0)$. $\sharp$

Theorem. Suppose that

\[\lim_{{\bf x}\rightarrow {\bf x}_{0}} f({\bf x})=L\mbox{ and }\lim_{{\bf x}\rightarrow {\bf x}_{0}} g({\bf x})=M. \]

Then, we have
\begin{align*} & \lim_{{\bf x}\rightarrow {\bf x}_{0}}[f({\bf x})+g({\bf x})]=L+M\\
& \lim_{{\bf x}\rightarrow {\bf x}_{0}}[f({\bf x})g({\bf x})]=LM\\
& \lim_{{\bf x}\rightarrow {\bf x}_{0}}[f({\bf x})/g({\bf x})]=L/M\mbox{ provided $M\neq 0$}\\
& \lim_{{\bf x}\rightarrow {\bf x}_{0}}[\alpha f({\bf x})]=\alpha L\mbox{ for $\alpha\in\mathbb{R}$}.\end{align*}

Suppose now that ${\bf x}_{0}$ is an interior point of the domain of $f$. Then $f$ is continuous at ${\bf x}_{0}$ if and only if
\[\lim_{{\bf x}\rightarrow {\bf x}_{0}} f({\bf x})=f({\bf x}_{0})\]
or, equivalently,
\[\lim_{{\bf h}\rightarrow {\bf 0}} f({\bf x}_{0}+{\bf h})=f({\bf x}_{0}).\]
For two variables, we can write
\[\lim_{(x,y)\rightarrow (x_{0},y_{0})} f(x,y)=f(x_{0},y_{0})\]
and, for three variables, we write
\[\lim_{(x,y,z)\rightarrow (x_{0},y_{0},z_{0})} f(x,y,z)=f(x_{0},y_{0},z_{0}).\]

Theorem. Suppose that $g$ is continuous at the point ${\bf x}_{0}$, and that $f$ is continuous at the point $g({\bf x}_{0})$. Then, the composition $f\circ g$ is continuous at the point ${\bf x}_{0}$. $\sharp$

A continuous function of several variables is continuous in each of its variables separately, which means that
\[\lim_{(x,y)\rightarrow (x_{0},y_{0})} f(x,y)=f(x_{0},y_{0})\mbox{ implies }\lim_{x\rightarrow x_{0}} f(x,y_{0})=f(x_{0},y_{0})\mbox{ and }
\lim_{y\rightarrow y_{0}} f(x_{0},y)=f(x_{0},y_{0}).\]
The converse is false. It is possible for a function to be continuous in each variable separately, and it fails to be continuous as a function of several variables.

Let $z=f(x,y)$ be a function of $x$ and $y$. Then, we have the following notations for the second-order partials
\begin{align*}
& (f_{x})_{x}=f_{xx}=\frac{\partial}{\partial x}\left (\frac{\partial f}{\partial x}\right )=\frac{\partial^{2} f}{\partial x^{2}}=\frac{\partial^{2} z}{\partial x^{2}},\\
& (f_{x})_{y}=f_{xy}=\frac{\partial}{\partial y}\left (\frac{\partial f}{\partial x}\right )=\frac{\partial^{2} f}{\partial y\partial x}=\frac{\partial^{2} z}{\partial y\partial x},\\
& (f_{y})_{x}=f_{yx}=\frac{\partial}{\partial x}\left (\frac{\partial f}{\partial y}\right )=\frac{\partial^{2} f}{\partial x\partial y}=\frac{\partial^{2} z}{\partial x\partial y},\\
& (f_{y})_{y}=f_{yy}=\frac{\partial}{\partial y}\left (\frac{\partial f}{\partial y}\right )=\frac{\partial^{2} f}{\partial y^{2}}=\frac{\partial^{2} z}{\partial y^{2}}.
\end{align*}

Example. The function $f(x,y)=\sin x^{2}y$ has first partials
\[\frac{\partial f}{\partial x}=2xy\cos x^{2}y\mbox{ and }\frac{\partial f}{\partial y}=x^{2}\cos x^{2}y.\]
The second-order partials are given by
\begin{align*}
& \frac{\partial^{2} f}{\partial x^{2}}=-4x^{2}y^{2}\sin x^{2}y+2x\cos x^{2}y\\ & \frac{\partial^{2} f}{\partial y\partial x}=-2x^{3}y\sin x^{2}y+2x\cos x^{2}y\\
& \frac{\partial^{2} f}{\partial x\partial y}=-2x^{3}y\sin x^{2}y+2x\cos x^{2}y\\ & \frac{\partial^{2} f}{\partial y^{2}}=-x^{4}\sin x^{2}y.
\end{align*}

In the case of function of three variables, we can look for three first partials
\[\frac{\partial f}{\partial x},\quad\frac{\partial f}{\partial y}\mbox{ and }\frac{\partial f}{\partial z},\]
and nine second partials
\begin{align*} & \frac{\partial^{2} f}{\partial x^{2}},\quad
\frac{\partial^{2} f}{\partial x\partial y},\quad
\frac{\partial^{2} f}{\partial x\partial z},\quad
\frac{\partial^{2} f}{\partial y^{2}},\quad
\frac{\partial^{2} f}{\partial y\partial x},\quad
\frac{\partial^{2} f}{\partial y\partial z},\quad
\frac{\partial^{2} f}{\partial z^{2}},\quad
\frac{\partial^{2} f}{\partial z\partial x}\mbox{ and }
\frac{\partial^{2} f}{\partial z\partial y}.\end{align*}

\begin{equation}{\label{d}}\tag{D}\mbox{}\end{equation}

Gradient and Directional Derivatives.

Let $g({\bf x})$ be a function of several variables which is defined in some neighborhood of ${\bf 0}$. We say that $g({\bf h})$ is $o({\bf h})$ (read as little o of ${\bf h}$) when
\[\lim_{{\bf h}\rightarrow 0}\frac{g({\bf h})}{\parallel\bf h\parallel}=0.\]

Definition. We say that $f$ is differentiable at ${\bf x}$ when there exists a vector ${\bf y}$ satisfying
\[f({\bf x}+{\bf h})-f({\bf x})={\bf y}\bullet {\bf h}+o({\bf h}).\]
Let $f$ be differentiable at ${\bf x}$. The gradient of $f$ at ${\bf x}$ is the unique vector $\nabla f({\bf x})$ satisfying
\[f({\bf x}+{\bf h})-f({\bf x})=\nabla f({\bf x})\bullet {\bf h}+o({\bf h}).\]

Theorem. Suppose that $f$ has continuous first partials in a neighborhood of ${\bf x}$. Then $f$ is differentiable at ${\bf x}$ and
\[\nabla f({\bf x})=\left (\frac{\partial f}{\partial x},\frac{\partial f}{\partial y}\right )\mbox{ or }
\nabla f({\bf x})=\left (\frac{\partial f}{\partial x},\frac{\partial f}{\partial y},\frac{\partial f}{\partial z}\right ).\]

Example. For $f(x,y,z)=\sin (xy^{2}z^{3})$, we have
\[\nabla f=\left (\frac{\partial f}{\partial x},\frac{\partial f}{\partial y},\frac{\partial f}{\partial z}\right )=
(y^{2}z^{3}\cos (xy^{2}z^{3}),2xyz^{3}\cos (xy^{2}z^{3}),3xy^{2}z^{2}\cos (xy^{2}z^{3})).\]

Theorem. If $f$ is differentiable at ${\bf x}$, then $f$ is continuous at ${\bf x}$. $\sharp$

Theorem. Suppose that $\nabla f({\bf x})$ and $\nabla g({\bf x})$ exist. Then, we have
\begin{align*}
& \nabla [f({\bf x})+g({\bf x})]=\nabla f({\bf x})+\nabla g({\bf x})\\
& \nabla [\alpha f({\bf x})]=\alpha \nabla f({\bf x})\\
& \nabla [f({\bf x})g({\bf x})]=f({\bf x})\nabla g({\bf x})+g({\bf x})\nabla f({\bf x}).
\end{align*}

Definition. (Directional Derivative). For each unit vector ${\bf u}$, if the limit
\[f’_{{\bf u}}({\bf x})=\lim_{h\rightarrow 0}\frac{f({\bf x}+h{\bf u})-f({\bf x})}{h},\]
exists, it is called the directional derivative of $f$ at ${\bf x}$ in the direction ${\bf u}$. $\sharp$

The definition of the directional derivative of $f$ in the direction ${\bf u}$ requires ${\bf u}$ to be a unit vector. However, we can extend the definition to arbitrary nonzero vectors as follows. The directional derivative of $f$ at ${\bf x}$ in the direction of a nonzero vector ${\bf a}$ is $f’_{{\bf u}}({\bf x})$ where ${\bf u}={\bf a}/\parallel {\bf a}\parallel$ is the unit vector having the same direction as ${\bf a}$.

For ${\bf u}={\bf i}=(1,0,0)$, we have
\[f’_{{\bf i}}({\bf x})=\lim_{h\rightarrow 0}\frac{f({\bf x}+h{\bf i})-
f({\bf x})}{h}=\lim_{h\rightarrow 0}\frac{f(x+h,y,z)-f(x,y,z)}{h}=\frac{\partial f}{\partial x}({\bf x}).\]
For ${\bf u}={\bf j}=(0,1,0)$, we have
\[f’_{{\bf j}}({\bf x})=\lim_{h\rightarrow 0}\frac{f({\bf x}+h{\bf j})-
f({\bf x})}{h}=\lim_{h\rightarrow 0}\frac{f(x,y+h,z)-f(x,y,z)}{h}=\frac{\partial f}{\partial y}({\bf x}).\]
For ${\bf u}={\bf k}=(0,0,1)$, we have
\[f’_{{\bf k}}({\bf x})=\lim_{h\rightarrow 0}\frac{f({\bf x}+h{\bf k})-
f({\bf x})}{h}=\lim_{h\rightarrow 0}\frac{f(x,y,z+h)-f(x,y,z)}{h}=\frac{\partial f}{\partial z}({\bf x}).\]

Theorem. Suppose that $f$ is differentiable at ${\bf x}$. Then $f$ has a directional derivative at ${\bf x}$ in every direction ${\bf u}$, where ${\bf u}$ is a unit vector, and
\[f’_{{\bf u}}({\bf x})=\nabla f({\bf x})\bullet {\bf u}.\]

Example. Find the directional derivative of the function $f(x,y,z)=x\cos y\sin z$ at the point $(1,\pi,\frac{1}{4}\pi )$ in the direction of the vector ${\bf a}=(2,-1,4)$. The unit vector in the direction of ${\bf a}$ is the vector ${\bf u}=1/\sqrt{21}(2,-1,4)$. Now, we have
\[\frac{\partial f}{\partial x}(x,y,z)=\cos y\sin z,\quad\frac{\partial f}{\partial y}(x,y,z)=-x\sin y\sin z\mbox{ and }\frac{\partial f}{\partial z}(x,y,z)=x\cos y\cos z.\]
Therefore, we obtain
\[\frac{\partial f}{\partial x}(1,\pi,\pi /4)=-\sqrt{2}/2,\quad\frac{\partial f}{\partial y}(1,\pi,\pi /4)=0\mbox{ and }\frac{\partial f}{\partial z}(1,\pi,\pi /4)=-\sqrt{2}/2,\]
which say $\nabla f(1,\pi,\pi /4)=(-\sqrt{2}/2,0,-\sqrt{2}/2)$ and
\[f’_{{\bf u}}(1,\pi ,\pi /4)=\nabla f(1,\pi ,\pi /4)\bullet {\bf u}=(-\sqrt{2}/2,0,-\sqrt{2}/2)\bullet 1/\sqrt{21}(2,-1,4)=\frac{3\sqrt{2}}{21}.\]

Theorem. (The Mean-Value Theorem for Several Variables) Suppose that $f$ is differentiable at each point of the line segment $\bar{{\bf ab}}$. Then, there exists on that line segment a pint ${\bf c}$ between ${\bf a}$ and ${\bf b}$ satisfying
\[f({\bf b})-f({\bf a})=\nabla f({\bf c})\bullet ({\bf a}-{\bf a}).\]

Theorem. We have the following properties.

(i) Let $U$ be an open connected set, and let $f$ be a differentiable function on $U$. If $\nabla f({\bf x})={\bf 0}$ for all ${\bf x}\in U$, then $f$ is constant on $U$.

(ii) Let $U$ be an open connected set, and let $f$ and $g$ be differentiable functions on $U$. If $\nabla f({\bf x})=\nabla g({\bf x})$ for all ${\bf x}\in U$, then $f$ and $g$ differ by a constant on $U$. $\sharp$

A vector-valued function ${\bf f}$ is said to be continuous provided that its components are continuous. If $f=f(x,y,z)$ is a real-valued function, then its gradient $\nabla f$ is a vector-valued function. We say that $f$ is continuously differentiable on an open set $U$ when $f$ is differentiable on $U$ and $\nabla f$ is continuous on $U$. If a curve ${\bf r}(t)$ lies in the domain of $f$, then we can form the composition
\[(f\circ {\bf r})(t)=f({\bf r}(t)).\]
The composition $f\circ {\bf r}$ is a real-valued function of a real variable $t$. The numbers $f({\bf r}(t))$ are the values taken on by $f$ along the curve ${\bf r}$.

Theorem. (Chain Rule Along a Curve). Suppose that $f$ is continuously differentiable on an open set $U$, and that ${\bf t}(t)$ is a differentiable curve that lies entirely in $U$. Then, the composition $f\circ {\bf r}$ is differentiable and
\[\frac{d}{dt}[f({\bf r}(t))]=\nabla f({\bf r}(t))\bullet{\bf r}'(t).\]

Example. Use the chain rule to find the rate of change of $f(x,y)=\frac{1}{3}(x^{3}+y^{3})$ with respect to $t$ along the ellipse ${\bf r}(t)=(a\cos t,b\sin t)$. The rate of change of $f$ with respect to $t$ along the curve ${\bf r}$ is the derivative ${\displaystyle \frac{d}{dt}[f({\bf r}(t))]}$. We have

\[\nabla f=(x^{2},y^{2})\mbox{ and }{\bf r}'(t)=(-a\sin t,b\cos t).\]

Using the chain rule, we obtain
\begin{align*}
\frac{d}{dt}[f({\bf r}(t))] & =\nabla f({\bf r}(t))\bullet {\bf r}'(t)\\
& =(a^{2}\cos^{2}t,b^{2}\sin^{2}t)\bullet (-a\sin t,b\cos t)\\
& =-a^{3}\sin t\cos^{2}t+b^{3}\sin^{2}t\cos t\\
& =\frac{1}{2}\sin 2t(b^{3}\sin t-a^{3}\cos t).
\end{align*}

Theorem. (Chain Rule). We have the following properties.

(i) Let $u(x,y)$ be a function of $x$ and $y$, and let $x=x(s,t)$ and $y=y(s,t)$ be the functions of $s$ and $t$. Then, we have
\[\frac{\partial u}{\partial s}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial s}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial s}\]

and
\[\frac{\partial u}{\partial t}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial t}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial t}.\]

(ii) Let $u(x,y,z)$ be a function of $x$, $y$ and $z$, and let $x=x(s,t)$, $y=y(s,t)$ and $z(s,t)$ be functions of $s$ and $t$. Then, we have
\[\frac{\partial u}{\partial s}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial s}+\frac{\partial u}{\partial y}\frac{\partial y}
{\partial s}+\frac{\partial u}{\partial z}\frac{\partial z}{\partial s}\]
and
\[\frac{\partial u}{\partial t}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial t}+\frac{\partial u}{\partial y}\frac{\partial y}
{\partial t}+\frac{\partial u}{\partial z}\frac{\partial z}{\partial t}.\]

Example. Let $u(x,y)=x^{2}-2xy+2y^{3}$, where $x(s,t)=s^{2}\ln t$ and $y(s,t)=2st^{3}$. Find $\partial u/\partial s$ and $\partial u/\partial t$. Since
\[\frac{\partial u}{\partial x}=(2x-2y),\frac{\partial u}{\partial y}=(-2x+6y^{2})\]
and
\[\frac{\partial x}{\partial s}=2s\ln t,\frac{\partial y}{\partial s}=2t^{3},
\frac{\partial x}{\partial t}=\frac{s^{2}}{t},\frac{\partial y}{\partial t}=6st^{2},\]
we have
\[\frac{\partial u}{\partial s}=(2x-2y)(2s\ln t)+(-2x+6y^{2})(2t^{3})\]

and

\[\frac{\partial u}{\partial t}=(2x-2y)\left (\frac{s^{2}}{t}\right )+(-2x+6y^{2})(6st^{2}).\]

Example. Let $u=x^{2}y^{3}e^{xz}$, where $x=s^{2}+t^{2}$, $y=2st$, $z=s\ln t$. Find $\partial u/\partial s$. Since
\[\frac{\partial u}{\partial x}=2xy^{3}e^{xz}+x^{2}y^{3}ze^{xz},\frac{\partial u}{\partial y}=3x^{2}y^{2}e^{xz},
\frac{\partial u}{\partial z}=x^{3}y^{3}e^{xz}\]
and
\[\frac{\partial x}{\partial s}=2s,\frac{\partial y}{\partial s}=2t,\frac{\partial z}{\partial s}=\ln t,\]
we have
\[\frac{\partial u}{\partial s}=(2xy^{3}e^{xz}+x^{2}y^{3}ze^{xz})(2s)+(3x^{2}y^{2}e^{xz})(2t)+(x^{3}y^{3}e^{xz})(\ln t).\]

Theorem. (Implicit Differentiation). We have the following properties.

(i) Let $u(x,y)$ be continuously differentiable. Suppose that the equation $u(x,y)=0$ defines $y$ implicitly as a differentiable function of $x$. If $\partial u/\partial y\neq 0$, then
\[\frac{dy}{dx}=-\frac{\partial u/\partial x}{\partial u/\partial y}.\]

(ii) Let $u(x,y,z)$ be continuously differentiable. Suppose that the equation $u(x,y,z)=0$ defines $z$ implicitly as a differentiable function of $x$ and $y$. If $\partial u/\partial z\neq 0$, then
\[\frac{\partial z}{\partial x}=-\frac{\partial u/\partial x}{\partial u/\partial z}\mbox{ and }\frac{\partial z}{\partial y}=
-\frac{\partial u/\partial y}{\partial u/\partial z}.\]

Proof. To prove part (i), we introduce a variable $t$ by setting $x=x$. Then we have $u=u(x,y)$ with $x=t$ and $y=y(t)$. Using the chain rule, we have
\[\frac{du}{dt}=\frac{\partial u}{\partial x}\frac{dx}{dt}+\frac{\partial u}{\partial y}\frac{dy}{dt}.\]
Since $u(t,y(t))=0$ for all $t$, we have $du/dt=0$. Also, since $x=t$, we have $dx/dt=1$ and $dy/dt=dy/dx$. Therefore, we obtain
\[0=\frac{\partial u}{\partial x}+\frac{\partial u}{\partial y}\frac{dy}{dx}.\]

To prove part (ii), we write $u=u(x,y,z)$ with $x=s$, $y=t$ and $z=z(s,t)$. Since $u(s,t,z(s,t))=0$ for all $s$ and $t$, we have $\partial u/\partial s=0$. Since $\partial x/\partial s=1$ and $\partial y/\partial s=0$, we also have
\begin{align*} 0 & =\frac{\partial u}{\partial s}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial s}+\frac{\partial u}{\partial y}\frac{\partial y}
{\partial s}+\frac{\partial u}{\partial z}\frac{\partial z}{\partial s}\\ & =\frac{\partial u}{\partial x}\cdot 1+\frac{\partial u}{\partial y}\cdot 0+
\frac{\partial u}{\partial z}\frac{\partial z}{\partial s}\\ & =\frac{\partial u}{\partial x}+\frac{\partial u}{\partial z}\frac{\partial z}{\partial x}.\end{align*}
Therefore, we obtain
\[\frac{\partial z}{\partial x}=-\frac{\partial u/\partial x}{\partial u/\partial z}.\]
The formula for $\partial z/\partial y$ can be obtained in a similar manner. $\blacksquare$

Example. Consider the equation $u(x,y)=2xy-y^{3}+1-x-2y=0$. Then, we have
\[\frac{dy}{dx}=-\frac{\partial u/\partial x}{\partial u/\partial y}=-\frac{2y-1}{2x-3y^{2}-2}=\frac{1-2y}{2x-3y^{3}-2}.\]

\begin{equation}{\label{e}}\tag{E}\mbox{}\end{equation}

Maximum and Minimum Values.

Definition. (Local Maximum and Local Minimum). Let $f$ be a function of several variables, and let ${\bf x}_{0}$ be an interior point of the domain.

The function $f$ is said to have a local maximum at ${\bf x}_{0}$ when $f({\bf x}_{0})\geq f({\bf x})$ for all ${\bf x}$ in some neighborhood of ${\bf x}_{0}$.
The function $f$ is said to have a local minimum at ${\bf x}_{0}$ when $f({\bf x}_{0})\leq f({\bf x})$ for all ${\bf x}$ in some neighborhood of
${\bf x}_{0}$.

The local maxima and local minima together comprise the local extreme values. $\sharp$

Theorem. Suppose that $f$ has a local extreme values. Then, either $\nabla f({\bf x}_{0})={\bf 0}$ or $\nabla f({\bf x}_{0})$ does not exist. $\sharp$

Interior points of the domain at which the gradient is zero or the gradient does not exist are called critical points. Critical points at which the gradient is zero are called stationary points. The stationary points that do not give rise to extreme values are called saddle points.

Example. For the function $f(x,y)=2x^{2}+y^{2}-xy-7y$, we have $\nabla f(x,y)=(4x-y,2y-x-7)$. We set $\nabla f(x,y)=0$. The point $(1,4)$ is the only stationary point. We now compare the value of $f$ with the value of $f$ at nearby points $(1+h,4+k)$: $f(1,4)=-14$ and $f(1+h,4+k)=2h^{2}+k^{2}-hk-14$. The difference
\begin{align*} f(1+h,4+k)-f(1,4) & =h^{2}+(h^{2}-hk+k^{2})\\ & \geq h^{2}+(h^{2}-2|h||k|+k^{2})\\ & =h^{2}+(|h|-|k|)^{2}\geq 0.\end{align*}
Therefore $f(1+h,4+k)\geq f(1,4)$ for all small $h$ and $k$. It follows that $f$ has a local minimum at $(1,4)$. This local minimum is $-14$. $\sjarp$

Example. For the function $f(x,y)=y^{2}-xy+2x+y+1$, set $\nabla f(x,y)=(2-y,2y-x+1)={\bf 0}$. Then, we obtain $x=5$ and $y=2$. The point $(5,2)$ is the
only stationary point. We now compare the value of $f$ at $(5,2)$ with the value of $f$ at nearby points $(5+h,2+k)$: $f(5,2)=7$ and $f(5+h,2+k)=k^{2}-hk+7$. The difference $d=f(5+h,2+k)-f(5,2)=k(k-h)$ does not keep a constant sign for small $h$ and $k$. Therefore, it follows that $(5,2)$ is a saddle point. $\sharp$

Theorem. (The Second-Partials Test). Suppose that $f$ has continuous second-order partial derivatives in a neighborhood of $(x_{0},y_{0})$, and that we have $\nabla f(x_{0},y_{0})={\bf 0}$. Set
\[A=\frac{\partial^{2} f}{\partial x^{2}}(x_{0},y_{0}),\quad B=\frac{\partial^{2} f}{\partial y\partial x}(x_{0},y_{0}),\quad
C=\frac{\partial^{2} f}{\partial y^{2}}(x_{0},y_{0})\]
and form the discriminant $D=B^{2}-AC$.

(i) If $D>0$, then $(x_{0},y_{0})$ is a saddle point.

(ii) If $D<0$, then $f$ has a local minimum at $(x_{0},y_{0})$ for $A>0$, and a local maximum at $(x_{0},y_{0})$ for $A<0$. $\sharp$

Example. For the function $f(x,y)=-xye^{-(x^{2}+y^{2})/2}$, we have
\[\frac{\partial f}{\partial x}=y(x^{2}-1)e^{-(x^{2}+y^{2})/2}\mbox{ and }
\frac{\partial f}{\partial y}=x(y^{2}-1)e^{-(x^{2}+y^{2})/2}.\]
Then $\nabla f(x,y)=0$ if and only if $y(1-x^{2})=0$ and $x(y^{2}-1)=0$. Therefore $(0,0),(1,1),(1,-1),(-1,1)$ and $(-1,-1)$ are the stationary points. The second partial derivatives are given by
\begin{align*} \frac{\partial^{2}f}{\partial x^{2}} & =xy(3-x^{2})e^{-(x^{2}+y^{2})/2}\\
\frac{\partial^{2}f}{\partial y^{2}} & =xy(3-y^{2})e^{-(x^{2}+y^{2})/2}\\
\frac{\partial^{2}f}{\partial y\partial x} & =(x^{2}-1)(1-y^{2})e^{-(x^{2}+y^{2})/2}.\end{align*}
The data for the second partials test are recorded in the following table
\[\begin{array}{cccccc}\hline
\mbox{Point} & A & B & C & D & \mbox{Result}\\ \hline
(0,0) & 0 & -1 & 0 & 1 & \mbox{Saddle point}\\
(1,1) & 2e^{-1} & 0 & 2e^{-1} & -4e^{-2} & \mbox{Local minimum}\\
(1,-1) & -2e^{-1} & 0 & -2e^{-1} & -4e^{-2} & \mbox{Local maximum}\\
(-1,1) & -2e^{-1} & 0 & -2e^{-1} & -4e^{-2} & \mbox{Local maximum}\\
(-1,-1) & 2e^{-1} & 0 & 2e^{-1} & -4e^{-2} & \mbox{Local minimum}\\ \hline
\end{array}\]

Theorem. Suppose that ${\bf x}_{0}$ maximizes (or minimizes) $f({\bf x})$ subject to the side condition $g({\bf x})=0$. Then $\nabla f({\bf x}_{0})$ and $\nabla g({\bf x}_{0})$ are parallel. In other words, if $\nabla g({\bf x}_{0})\neq {\bf 0}$, then there exists a scalar $\lambda$ satisfying$\nabla f({\bf x}_{0})=\lambda\nabla g({\bf x}_{0})$. Such a scalar $\lambda$ is called a Lagrange multiplier. $\sharp$

Example. Maximize or minimize $f(x,y)=xy$ on the unit circle $x^{2}+y^{2}=1$. We set $g(x,y)=x^{2}+y^{2}-1$. The gradients are
\[\nabla f(x,y)=(y,x)\mbox{ and }\nabla g(x,y)=(2x,2y).\]
Setting $\nabla f(x,y)=\lambda\nabla g(x,y)$, we obtain $y=2\lambda x$ and $x=2\lambda y$. Multiplying the first equation by $y$ and the second equation by $x$, we find $y^{2}=2\lambda xy=x^{2}$. The side condition $x^{2}+y^{2}=1$ implies $2x^{2}=1$. Therefore, we obtain $x=\pm\frac{1}{2} \sqrt{2}=y$. The only points that can rise to an extreme value are $(\frac{1}{2}\sqrt{2},\frac{1}{2}\sqrt{2})$, $(\frac{1}{2}\sqrt{2},-\frac{1}{2}\sqrt{2})$, $(-\frac{1}{2}\sqrt{2},\frac{1}{2}\sqrt{2})$ and $(-\frac{1}{2}\sqrt{2},-\frac{1}{2}\sqrt{2})$. At the first and fourth points, the function $f$ takes on the value $1/2$. At the second and the third points, the function $f$ takes on the value $-1/2$. Therefore, we conclude that $1/2$ is the maximum value, and that $-1/2$ the minimum value. $\sharp$

Example. Find the minimum value taken on by the function $f(x,y)=x^{2}+(y-2)^{2}$ on the hyperbola $x^{2}-y^{2}=1$. Set $g(x,y)=x^{2}-y^{2}-1$. We have
\[\nabla f(x,y)=(2x,2(y-2))\mbox{ and }\nabla g(x,y)=(2x,-2y).\]
The Lagrange condition $\nabla f(x,y)=\lambda\nabla g(x,y)$ gives $2x=2\lambda x$ and $2(y-2)=-2\lambda y$. The side condition $x^{2}-y^{2}=1$ shows that $x$ cannot be zero. We also get $\lambda =1$. This means that $y-2=-y$, i.e., $y=1$. With $y=1$, the side condition gives $x=\pm\sqrt{2}$. The points to be checked are $(-\sqrt{2},1)$ and $(\sqrt{2},1)$. At each of these points $f$ takes on the value $3$. $\sharp$

Example. Maximize $f(x,y,z)=xyz$ subject to the side condition $x^{3}+y^{3}+z^{3}=1$ with $x\geq 0,y\geq 0,z\geq 0$. Set $g(x,y,z)=x^{3}+y^{3}+z^{3}-1$. The gradients are
\[\nabla f(x,y,z)=(yz,xz,xy)\mbox{ and }\nabla g(x,y,z)=(3x^{2},3y^{2},3z^{2}).\]
The Lagrange condition $\nabla f(x,y,z)=\lambda\nabla g(x,y,z)$ gives $yz=3\lambda x^{2}, xz=3\lambda y^{2}, xy=3\lambda z^{2}$. Multiplying the first equation by $x$, the second equation by $y$, and the third equation by $z$, we get $xyz=3\lambda x^{3}=3\lambda y^{3}=3\lambda z^{3}$. If $\lambda =0$, then $x,y,$ or $z$ would have to be zero, which would force $xyz$ to be $0$. Since $0$ is obviously not a maximum., having excluded $\lambda =0$, we can divide by $\lambda$ to get $x^{3}=y^{3}=z^{3}$, i.e., $x=y=z$. The side condition $x^{3}+y^{3}+z^{3}=1$ now gives $x=1/\sqrt[3]{3}=y=z$. Therefore, the desired maximum is $1/3$. $\sharp$