Jan Frederik Pieter Portielje (1829-1908 ) was a Dutch-Belgian painter.
We have sections
\begin{equation}{\label{a}}\tag{A}\mbox{}\end{equation}
Inner Product.
In the vector space, we can endow a structure to consider the distance between any two vectors or length of a vector. This is the so-called inner product space. Also, the geometric notions such as angle and perpendicularity in Euclidean space \(\mathbb{R}^{n}\) can be extended to the more general vector space via the inner product structure.
The scalar field \({\cal F}\) is assumed to be the field of real number \(\mathbb{R}\) or the field of complex number \(\mathbb{C}\). For any \(\alpha\in\mathbb{C}\), the complex conjugate of \(\alpha\) is denoted by \(\bar{\alpha}\). In general, if we write \(\bar{\alpha}\) for any \(\alpha\in {\cal F}\), it will mean \(\bar{\alpha}=\alpha\) for \(\alpha\in\mathbb{R}\).
Definition. Let \(V\) be a vector space over the scalar field \({\cal F}\). An inner product on \(V\) is amapping defined on \(V\times V\) that assigns each pair of elements \(v,w\in V\) as a scalar in \({\cal F}\), denoted by \(\langle u,v\rangle\), such that the following conditions are satisfied.
- We have \(\langle u,v\rangle =\overline{\langle v,u\rangle}\) for any \(u,v\in V\).
- If \(u,v,w\in V\), then
\[\langle u+v,w\rangle =\langle u,w\rangle +\langle v,w\rangle .\] - If \(u,v\in V\) and \(\alpha\in\mathbb{C}\), then
\[\langle\alpha u,v\rangle =\alpha\langle u,v\rangle.\]
In this case, \((V,\langle\cdot ,\cdot\rangle )\) is also called an inner product space. The inner product is said to be positive definite when \(\langle v,v\rangle\geq 0\) and \(\langle v,v\rangle >0\) for \(v\neq\theta\). The inner product is also said to be non-degenerate when \(v\in V\) and \(\langle v,w\rangle =0\) for all \(w\in V\) imply \(v=\theta\). \(\sharp\)
By definition, the following properties are very useful for further discussion.
- We have
\begin{align*}\langle u,\alpha v\rangle & =\overline{\langle\alpha v ,u\rangle}
\\ & =\overline{\alpha\langle v,u\rangle}=\bar{\alpha}\overline{\langle v,u\rangle}
\\ & =\bar{\alpha}\langle u,v\rangle .\end{align*} - We have
\begin{align*}\langle u,v+w\rangle & =\overline{\langle v+w,u\rangle}\\ & =\overline{\langle v,u\rangle +\langle w,u\rangle}\\ & =
\overline{\langle v,u\rangle}+\overline{\langle w,u\rangle}\\ & =\langle u,v\rangle +\langle u,w\rangle .\end{align*} - We have
\begin{align*} \langle u,\theta\rangle & =\langle u,\theta +\theta\rangle\\ & =\langle u,\theta\rangle +\langle u,\theta\rangle ,\end{align*}
which implies
\[\langle u,\theta\rangle =\langle\theta ,u\rangle=\theta .\]
Example. Let \(V=\mathbb{C}^{n}\). Give any \(v=(\alpha_{1},\cdots ,\alpha_{n})\) and \(u=(\beta_{1},\cdots ,\beta_{n})\) in \(\mathbb{C}^{n}\), we define
\[\langle u,v\rangle =\alpha_{1}\bar{\beta_{1}}+\cdots +\alpha_{n}\bar{\beta_{n}}.\]
Then, we can show that this is an inner product on \(\mathbb{C}^{n}\). Moreover, this inner product is positive definite. \(\sharp\)
Example. Let \(V\) be the space of all continuous complex-valued functions defined on the unit interval \([0,1]\). For \(f,g\in V\), we define
\[\langle f,g\rangle =\int_{0}^{1}f(x)\cdot\overline{g(x)}dx.\]
Then, the properties of integral can show that this is an inner product. \(\sharp\)
Let \(V\) be an \(n\)-dimensional vector space over \(\mathbb{R}\), and let \(\{v_{1},\cdots ,v_{2}\}\) be any basis of \(V\). Given any \(x,y\in V\), there exist \(\alpha_{i},\beta_{i}\in\mathbb{R}\) for \(i=1,\cdots ,n\) satisfying
\[x=\alpha_{1}v_{1}+\cdots +\alpha_{n}v_{n}\mbox{ and }y=\beta_{1}v_{1}+\cdots +\beta_{n}v_{n}.\]
Then, we have
\begin{align*}
\langle x,y\rangle & =\left\langle\alpha_{1}v_{1}+\cdots +\alpha_{n}v_{n},\beta_{1}v_{1}+\cdots +\beta_{n}v_{n}\right\rangle\\
& =\sum_{i=1}^{n}\sum_{j=1}^{n}\alpha_{i}\beta_{j}\langle v_{i},v_{j}\rangle.
\end{align*}
Let \(V\) be a vector space over the scalar field \({\cal F}\) with an inner product satisfying \(\langle v,v\rangle =0\) for all \(v\in V\). Then, given any \(u,w\in V\), since
\[\langle u+w,u+w\rangle =\langle u,u\rangle +\langle w,w\rangle+2\langle u,w\rangle ,\]
we have
\begin{align*}
\langle u,w\rangle & =\frac{1}{2}\cdot\left (\langle u+w,u+w\rangle-\langle u,u\rangle -\langle w,w\rangle\right )\\
& =0\mbox{ (by the assumption \(\langle v,v\rangle =0\) for all \(v\in V\))}.
\end{align*}
In other words, if \(\langle v,v\rangle =0\) for all \(v\in V\), then \(\langle u,w\rangle =0\) for all \(u,w\in V\).
Definition. Let \(A\) be an \(m\times n\) matrix over the scalar field \({\cal F}\). The conjugate transpose of \(A\) is denoted by \(A^{*}\) whose entry \(a^{*}_{ij}\) is defined to be the complex conjugate \(\overline{a_{ji}}\) of entry \(a_{ji}\) of \(A\). \(\sharp\)
For example, if
\[A=\left [\begin{array}{cc}
i & 2+2i\\ 7 & 4-3i
\end{array}\right ]\]
then
\[A^{*}=\left [\begin{array}{cc}
-i & 7\\ 2-2i & 4+3i
\end{array}\right ].\]
It is obvious that if \({\cal F}=\mathbb{R}\), then \(A^{*}=A^{\top}\).
Example. Let \(V\) be vector space of all \(n\times n\) matrices over the scalar field. We define
\begin{equation}{\label{fgaex260}}\tag{1}
\langle A,B\rangle =\mbox{tr}(B^{*}A).
\end{equation}
Then, we can show that is an inner product. Moreover, it is positive definite, which is shown below
\begin{align*}\langle A,A\rangle & =\mbox{tr}(A^{*}A)\\ & =\sum_{i=1}^{n}\sum_{j=1}^{n}a^{*}_{ij}a_{ji}
\\ & =\sum_{i=1}^{n}\sum_{j=1}^{n}\overline{a_{ji}}a_{ji}\\ & =\sum_{i=1}^{n}\sum_{j=1}^{n}|a_{ji}|^{2},\end{align*}
which says that if \(A\) is not a zero matrix, then \(a_{ji}\neq 0\) for some \(j\). This shows that \(\langle A,A\rangle >0\) when \(A\) is not a zero matrix. The inner product defined in (\ref{fgaex260}) is also called the Frobenius inner product. \(\sharp\)
Let \(V\) be a vector space over the scalar field \({\cal F}\) with a positive definite inner product. We define the length or norm of an element \(v\in V\) by
\[\parallel v\parallel =\sqrt{\langle v,v\rangle}.\]
For any \(\alpha\in {\cal F}\), we have
\begin{align*} \parallel\alpha v\parallel & =\sqrt{\langle\alpha v,\alpha v\rangle}\\ & =\sqrt{c^{2}\langle v,v\rangle}\\ & =|c|\cdot\parallel v\parallel .\end{align*}
We say that \(v\in V\) is a unit vector if and only if \(\parallel v\parallel =1\). If \(v\in V\) with \(v\neq\theta\), then we see that \(v/\parallel v\parallel\) is a unit vector. Let \(V\) be a vector space over the scalar field \({\cal F}\) with a positive definite inner product. Then, we have the following observations:
- Given any \(v\in V\), \(\parallel v\parallel =0\) if and only if \(v=\theta\).
- Given any \(\alpha\in {\cal F}\) and \(v\in V\), we have
\[\parallel\alpha v\parallel=|\alpha |\cdot\parallel v\parallel .\]
Proposition. Let \(V\) be a vector space over the scalar field \({\cal F}\) with a positive definite inner product.
(i) (Pythagoras Theorem). If \(u\) and \(v\) are perpendicular, then
\[\parallel u+v\parallel^{2}=\parallel u\parallel^{2}+\parallel v\parallel^{2}.\]
(ii) (Parallelogram Law). If \(u\) and \(v\) are perpendicular, then
\[\parallel u+v\parallel^{2}+\parallel u-v\parallel^{2}=2\parallel u\parallel^{2}+2\parallel v\parallel^{2}.\]
Proof. To prove part (i), we have
\begin{align*}
\parallel u+v\parallel^{2} & =\langle u+v,u+v\rangle=\langle u,u\rangle +\langle u,v\rangle +\langle v,u\rangle +\langle v,v\rangle\\
& =\langle u,u\rangle +2|\langle u,v\rangle |+\langle v,v\rangle =\parallel u\parallel^{2}+\parallel v\parallel^{2}.
\end{align*}
Theorem. Let \(V\) be a vector space over the scalar field \({\cal F}\) with a positive definite inner product. For any \(u,v\in V\), we have
\begin{equation}{\label{laeq24}}\tag{2}
\left |\langle u,v\rangle\right |\leq\parallel u\parallel\cdot\parallel v\parallel ,
\end{equation}
which is called the Schwartz inequality.
Proof. If \(v=\theta\), then the inequality holds true obviously. Now, we assume that \(v=e\) is a unit vector. We define \(\alpha =\langle u,e\rangle\). Then we have
\begin{align*} \langle u-\alpha e,\alpha e\rangle & =\langle u,\alpha e\rangle -\langle\alpha e,\alpha e\rangle\\ & =\bar{\alpha}\langle u,e\rangle -\bar{\alpha}\alpha\langle e,e\rangle\\ & =\bar{\alpha}\alpha- \bar{\alpha}\alpha =0,\end{align*}
which says that \(u-\alpha e\) is perpendicular to \(\alpha e\). Using the Pythagoras theorem, we obtain
\begin{align*} \parallel u\parallel^{2} & =\parallel u-\alpha e\parallel^{2}+\parallel\alpha e\parallel^{2}\\ & =\parallel u-\alpha e\parallel^{2}+\alpha^{2},\end{align*}
which says \(\alpha^{2}\leq\parallel u\parallel^{2}\), i.e.,
\begin{equation}{\label{laeq19}}\tag{3}
|\alpha |\leq\parallel u\parallel .
\end{equation}
In general, for \(v\neq\theta\), let \(e=v/\parallel v\parallel\). Then \(e\) is an unit vector. According to (\ref{laeq19}), we have
\begin{align*} \parallel u\parallel\geq |\alpha | & =\left |\langle u,e\rangle\right |\\ & =\left |\left\langle u,\frac{v}{\parallel v\parallel}\right\rangle\right |,\end{align*}
which implies
\[\left |\langle u,v\rangle\right |\leq\parallel u\parallel\cdot\parallel v\parallel .\]
This completes the proof. \(\blacksquare\)
Theorem. Let \(V\) be a vector space over the scalar field \({\cal F}\) with a positive definite inner product. For any \(u,v\in V\), we have the triangle inequality
\[\parallel u+v\parallel\leq\parallel u\parallel +\parallel v\parallel .\]
Proof. We have
\begin{align*}
\parallel u+v\parallel^{2} & =\langle u+v,u+v\rangle\\
& =\langle u,u\rangle +\langle u,v\rangle +\langle v,u\rangle +\langle v,v\rangle\\
& =\parallel u\parallel^{2}+\parallel v\parallel^{2}+\langle u,v\rangle +\overline{\langle u,v\rangle}\\
& \leq\parallel u\parallel^{2}+\parallel v\parallel^{2}+2\left |\langle u,v\rangle\right |\\
& \leq\parallel u\parallel^{2}+\parallel v\parallel^{2}+2\parallel u\parallel\cdot\parallel v\parallel\mbox{ (by Schwartz inequality)}\\
& =\left (\parallel u\parallel +\parallel v\parallel\right )^{2}.
\end{align*}
This completes the proof. \(\blacksquare\)
Proposition. Let \(V\) be a vector space over the scalar field \({\cal F}\) with a positive definite inner product. Let \(v_{1},\cdots ,v_{n}\) be elements in \(V\) such that \(\parallel v_{i}\parallel\neq 0\) for all \(i\) and \(\langle v_{i},v_{j}\rangle =0\) for all \(i\neq j\). Given any \(u\in V\), we define
\[c_{i}=\frac{\langle u,v_{i}\rangle}{\langle v_{i},v_{i}\rangle }.\]
Given any numbers \(\alpha_{1},\cdots ,\alpha_{n}\) in \({\cal F}\), we have the following closest approximation to \(u\) as a linear combination of \(v_{1},\cdots ,v_{n}\):
\[\left |\!\left |u-\sum_{k=1}^{n}c_{k}v_{k}\right |\!\right |\leq\left |\!\left |u-\sum_{k=1}^{n}\alpha_{k}v_{k}\right |\!\right |.\]
Proof. Since \(\langle v_{i},v_{j}\rangle =0\) for all \(i\neq j\), we have
\begin{align*} \left\langle u-\sum_{k=1}^{n}c_{k}v_{k},v_{i}\right\rangle & =\langle u,v_{i}\rangle -\sum_{k=1}^{n}c_{k}\cdot\langle v_{k},v_{i}\rangle\\ & =\langle u,v_{i}\rangle -c_{i}\cdot\langle v_{i},v_{i}\rangle =0,\end{align*}
which says that \(u-\sum_{k=1}^{n}c_{k}v_{k}\) is perpendicular to each \(v_{i}\). Therefore, it is also perpendicular to any linear combination of \(v_{1},\cdots ,v_{n}\). Using the Pythagoras theorem, we obtain
\begin{align*}
\left |\!\left |u-\sum_{k=1}^{n}\alpha_{k}v_{k}\right |\!\right |^{2} & =\left |\!\left |u-\sum_{k=1}^{n}\alpha_{k}v_{k}+\sum_{k=1}^{n}\left (c_{k}-\alpha_{k}\right )v_{k}\right |\!\right |^{2}\\
& =\left |\!\left |u-\sum_{k=1}^{n}c_{k}v_{k}\right |\!\right |^{2}+\left |\!\left |u-\sum_{k=1}^{n}\left (c_{k}-\alpha_{k}\right )v_{k}\right |\!\right |^{2}\\
& \geq\left |\!\left |u-\sum_{k=1}^{n}c_{k}v_{k}\right |\!\right |^{2}.
\end{align*}
This completes the proof. \(\sharp\)
Proposition. (Bessel Inequality). Let \(V\) be a vector space over the scalar field \({\cal F}\) with a positive definite inner product. Let \(v_{1},\cdots ,v_{n}\) be elements in \(V\) such that \(\parallel v_{i}\parallel =1\) for all \(i\) and \(\langle v_{i},v_{j}\rangle =0\) for all \(i\neq j\). Given any \(u\in V\), we define \(c_{i}=\langle u,v_{i}\rangle\). Then, we have
\[\sum_{i=1}^{n}c_{i}^{2}\leq\parallel u\parallel^{2}.\]
Proof. We have
\begin{align*}
0 & \leq\left\langle u-\sum_{i=1}^{n}c_{i}v_{i},u-\sum_{i=1}^{n}c_{i}v_{i}\right\rangle\\
& =\langle u,u\rangle -\sum_{i=1}^{n}2c_{i}\cdot\langle u,v_{i}\rangle +\sum_{i=1}^{n}c_{i}^{2}\\
& =\langle u,u\rangle -\sum_{i=1}^{n}2c_{i}^{2}+\sum_{i=1}^{n}c_{i}^{2}\\
& =\parallel u\parallel^{2}-\sum_{i=1}^{n}c_{i}^{2}.
\end{align*}
This completes the proof. \(\blacksquare\)
\begin{equation}{\label{b}}\tag{B}\mbox{}\end{equation}
Orthogonal Bases.
In the Euclidean space \(\mathbb{R}^{n}\), we recall
\[\langle {\bf x},{\bf y}\rangle =\parallel {\bf x}\parallel\cdot\parallel {\bf y}\parallel\cdot\cos\theta ,\]
where \(\theta\) denotes the angle between the vectors \({\bf x}\) and \({\bf y}\). It is also obvious that \({\bf x}\) and \({\bf y}\) are perpendicular if and only if \(\cos\theta =0\). Equivalently, \({\bf x}\) and \({\bf y}\) are perpendicular if and only if \(\langle {\bf x},{\bf y}\rangle =0\). Now, we are going to extend the concept of perpendicularity to the inner product space.
Definition. Let \((V,\langle\cdot ,\cdot\rangle )\) be an inner product space over the scalar field \({\cal F}\). Given any \(u,v\in V\), we say that \(u\) and \(v\) are orthogonal (or perpendicular) when \(\langle u,v\rangle =0\). In this case, we write \(u\perp v\). \(\sharp\)
Let \(S\) be a subset of a vector space \(V\). We define
\[S^{\perp}=\left\{v\in V:\langle v,w\rangle =0\mbox{ for all }w\in S\right\}.\]
Then, we can show that \(S^{\perp}\) is a subspace of \(V\). We call \(S^{\perp}\) the orthogonal space of \(S\). If \(w\) is perpendicular to \(S\), we also write \(w\perp S\).
Definition. Let \(V\) be a vector space over the scalar field \({\cal F}\) with an inner product, and let \(\{v_{1},\cdots ,v_{n}\}\) be a basis of \(V\). We shall says that it is an orthogonal basis when \(\langle v_{i},v_{j}\rangle =0\) for \(i\neq j\). The orthogonal basis is also said to be an orthonormal basis when \(\parallel v_{i}\parallel =1\) for all \(i=1,\cdots ,n\). \(\sharp\)
Example. Let \(V\) be an \(n\)-dimensional vector space over \(\mathbb{C}\), and let \(\{e_{1},\cdots ,e_{n}\}\) be an orthonormal basis of \(V\). Given any \(x,y\in V\), there exist \(\alpha_{i},\beta_{i}\in\mathbb{R}\) for \(i=1,\cdots ,n\) satisfying
\[x=\alpha_{1}e_{1}+\cdots +\alpha_{n}e_{n}\]
and
\[y=\beta_{1}e_{1}+\cdots +\beta_{n}e_{n}.\]
Then, we have
\begin{align*}
\langle x,y\rangle & =\left\langle\alpha_{1}e_{1}+\cdots +\alpha_{n}e_{n},\beta_{1}e_{1}+\cdots +\beta_{n}e_{n}\right\rangle\\
& =\sum_{i=1}^{n}\sum_{j=1}^{n}\alpha_{i}\bar{\beta_{j}}\langle e_{i},e_{j}\rangle\\
& =\alpha_{1}\bar{\beta_{1}}+\cdots +\alpha_{n}\bar{\beta_{n}},
\end{align*}
since \(\langle e_{i},e_{i}\rangle =1\) for \(i=1,\cdots ,n\) and \(\langle e_{i},e_{j}\rangle =0\) for \(i\neq j\) and \(i,j=1,\cdots ,n\). \(\sharp\)
\begin{equation}{\label{lac8}}\tag{4}\mbox{}\end{equation}
Lemma \ref{lac8}. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\) with \(\dim (V)=n\). Let \(r\in\mathbb{N}\) with \(r<n\), and let \(u_{1},\cdots ,u_{r}\in V\) be linearly independent. Then, there exist \(v_{r+1},\cdots ,v_{n}\in V\) such that \(\{u_{1},\cdots ,u_{r},v_{r+1},\cdots ,v_{n}\}\) forms a basis of \(V\). In other words, every linearly independent subset of \(V\) can be extended to a basis for \(V\).
The above Lemma \ref{lac8} can refer to the page Vector Spaces.
\begin{equation}{\label{lat20}}\tag{5}\mbox{}\end{equation}
Theorem \ref{lat20}. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\) with a positive definite inner product. Let \(W\) be a subspace of \(V\), and let \(\{w_{1},\cdots ,w_{m}\}\) be an orthogonal basis of \(W\). If \(W\neq V\), then there exists elements \(w_{m+1},\cdots ,w_{n}\) in \(V\) such that \(\{w_{1},\cdots ,w_{n}\}\) is an orthogonal basis of \(V\).
Proof. From Lemma \ref{lac8}, we see that there exist elements \(v_{m+1},\cdots ,v_{n}\) of \(V\) such that
\[\{w_{1},\cdots ,w_{m},v_{m+1},\cdots ,v_{n}\}\]
is a basis of \(V\). However, it is not necessary a orthogonal basis. We shall prove this theorem by induction. Let \(W_{m+1}\) be the space generated by \(w_{1},\cdots ,w_{m},v_{m+1}\). We shall obtain an orthogonal basis of \(W_{m+1}\). Let
\[c_{i}=\frac{\langle v_{m+1},w_{i}\rangle}{\langle w_{i},w_{i}\rangle}\]
for \(i=1,\cdots ,m\), and let
\[w_{m+1}=v_{m+1}-\sum_{i=1}^{m}c_{i}w_{i}.\]
Then \(\langle w_{m+1},w_{i}\rangle =0\) for all \(i=1,\cdots ,m\). If \(w_{m+1}=\theta\), then \(v_{m+1}\) is linearly dependent on \(w_{1},\cdots ,w_{m}\), which contradicts that \(w_{1},\cdots ,w_{m},v_{m+1}\) are linearly independent. Therefore, we must have \(w_{m+1}\neq\theta\). We also see that \(v_{m+1}\) is in the space generated by \(w_{1},\cdots ,w_{m},w_{m+1}\), since
\[v_{m+1}=w_{m+1}+\sum_{i=1}^{m}c_{i}w_{i},\]
which says that \(\{w_{1},\cdots ,w_{m},w_{m+1}\}\) is an orthogonal basis of \(W_{m+1}\). By induction, we can show that the space \(w_{m+s}\) generated by \(w_{1},\cdots ,w_{m},v_{m+1},\cdots ,v_{m+s}\) has an orthogonal basis
\[\{w_{1},\cdots ,w_{m},w_{m+1},\cdots ,w_{m+s}\}\]
for \(s=1,\cdots ,n=m\). This completes the proof. \(\blacksquare\)
\begin{equation}{\label{lac22}}\tag{6}\mbox{}\end{equation}
Corollary \ref{lac22}. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\) with a positive definite inner product. If \(V\neq\{\theta\}\), then \(V\) has an orthogonal basis.
Proof. Since \(V\neq\{\theta\}\), there exists an element \(v_{1}\) of \(V\) satisfying \(v_{1}\neq\theta\). Let \(W\) be the subspace generated by \(v_{1}\). Then, the desired basis can be obtained by Theorem \ref{lat20} immediately. \(\blacksquare\)
Given an arbitrary basis \(\{v_{1},\cdots ,v_{n}\}\) of \(V\), according to the proof of Theorem \ref{lat20}, we can obtain an orthogonal basis as follows. Let
\begin{align*}
v’_{1} & =v_{1}\\
v’_{2} & =v_{2}-\left (\frac{\langle v_{2},v’_{1}\rangle}{\langle v’_{1},v’_{1}\rangle}\right )v’_{1}\\
v’_{3} & =v_{3}-\left (\frac{\langle v_{3},v’_{2}\rangle}{\langle v’_{2},v’_{2}\rangle}\right )v’_{2}-\left (\frac{\langle v_{3},v’_{1}\rangle}{\langle v’_{1},v’_{1}\rangle}\right )v’_{1}\\
& \vdots\\
v’_{n} & =v_{n}-\left (\frac{\langle v_{n},v’_{n-1}\rangle}{\langle v’_{n-1},v’_{n-1}\rangle}\right )v’_{n-1}-\cdots -\left (\frac{\langle v_{n},v’_{1}\rangle}{\langle v’_{1},v’_{1}\rangle}\right )v’_{1}.
\end{align*}
Then \(\{v’_{1},\cdots ,v’_{n}\}\) is an orthogonal basis of \(V\). This procedure is called the Gram-Schimidt orthogonalization process. Also, given an orthogonal basis, we can always obtain an orthonormal basis by dividing each vector by its length.
Example. Given three vectors \(v_{1}=(1,1,0,1)\), \(v_{2}=(1,-2,0,0)\) and \(v_{3}=(1,0,-1,2)\) in \(\mathbb{R}^{4}\), we shall find an orthonormal basis for the vector subspace generated by \(v_{1},v_{2},v_{3}\). Let \(v’_{1}=v_{1}=(1,1,0,1)\). Then, we have
\[v’_{2}=v_{2}-\left (\frac{\langle v_{2},v’_{1}\rangle}{\langle v’_{1},v’_{1}\rangle}\right )v’_{1}=\left (\frac{4}{3},-\frac{5}{3},0,\frac{1}{3}\right )\]
and
\begin{align*} v’_{3} & =v_{3}-\left (\frac{\langle v_{3},v’_{2}\rangle}{\langle v’_{2},v’_{2}\rangle}\right )v’_{2}-\left (\frac{\langle v_{3},v’_{1}\rangle}
{\langle v’_{1},v’_{1}\rangle}\right )v’_{1}\\ & =\left (-\frac{4}{7},-\frac{2}{7},-\frac{1}{7},\frac{6}{7}\right ).\end{align*}
Finally, we obtain
\begin{align*}
v”_{1} & =\frac{v’_{1}}{\parallel v’_{1}\parallel}=\frac{1}{\sqrt{3}}(1,1,0,1)\\
v”_{2} & =\frac{v’_{2}}{\parallel v’_{2}\parallel}=\frac{1}{\sqrt{42}}(4.-5,0,1)\\
v”_{3} & =\frac{v’_{3}}{\parallel v’_{3}\parallel}=\frac{1}{\sqrt{57}}(-4,-2,-1,6)
\end{align*}
as an orthonormal basis. \(\sharp\)
\begin{equation}{\label{lat21}}\tag{7}\mbox{}\end{equation}
Theorem \ref{lat21}. Let \(V\) be an \(n\)-dimensional vector space over the scalar field \({\cal F}\) with a positive definite inner product, and let \(W\) be a subspace of \(V\). Then, we have
\[V=W\oplus W^{\perp}\mbox{ and }\dim (V)=\dim (W)+\dim (W)^{\perp}.\]
Proof. If \(W=\{\theta\}\) or \(W=V\), then the results are obvious. Therefore, we assume that \(W\neq\{\theta\}\) and \(W\neq V\). Let \(\{w_{1},\cdots ,w_{r}\}\) be an orthonormal basis in which the existence is guaranteed by Corollary \ref{lac22}. Using Theorem \ref{lat20}, there exists elements \(v_{r+1},\cdots ,v_{n}\) of \(V\) such that
\[\{w_{1},\cdots ,,w_{r},v_{r+1},\cdots ,v_{n}\}\]
is an orthonormal basis of \(V\). We shall prove that \(\{v_{r+1},\cdots ,v_{n}\}\) is an orthonormal basis of \(W^{\perp}\). Given any \(u\in W^{\perp}\), there exist \(\alpha_{1},\cdots ,\alpha_{n}\in {\cal F}\) satisfying
\[u=\alpha_{1}w_{1}+\cdots +\alpha_{r}w_{r}+\alpha_{r+1}v_{r+1}+\cdots +\alpha_{n}v_{n}.\]
Since \(u\) is perpendicular to \(W\), we have
\[0=\langle u,w_{i}\rangle =\alpha_{i}\langle w_{i},w_{i}\rangle =\alpha_{i}\cdot\parallel w_{i}\parallel^{2}=\alpha_{i}\]
for \(i=1,\cdots ,r\). This shows that \(u\) is actually a linear combination of \(\{v_{r+1},\cdots ,v_{n}\}\). Conversely, let
\[u=\alpha_{r+1}v_{r+1}+\cdots +\alpha_{n}v_{n}\]
be a linear combination of \(\{v_{r+1},\cdots ,v_{n}\}\). Then we obtain \(\langle u,w_{i}\rangle =0\) for \(i=1,\cdots ,r\), since \(\{w_{1},\cdots ,w_{r},v_{r+1},\cdots ,v_{n}\}\) is an orthonormal basis of \(V\). Therefore, we see that \(u\) is perpendicular to \(W\), i.e., \(u\in W^{\perp}\). This proves that \(\{v_{r+1},\cdots ,v_{n}\}\) generates \(W^{\perp}\). Since the elements in the set \(\{v_{r+1},\cdots ,v_{n}\}\) are mutually perpendicular with \(\parallel v_{j}\parallel =1\) for \(j=r+1,\cdots ,n\), they form a orthonormal basis of \(W^{\perp}\). Since an element \(u\) of \(V\) has a unique expression as a linear combination
\[u=\left (\alpha_{1}w_{1}+\cdots +\alpha_{r}w_{r}\right )+\left (\alpha_{r+1}v_{r+1}+\cdots +\alpha_{n}v_{n}\right )\equiv w+v,\]
where \(w\in W\) and \(v\in W^{\perp}\), this says that \(V\) is a direct sum of \(W\) and \(W^{\perp}\). This completes the proof. \(\blacksquare\)
The space \(W^{\perp}\) in Theorem \ref{lat21} is called the orthogonal complement of \(W\).
Example. Let \(V\) be an \(n\)-dimensional vector space over \(\mathbb{R}\), and let \(\{e_{1},\cdots ,e_{n}\}\) be an orthonormal basis of \(V\). Given any \(x,y\in V\), there exist \(\alpha_{i},\beta_{i}\in\mathbb{R}\) for \(i=1,\cdots ,n\) satisfying
\[x=\alpha_{1}e_{1}+\cdots +\alpha_{n}e_{n}\]
and
\[y=\beta_{1}e_{1}+\cdots +\beta_{n}e_{n}.\]
Then, we have
\begin{align*}
\langle x,y\rangle & =\left\langle\alpha_{1}e_{1}+\cdots +\alpha_{n}e_{n},\beta_{1}e_{1}+\cdots +\beta_{n}e_{n}\right\rangle\\
& =\sum_{i=1}^{n}\sum_{j=1}^{n}\alpha_{i}\beta_{j}\langle e_{i},e_{j}\rangle\\
& =\alpha_{1}\beta_{1}+\cdots +\alpha_{n}\beta_{n},
\end{align*}
since \(\langle e_{i},e_{i}\rangle =1\) for \(i=1,\cdots ,n\) and \(\langle e_{i},e_{j}\rangle =0\) for \(i\neq j\) and \(i,j=1,\cdots ,n\). \(\sharp\)
Suppose that we consider the inner product space \((\mathbb{C}^{2},\langle\cdot ,\cdot\rangle )\) over \(\mathbb{C}\) with the standard inner product defined by
\[\langle (u_{1},u_{2}),(v_{1},v_{2})\rangle =u_{1}v_{1}+u_{2}v_{2}.\]
Then, we have
\[\langle (1,i),(1,i)\rangle =1+i^{2}=0,\]
which says that this inner product is not positive definite. Next, we want to show that Corollary \ref{lac22} still holds true if the inner product is not assumed to be positive definite.
Theorem. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\). If \(V\neq\{\theta\}\), then \(V\) has an orthogonal basis.
\begin{equation}{\label{c}}\tag{C}\mbox{}\end{equation}
Dual Spaces.
Let \(V\) be a vector space over the scalar field \({\cal F}\). We denote by \(V^{*}\) the set of all linear mappings defined on \(V\) into \({\cal F}\). Given any \(f,g\in V^{*}\) and \(\alpha\in {\cal F}\), the addition and scalar multiplication are defined by
\[(f+g)(x)=f(x)+g(x)\mbox{ and }(\alpha f)=\alpha f(x).\]
We have known that the mappings \(f+g\) and \(\alpha f\) are also linear.
Proposition. \(V^{*}\) is a vector space over the scalar field \({\cal F}\). \(\sharp\)
The elements of \(V^{*}\) will be called the linear functionals on \(V\), and \(V^{*}\) is called the dual space of \(V\).
Example. Let \(V={\cal F}^{n}\). The projection \(\phi_{i}:{\cal F}^{n}\rightarrow {\cal F}\) defined by
\[\phi_{i}(x_{1},\cdots ,x_{n})=x_{i}\]
is a linear functional on \({\cal F}^{n}\) for \(i=1,\cdots ,n\). \(\sharp\)
Example. Let \(V\) be a vector space over the scalar field \({\cal F}\) with an inner product. Given any fixed element \(v_{0}\) in \(V\), the mapping \(f:V\rightarrow\mathbb{R}_{+}\) defined by
\[f(v)=\langle v_{0},v\rangle =\langle v,v_{0}\rangle\]
is a linear functional on \(V\). Now, we assume that \(V\) is a vector space over \(\mathbb{C}\) with a Hermitian product. Then, the mapping defined by
\[v\mapsto\langle v,v_{0}\rangle\]
is also a linear functional on \(V\). However, the mapping
\[v\mapsto\langle v_{0},v\rangle\]
cannot be a linear functional, since
\[\langle v_{0},\alpha v\rangle =\bar{\alpha}\langle v_{0},v\rangle .\]
\begin{equation}{\label{lat10}}\tag{8}\mbox{}\end{equation}
Lemma \ref{lat10}. Let \(V\) be an \(n\)-dimensional vector spaces over the scalar field \({\cal F}\), and let \(\{v_{1},v_{2},\cdots ,v_{n})\) be a basis of \(V\). Let \(W\) be another vector space over the same scalar field \({\cal F}\), and let \(w_{1},w_{2},\cdots ,w_{n}\) be any elements in \(W\). Then, there exists a unique linear mapping \(T:V\rightarrow W\) such that \(T(v_{i})=w_{i}\) for \(i=1,\cdots ,n\).
Proof. Given any element \(v\in V\), there exist \(\alpha_{1},\cdots ,\alpha_{n}\in {\cal F}\) satisfying
\[x=\alpha_{1}v_{1}+\alpha_{2}v_{2}+\cdots +\alpha_{n}v_{n}.\]
Therefore, we can define a mapping \(T:V\rightarrow W\) by
\[T(v)=T\left (\alpha_{1}v_{1}+\alpha_{2}v_{2}+\cdots +\alpha_{n}v_{n}\right )
=\alpha_{1}w_{1}+\alpha_{2}w_{2}+\cdots +\alpha_{n}w_{n}\in W.\]
We are going to show that \(T\) is linear.
Let \(u\) be another element of \(V\). There exist \(\beta_{1},\cdots ,\beta_{n}\in {\cal F}\) satisfying
\[u=\beta_{1}v_{1}+\beta_{2}v_{2}+\cdots +\beta_{n}v_{n}.\]
Therefore, we have
\[v+u=(\alpha_{1}+\beta_{1})v_{1}+(\alpha_{2}+\beta_{2})v_{2}+\cdots +(\alpha_{n}+\beta_{n})v_{n}.\]
By the definition of \(T\), we obtain
\begin{align*}
T(v+u) & =(\alpha_{1}+\beta_{1})w_{1}+(\alpha_{2}+\beta_{2})w_{2}+\cdots +(\alpha_{n}+\beta_{n})w_{n}\\
& =\left (\alpha_{1}w_{1}+\alpha_{2}w_{2}+\cdots +\alpha_{n}w_{n}\right )+\left (\beta_{1}w_{1}+\beta_{2}w_{2}+\cdots +\beta_{n}w_{n}\right )\\
& =T(v)+T(y).
\end{align*}
Given any \(\alpha\in {\cal F}\), we have
\[\alpha v=(\alpha\alpha_{1})v_{1}+(\alpha\alpha_{2})v_{2}+\cdots +(\alpha\alpha_{n})v_{n}.\]
By the definition of \(T\), we obtain
\begin{align*}
T(\alpha v) & =(\alpha\alpha_{1})w_{1}+(\alpha\alpha_{2})w_{2}+\cdots +(\alpha\alpha_{n})w_{n}\\
& =\alpha\left (\alpha_{1}w_{1}+\alpha_{2}w_{2}+\cdots +\alpha_{n}w_{n}\right )=\alpha T(v).
\end{align*}
It is easy to see that this mapping satisfies \(T(v_{i})=w_{i}\) for \(i=1,\cdots ,n\). Next, we want to claim that this mapping is unique. Suppose there exists another linear mapping \(\widehat{T}\) satisfying \(\widehat{T}(v_{i})=w_{i}\) for \(i=1,\cdots ,n\). Then, we have
\begin{align*}
\widehat{T}(v) & =\widehat{T}\left (\alpha_{1}v_{1}+\alpha_{2}v_{2}+\cdots +\alpha_{n}v_{n}\right )\\
& =\alpha_{1}\widehat{T}(v_{1})+\alpha_{2}\widehat{T}(v_{2})+\cdots +\alpha_{n}\widehat{T}(v_{n})\\
& =\alpha_{1}w_{1}+\alpha_{2}w_{2}+\cdots +\alpha_{n}w_{n}\\
& =T(v).
\end{align*}
This shows that \(\widehat{T}=T\), and the proof is complete. \(\blacksquare\)
The above Lemma \ref{lat10} can refer to the page Linear Mappings.
\begin{equation}{\label{lat25}}\tag{9}\mbox{}\end{equation}
Theorem \ref{lat25}. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\) with an inner product. Then, we have \(\dim (V)^{*}=\dim (V)\). Moreover, if \(\{v_{1},\cdots ,v_{n}\}\) is a basis of \(V\), then \(\{v_{1}^{*},\cdots ,v_{n}^{*}\}\) is a basis of \(V^{*}\) defined by
\[v_{i}^{*}(v_{j})=\left\{\begin{array}{ll}
1 & \mbox{if \(i=j\)}\\
0 & \mbox{if \(i\neq j\)}.
\end{array}\right .\]
Proof. Let \(\{v_{1},\cdots ,v_{n}\}\) be a basis of \(V\). We shall find a basis of \(V^{*}\). Lemma \ref{lat10} says that we can find a linear mapping having prescribed values on basis elements. Therefore, there exists a linear function \(v_{i}^{*}\) for \(i=1,\cdots ,n\) satisfying
\[v_{i}^{*}(v_{j})=\left\{\begin{array}{ll}
1 & \mbox{if \(i=j\)}\\
0 & \mbox{if \(i\neq j\)}.
\end{array}\right .\]
We are going to claim that \(\{v_{1}^{*},\cdots ,v_{n}^{*}\}\) is a basis of \(V^{*}\). Given any \(\phi\in V^{*}\), let \(c_{i}=\phi (v_{i})\). Now, we have
\[\left (c_{1}v_{1}^{*}+\cdots +c_{n}v_{n}^{*}\right )(v_{i})=c_{i}\cdot v_{i}^{*}(v_{i})=c_{i}.\]
This says that \(\phi\) and \(c_{1}v_{1}^{*}+\cdots +c_{n}v_{n}^{*}\) have the same values on all elements of the basis \(\{v_{1},\cdots ,v_{n}\}\), which also means that they have the same values on all linear combination of these basis elements. In other words, they have the same values on \(V\). Therefore, we obtain
\[\phi =c_{1}v_{1}^{*}+\cdots +c_{n}v_{n}^{*},\]
i.e., the set \(\{v_{1}^{*},\cdots ,v_{n}^{*}\}\) generates \(V^{*}\). Next, we are going to show that the elements \(v_{1}^{*},\cdots ,v_{n}^{*}\) are linearly independent. Suppose that
\[\alpha_{1}v_{1}^{*}+\cdots +\alpha_{n}v_{n}^{*}=\theta_{V^{*}},\]
where \(\alpha_{i}\in {\cal F}\) for \(i=1,\cdots ,n\). Then, we have
\begin{align*} 0 & =\theta_{V^{*}}(v_{i})\\ & =\left (\alpha_{1}v_{1}^{*}+\cdots +\alpha_{n}v_{n}^{*}\right )(v_{i})\\ & =\alpha_{i}\cdot v_{i}^{*}(v_{i})=\alpha_{i}\end{align*}
for all \(i=1,\cdots ,n\). This completes the proof. \(\blacksquare\)
The basis \(\{v_{1}^{*},\cdots ,v_{n}^{*}\}\) of \(V\) as shown in Theorem \ref{lat25} is called a dual basis of \(\{v_{1},\cdots ,v_{n}\}\). Let \(V\) be a vector space over the scalar field \({\cal F}\), and let \(S\) be a subset of \(V\). For \(\phi\in V^{*}\), we say that \(\phi\) is perpendicular to \(S\) if and only if \(\phi (s)=0\) for all \(s\in S\). We denote by \(S^{*\perp}\) the set of elements in \(V\) which are perpendicular to \(S\).
Proposition. Let \(V\) be a vector space over the scalar field \({\cal F}\), and let \(S\) be a subset of \(V\). Then \(S^{*\perp}\) is a subspace of \(V^{*}\). \(\sharp\)
\begin{equation}{\label{lat26}}\tag{10}\mbox{}\end{equation}
Lemma \ref{lat26}. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\) with \(\dim (V)=n\). If \(x_{1},\cdots ,x_{n}\) are linearly independent vectors of \(V\), then \(\{x_{1},\cdots ,x_{n}\}\) forms a basis of \(V\).
\begin{equation}{\label{lac161}}\tag{11}\mbox{}\end{equation}
Lemma \ref{lac161}. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\) with \(\dim (V)=n\), and let \(W\) be a subspace of \(V\) with \(\dim (W)=n\). Then, we have \(V=W\).
The above Lemmas \ref{lat26} and \ref{lac161} can refer to the page Vector Spaces.
\begin{equation}{\label{lat27}}\tag{12}\mbox{}\end{equation}
Theorem \ref{lat27}. Let \(V\) be a finite-dimensional vector space over the scalar field \({\cal F}\), and let \(W\) be a subspace of \(V\). Then, we have
\[\dim (V)=\dim (W)+\dim (W)^{*\perp}.\]
Proof. Let \(\{w_{1},\cdots ,w_{r}\}\) be a basis of \(W\). According to the Lemmas \ref{lat26} and \ref{lac161}, we can extend this basis to a basis \(\{w_{1},\cdots ,w_{r},v_{r+1},\cdots ,v_{n}\}\) of \(V\). Let \(\{w_{1}^{*},\cdots ,w_{r}^{*},v_{r+1}^{*},\cdots ,v_{n}^{*}\}\) be the dual basis. We are going to prove that \(\{v_{r+1}^{*},\cdots ,v_{n}^{*}\}\) is a basis of \(W^{*\perp}\). Since they are linearly independent, it suffices to prove that \(\{v_{r+1}^{*},\cdots ,v_{n}^{*}\}\) generates \(W^{*\perp}\). For \(\phi\in W^{*\perp}\), there exists \(\alpha_{i}\in {\cal F}\) for \(i=1,\cdots ,n\) satisfying
\[\phi =\alpha_{1}w_{1}^{*}+\cdots +\alpha_{r}w_{r}^{*}+\alpha_{r+1}v_{r+1}^{*}+\cdots +\alpha_{n}v_{n}^{*}.\]
Since \(\phi\in W^{*\perp}\), for \(i=1,\cdots ,r\), we also have
\[0=\phi (w_{i})=\alpha_{i}\cdot w_{i}^{*}(w_{i})=\alpha_{i}.\]
Therefore, we obtain
\[\phi =\alpha_{r+1}v_{r+1}^{*}+\cdots +\alpha_{n}v_{n}^{*},\]
which shows that \(W^{*\perp}\) is contained in the space generated by \(\{v_{r+1}^{*},\cdots ,v_{n}^{*}\}\). For the converse, if \(r+1\leq j\leq n\), then \(w_{j}^{*}(w_{i})=0\) for \(i=1,\cdots ,r\). This says that \(w_{j}^{*}\) is contained in \(W^{*\perp}\), which says that \(W^{*\perp}\) contains the space generated by \(\{v_{r+1}^{*},\cdots ,v_{n}^{*}\}\). This complete the proof. \(\blacksquare\)
Let \(V\) be a vector space over the scalar field \({\cal F}\) with an inner product. For each element \(v\in V\), we can associate a functional \(L_{v}\) in the dual space defined by \(L_{v}(w)=\langle v,w\rangle\) for all \(w\in V\). Therefore, we can obtain the following properties.
- If \(v_{1},v_{2}\in V\), then \(L_{v_{1}+v_{2}}=L_{v_{1}}+L_{v_{2}}\).
- If \(v\in V\) and \(\alpha\in {\cal F}\), then \(L_{\alpha v}=\alpha L_{v}\).
It follows that the mapping \(T:V\rightarrow V^{*}\) defined by \(T(v)=L_{v}\) is a linear mapping.
\begin{equation}{\label{lat44}}\tag{13}\mbox{}\end{equation}
Theorem \ref{lat44}. Let \(V\) be a finite-dimensional vector space over the scalar field with a non-degenerate inner product. Given a linear functional \(L\in V^{*}\) with \(L:V\rightarrow {\cal F}\), there exists a unique element \(v\in V\) satisfying \(L(w)=\langle v,w\rangle\) for all \(w\in V\). In other words, the mapping \(T:V\rightarrow V^{*}\) defined by \(T(v)=L_{v}\) is an isomorphism, and \(V\) and \(V^{*}\) are isomorphic.
Proof. We denote by \(\widehat{V}^{*}\) the set of all linear functionals on \(V\) which are of type \(L_{v}\) for some \(v\in V\). Then, we shall prove \(V^{*}=\widehat{V}^{*}\). Since the zero functional is in \(\widehat{V}^{*}\), and we have \(L_{v_{1}+v_{2}}=L_{v_{1}}+L_{v_{2}}\) and \(L_{\alpha v}=\alpha L_{v}\), it follows that \(\widehat{V}^{*}\) is a subspace of \(V^{*}\). Therefore, it remains to prove \(\dim (V)^{*}=\dim\widehat{V}^{*}\). Let \(\{v_{1},\cdots ,v_{n}\}\) be a basis of \(V\). We shall show that \(\{L_{v_{1}},\cdots ,L_{v_{n}}\}\) is a basis of \(\widehat{V}^{*}\). Let \(\theta^{*}\) be the zero functional. Suppose that
\[\alpha_{1}L_{v_{1}}+\cdots +\alpha_{n}L_{v_{n}}=\theta^{*}\]
for \(\alpha_{1},\cdots ,\alpha_{n}\in {\cal F}\), which also says
\[L_{\alpha_{1}v_{1}+\cdots +\alpha_{n}v_{n}}=\theta^{*}.\]
The non-degeneracy says that \(L_{v}=\theta^{*}\) implies \(v=\theta\). We obtain
\[\alpha_{1}v_{1}+\cdots +\alpha_{n}v_{n}=\theta ,\]
which also says \(\alpha_{i}=0\) for all \(i=1,\cdots ,n\). Therefore, the elements \(L_{v_{1}},\cdots ,L_{v_{n}}\) are linearly independent. It is also easy to realize that the set \(\{L_{v_{1}},\cdots ,L_{v_{n}}\}\) generates \(\widehat{V}^{*}\). Therefore, the set \(\{L_{v_{1}},\cdots ,L_{v_{n}}\}\) is indeed a basis of \(\widehat{V}^{*}\), which says that \(\dim (V)=\dim\widehat{V}^{*}\). Since \(\dim (V)^{*}=\dim (V)\) by Theorem \ref{lat25}, we conclude that \(\dim (V)^{*}=\dim\widehat{V}^{*}\). This completes the proof. \(\blacksquare\)
As an application of results concerning the dual space, we are going to obtain the analogue of Theorem \ref{lat21} for non-degenerate inner product.
Theorem. Let \(V\) be an \(n\)-dimensional vector space over the scalar field \({\cal F}\) with a non-degenerate inner product, and let \(W\) be a subspace of \(V\). Then, we have
\[\dim (V)=\dim (W)+\dim (W)^{\perp}.\]
Proof. We recall that \(v\in W^{\perp}\) if and only if \(\langle v,w\rangle\) for all \(w\in W\), and that \(\phi\in W^{*\perp}\) if and only if \(\phi (w)=0\) for all \(w\in W\). Therefore, we see that \(v\in W^{\perp}\) if and only if the linear functional \(L_{v}\) lies in \(W^{*\perp}\). In other words, the mapping \(v\mapsto L_{v}\) is an isomorphism between \(W^{\perp}\) and \(W^{*\perp}\). According to Theorems \ref{lat21} and \ref{lat27}, we complete the proof. \(\blacksquare\)
Eigenvectors of Symmetric Linear Mappings.
Let \(V\) be a finite-dimensional vector space over the real numbers with \(\dim (V)\geq 1\). We also assume that \(V\) is endowed with a symmetric positive definite inner product. Let \(T\) be a linear operator on \(V\). We say that \(T\) is symmetric when
\[\langle T(v),u\rangle =\langle v,A(u)\rangle =\langle A(u),v\rangle\]
for all \(u,v\in V\). We have the following observations
- Suppose that we take an orthonormal basis of \(V\). Let \(A\) be the matrix representing \(T\) with respect to this orthonormal basis. Then, the linear mapping \(T\) is symmetric if and only if the matrix \(A\) is symmetric.
- Suppose that we take \(V=\mathbb{R}^{n}\) and the inner product as the ordinary dot product. Let \(A\) be a matrix representing \(T\) with respect to this basis. Then the linear mapping \(T\) is symmetric if and only if the matrix \(A\) is symmetric.
Proposition. Let \(A\) be a symmetric \(n\times n\) matrix with real numbers as the entries. If \(\lambda\) is an eigenvalue of \(A\) in \(\mathbb{C}\), then \(\lambda\) is real.
Proposition. Let \(A\) be a real \(n\times n\) symmetric matrix. Then \(A\) has a real non-zero eigenvector.
Corollary. Let \(V\) be a finite-dimensional vector space over \(\mathbb{R}\) with \(\dim (V)\geq 1\). Suppose that \(V\) is endowed with a positive definite inner product. Let \(T:V\rightarrow V\) be a symmetric linear mapping. Then \(T\) has a non-zero eigenvector in \(V\).
Proposition. Let \(V\) be a finite-dimensional vector space over \(\mathbb{R}\) with \(\dim (V)\geq 1\). Suppose that \(V\) is endowed with a positive definite inner product. Let \(T:V\rightarrow V\) be a symmetric linear mapping, and let \(v\) be a non-zero eigenvector of \(T\). If \(w\in V\) with \(\langle w,v\rangle =0\), then \(\langle T(w),v\rangle =0\).
Proof. We have
\begin{align*} \langle T(w),v\rangle & =\langle w, T(v)\rangle \\ & =\langle w,\lambda v\rangle\\ & =\lambda\langle w,v\rangle =0.\end{align*}
This completes the proof. \(\blacksquare\)
Theorem. Let \(V\) be a finite-dimensional vector space over \(\mathbb{R}\) with \(\dim (V)\geq 1\). Suppose that \(V\) is endowed with a positive definite inner product. Let \(T:V\rightarrow V\) be a symmetric linear mapping. Then, there exists an orthogonal basis of \(V\) consisting of eigenvectors of \(T\).
Example. Let \(A\) be the \(2\times 2\) matrix given below
\[\left [\begin{array}{rr}
2 & 1\\ 1 & 3
\end{array}\right ].\]
We are going to find an orthogonal basis of \(\mathbb{R}^{2}\) consisting of eigenvectors of \(A\). The characteristic polynomial of \(A\) is \(\lambda^{2}-5\lambda +5\). Therefore, the eigenvalues are \(\lambda_{1}=\frac{5+\sqrt{5}}{2}\) and \(\lambda_{2}=\frac{5-\sqrt{5}}{2}\).
For \(v_{1}=(x,y)\), we can solve \(Av=\lambda_{1}v\). Equivalent, we need to solve the equations
\begin{align*}
2x+y & =\frac{5+\sqrt{5}}{2}\cdot x\\
x+3y & =\frac{5+\sqrt{5}}{2}\cdot y.
\end{align*}
We find \(x=2\) and \(y=1+\sqrt{5}\). Therefore, the first eigenvector is
\[v_{1}=\left [\begin{array}{c}
2\\ 1+\sqrt{5}
\end{array}\right ].\]
We can similarly obtain the second eigenvector
\[v_{2}=\left [\begin{array}{c}
2\\ 1-\sqrt{5}
\end{array}\right ].\]
It is easy to check that \(\{v_{1},v_{2}\}\) forms an orthogonal basis of \(\mathbb{R}^{2}\). \(\sharp\)
Corollary. Let \(A\) be a symmetric \(n\times n\) real matrix. Then there exists an \(n\times n\) unitary matrix \(U\) such that \(U^{t}AU=U^{-1}AU\) is a diagonal matrix.
Theorem. Let \(V\) be a finite-dimensional vector space over \(\mathbb{C}\) with \(\dim (V)\geq 1\). Suppose that \(V\) is endowed with a positive definite Hermitian product. Let \(T:V\rightarrow V\) be a Hermitian linear mapping. Then, there exists an orthogonal basis of \(V\) consisting of eigenvectors of \(T\). \(\sharp\)
Let \(V\) be a finite-dimensional vector space over \(\mathbb{R}\) and endowed with a positive definite inner product. The linear mapping \(T:V\rightarrow V\) is called a unitary operator when \(T\) maps unit vectors to unit vectors. The following conditions are equivalent.
- \(T\) is a unitary operator.
- \(T\) preserves inner product, i.e., \(\langle T(u),T(v)\rangle=\langle u,v\rangle\) for all \(u,v\in V\).
- \(T\) preserves length, i.e., \(\parallel T(v)\parallel =\parallel v\parallel\) for all \(v\in V\).
Theorem. Let \(V\) be a finite-dimensional vector space over \(\mathbb{R}\) with \(\dim (V)\geq 1\). Suppose that \(V\) is endowed with a positive definite inner product. Let \(T:V\rightarrow V\) be a unitary operator. Then \(V\) can be expressed as a direct sum
\[V=V_{1}\oplus V_{2}\oplus\cdots\oplus V_{r}\]
of \(T\)-invariant subspaces such that \(V_{i}\) is orthogonal to \(V_{j}\) for \(i\neq j\) and \(\dim (V)_{i}=1\) or \(2\) for each \(i=1,\cdots ,r\). \(\blacksquare\)
Corollary. Let \(V\) be a finite-dimensional vector space over \(\mathbb{R}\) with \(\dim (V)\geq 1\). Suppose that \(V\) is endowed with a positive definite inner product. Let \(T:V\rightarrow V\) be a unitary operator. Then, there exists a basis of \(V\) such that the matrix of \(T\) with respect to this basis consists of blocks
\[\left [\begin{array}{cccc}
M_{1} & {\bf 0} & \cdots & {\bf 0}\\
{\bf 0} & M_{2} & \cdots & {\bf 0}\\
{\bf 0} & {\bf 0} &\cdots & M_{r}
\end{array}\right ]\]
such that each \(M_{i}\) is a \(1\times 1\) matrix or a \(2\times 2\) matrix that has one of the following type
\[\begin{array}{cccc}
[1], & [-1], & \left [\begin{array}{cc}
\cos\theta & -\sin\theta\\ \sin\theta & \cos\theta\end{array}\right ], &
\left [\begin{array}{cc}
-\cos\theta & \sin\theta\\ \sin\theta & \cos\theta\end{array}\right ].
\end{array}\]
Theorem. Let \(V\) be a finite-dimensional vector space over \(\mathbb{C}\) with \(\dim (V)\geq 1\). Suppose that \(V\) is endowed with a positive definite Hermitian product. Let \(T:V\rightarrow V\) be a unitary operator. Then, there exists an orthogonal basis of \(V\) consisting of eigenvectors of \(T\). \(\blacksquare\)
