Angle

Inner Product Spaces: Inner product, length, and angle

Angle

Introducing the concept of angle is more subtle than length. To this end we need the Cauchy-Schwarz inequality:

In each inner product space $V$ we have, for all $\vec{a},\vec{b}\in V$ ,
$|\dotprod{\vec{a}}{\vec{b}}|\leq\norm{\vec{a}} \cdot \norm{\vec{b}}$

This inequality is an equality if and only if the vectors $\vec{a}$ and $\vec{b}$ are linearly dependent.

Let $\vec{a}$ and $\vec{b}$ be two vectors from $V$ . If one of the two vectors is the zero vector, both statements are obviously true, since then the left-hand and right-hand side of the equation both are equal to $0$ .

We will now assume that the vectors $\vec{a}$ and $\vec{b}$ are distinct from the zero vector. Because of the definition of the inner product, we have $\dotprod{(\vec{a}+\lambda \vec{b})}{(\vec{a}+\lambda \vec{b})}\geq 0$ for each scalar $\lambda$ . If we expand this inner product using linearity we find $\dotprod{\vec{a}}{\vec{a}}+2\lambda(\dotprod{\vec{a}}{\vec{b}})+\lambda^2(\dotprod{\vec{b}}{\vec{b}})\geq 0$ We can see the expression before the inequality sign as a quadratic polynomial $f(\lambda)$ . Since this function has at most one zero, we know that the discriminant is less than or equal to $0$ . If the function $f(\lambda)$ has no roots, the discriminant is negative. The well-known discriminant formula gives us $4(\dotprod{\vec{a}}{\vec{b}})^2-4(\dotprod{\vec{a}}{\vec{a}})(\dotprod{\vec{b}}{\vec{b}})\leq 0$ We can rewrite this to $4(\dotprod{\vec{a}}{\vec{b}})^2\leq 4(\dotprod{\vec{a}}{\vec{a}})(\dotprod{\vec{b}}{\vec{b}})$ Dividing this by $4$ and writing the right-hand side in terms of norms, this gives us $(\dotprod{\vec{a}}{\vec{b}})^2\leq \norm{\vec{a}}^2\cdot \norm{\vec{b}}^2$ Because both sides are positive, we can take the square root. This leads to the Cauchy-Schwarz inequality: $|\dotprod{\vec{a}}{\vec{b}}|\leq \norm{\vec{a}}\cdot \norm{\vec{b}}$

It remains for us to consider the case of equality. Equality occurs if and only if the discriminant is zero. If the discriminant equals $0$ , there is a unique solution to the equation $f(\lambda)=0$ . In other words, there is a unique $\lambda_0$ satisfying $\dotprod{(\vec{a}+\lambda_0 \vec{b})}{(\vec{a}+\lambda_0\vec{b})}=0$ In view of the definition of the inner product this means that $\vec{a}+\lambda_0\vec{b}=\vec{0}$ , which we can rewrite to $\vec{a}=-\lambda_0\vec{b}$ . This expresses linear dependence of $\vec{a}$ and $\vec{b}$ .

In the inner product space $\mathbb{R}^3$ with standard inner product the inequality for $\vec{a}=\rv{1,1,2}$ and $\vec{b} =\rv{x,y,z}$ reads
$(x+y+2z)^2 \leq (1^2 + 1^2 + 2^2)\cdot (x^2+y^2+z^2)=6(x^2+y^2+z^2)$ Note that here both sides have been squared.

In the inner product space of all real continuous functions on $\ivcc{a}{b}$ with the inner product of functions $\dotprod{f}{g} = \int_a^bf(x)\cdot g(x)\dd x$ we find, for each pair of real continuous functions $f$ and $g$ , on $\ivcc{a}{b}$
$\left|\int_a^b f(x){g(x)}\,\dd x\right|^2\ \leq \ \int_a^b\left|f(x)\right|^2\,\dd x\int_a^b \left|g(x)\right|^2\,\dd x$ These formulas are often used for estimates in Analysis of functions. For the functions $\ee^{x^2}$ and $\ee^{-x^2}$ on the interval $\ivcc{0}{1}$ the Cauchy-Schwarz inequality gives
$1= \left|\int_0^1 \ee^{x^2}\cdot\ee^{-x^2}\, \dd x \right|^2 \leq \int_0^1 \ee^{2x^2}\, \dd x \, \int_0^1\ee^{-2x^2}\, \dd x$ Here too, both sides of the Cauchy-Schwarz inequality have been squared.

The following characteristics of the norm have been announced previously.

Properties of the norm Let $V$ be an inner product space. For all vectors $\vec{a}$ and $\vec{b}$ and all scalars $\lambda$ we have

Positivity: $\norm{\vec{a}}\geq 0$ , with equality if and only if $\vec{a} =\vec{0}$
Triangle inequality: $\norm{\vec{a} + \vec{b}} \leq \norm{\vec{a}} + \norm{\vec{b}}$
Multiplicativity: $\norm{\lambda\cdot\vec{a}} = | \lambda | \cdot \norm{\vec{a}}$

The triangle inequality reflects the inequality known from the space of arrows:

The diagonal is the segment from $\vec{0}$ to $\vec{a}+\vec{b}$ . This is the shortest path from $\vec{0}$ to $\vec{a}+\vec{b}$ .

The first part follows immediately from the third property of the definition of inner product.

For the triangle inequality, we need the Cauchy-Schwarz inequality: $\begin{array}{rcl} \norm{\vec{a}+\vec{b}}^2&=& \dotprod{(\vec{a}+\vec{b})}{(\vec{a}+\vec{b})}\\ &&\phantom{xx}\color{blue}{\text{definition norm}}\\& = & \dotprod{\vec{a}}{\vec{a}}+\dotprod{\vec{b}}{\vec{b}}+\dotprod{\vec{a}}{\vec{b}}+\dotprod{\vec{b}}{\vec{a}}\\&&\phantom{xx}\color{blue}{\text{bilinearity}}\\&=&\norm{\vec{a}}^2+ \norm{\vec{b}}^2+2\dotprod{\vec{a}}{\vec{b}}\\&&\phantom{xx}\color{blue}{\text{definition norm and symmetry}}\\&\leq&\norm{\vec{a}}^2+ \norm{\vec{b}}^2+2|\dotprod{\vec{a}}{\vec{b}}|\\&&\phantom{xx}\color{blue}{{x}\leq \abs{x}}\\& \leq& \norm{\vec{a}}^2+ \norm{\vec{b}}^2+2 \norm{\vec{a}}\cdot \norm{\vec{b}}\\&&\phantom{xx}\color{blue}{\text{Cauchy-Schwarz}}\\&=&(\norm{\vec{a}}+ \norm{\vec{b}})^2\\&&\phantom{xx}\color{blue}{\text{factored}}\\ \end{array}$ The triangle inequality follows by taking square roots at both sides.

The third part follows from the linearity and symmetry of the inner product: $(\lambda \vec{a},\lambda \vec{a}) = \lambda (\vec{a},\lambda\vec{a})= \lambda \cdot\lambda(\vec{a},\vec{a})= \lambda^2\cdot(\dotprod{\vec{a}}{\vec{a}})=\abs{\lambda}^2\cdot(\dotprod{\vec{a}}{\vec{a}})$ If we take the square root of the rightmost and the leftmost side, we get the equality $\norm{\lambda\cdot \vec{a}} = |\lambda |\cdot \norm{\vec{a}}$

The triangle inequality gives us an upper bound on the length of a sum of two vectors. However, from the triangle inequality we can also find a lower bound. The inequality holds for all $\vec{a}$ and $\vec{b}$ , and so also for the vectors $\vec{a}-\vec{b}$ and $\vec{b}$ . We then find $\norm{\vec{a}} \,=\, \norm{(\vec{a} -\vec{b})+\vec{b}}\leq\norm{\vec{a} -\vec{b}} + \norm{\vec{b}}$ so $\norm{\vec{a} -\vec{b}}\ \geq\ \norm{\vec{a}} - \norm{\vec{b}}$ This statement applies to all vectors. In particular for the vectors $\vec{a}$ and $-\vec{b}$ , it gives us $\norm{\vec{a} + \vec{b}} \,\geq \, \norm{\vec{a}} -\norm{-\vec{b}} \,=\, \norm{\vec{a}} -\norm{\vec{b}}$ Interchanging $\vec{a}$ and $\vec{b}$ now produces $\norm{\vec{b} +\vec{a}} \geq \norm{\vec{b}} - \norm{\vec{a}}$ so, for all vectors $\vec{a}$ and $\vec{b}$ , we find $\norm{\vec{a} +\vec{b}}\ \geq \abs{\norm{\vec{a}} - \norm{\vec{b}}}$ We conclude that $\abs{\norm{\vec{a}} - \norm{\vec{b}}}\, \leq\, \norm{\vec{a}+\vec{b}}\, \leq \,\norm{\vec{a}}+\norm{\vec{b}}$

A vector space having a length concept that meets the requirements of the theorem is also referred to as a normed vector space.

The triangle inequality for distance, $d(\vec{a},\vec{c})\le d(\vec{a},\vec{b}) + d(\vec{b},\vec{c})$ , is a direct consequence of the triangle inequality for length: $\begin{array}{rcl}d(\vec{a},\vec{c})&=&\norm{\vec{a}-\vec{c}}\\ &&\phantom{xx}\color{blue}{\text{definition distance}}\\ &=&\norm{(\vec{a}-\vec{b})+(\vec{b}-\vec{c})}\\ &&\phantom{xx}\color{blue}{\text{sum rewritten with }\vec{b}}\\ &\le &\norm{\vec{a}-\vec{b}}+\norm{\vec{b}-\vec{c}}\\ &&\phantom{xx}\color{blue}{\text{triangle inequality for length}}\\ &=&d(\vec{a},\vec{b}) + d(\vec{b},\vec{c}) \\ &&\phantom{xx}\color{blue}{\text{definition distance}}\end{array}$

Thanks to the Cauchy-Schwarz inequality we can also use the inner product on a real vector space to define the angle between two vectors:

Let $V$ be an inner product space and let $\vec{a}$ and $\vec{b}$ be two vectors distinct from the zero vector. Then there exists a real number $\varphi\in\ivcc{0}{\pi}$ such that $\cos(\varphi) = \frac{\dotprod{\vec{a}}{\vec{b}}}{\norm{\vec{a}}\cdot \norm{\vec{b}}}$ The number $\varphi$ is called the angle between the two vectors $\vec{a}$ and $\vec{b}$ . The angle does not depend on the length of $\vec{a}$ or the length of $\vec{b}$ .

We recall the Cauchy-Schwarz inequality $|\dotprod{\vec{a}}{\vec{b}}|\ \leq\ \norm{\vec{a}} \cdot \norm{\vec{b}}$ We divide both sides by $\norm{\vec{a}}\cdot\norm{\vec{b}}$ to get $\frac{|\dotprod{\vec{a}}{\vec{b}}|}{\norm{\vec{a}} \cdot \norm{\vec{b}}}\leq 1$ Application of the definition of absolute value gives the following inequalities: $-1\ \leq \ \frac{\dotprod{\vec{a}}{\vec{b}}}{\norm{\vec{a}} \cdot \norm{\vec{b}}} \leq \ 1$ Thus there exists an angle $\varphi$ such that
$\cos(\varphi) = \frac{\dotprod{\vec{a}}{\vec{b}}}{\norm{\vec{a}}\cdot \norm{\vec{b}}}$

The angle does not depend on the length: $\cos(\varphi) = \frac{\dotprod{\vec{a}}{\vec{b}}}{\norm{\vec{a}}\cdot \norm{\vec{b}}}\ = \dotprod{\left( \frac{1}{\norm{\vec{a}}} \vec{a}\right)}{\left( \frac{1}{\norm{\vec{b}}} \vec{b}\right)}$

Figure

In the figure below we see how the angle and the inner product are related to each other. If we rewrite the equation we get

$\dotprod{\vec{a}}{\vec{b}}=\norm{\vec{a}} \cdot \norm{\vec{b}} \cdot \cos (\varphi)$

We keep the lengths $\vec{a}$ and $\vec{b}$ equal (this does not matter for the inner product). You can drag the non-horizontal vector in order to see what happens to the angle. We can see from the equation that the absolute value of the inner product is at a maximum if the angle equals $0$ or $180$ degrees.

geogebra picture

What angle do the vectors $\vec{a} = \rv{1,0,-1}$ and $\vec{b}=\rv{3,-2,3}$ make in the inner product space $\mathbb{R}^3$ with standard inner product?]

Give your answer in radians.

$\varphi =$ ${{\pi}\over{2}}$

First we calculate the inner product of the vectors $\vec{a}$ and $\vec{b}$ and their lengths
$\begin{array}{rclcl} \dotprod{\vec{a}}{\vec{b}} &=& (1)\cdot (3) + (0)\cdot (-2) + (-1)\cdot (3) &=&0 \\ \norm{\vec{a}} &=& \sqrt{(1)^2+ (0)^2 + (-1)^2} &=&\sqrt{2} \\ \norm{\vec{b}} &=& \sqrt{(3)^2+ (-2)^2 + (3)^2} &=&\sqrt{22} \end{array}$ It follows that the angle $\varphi$ between $\vec{a}$ and $\vec{b}$ satisfies
$\cos(\varphi) = \frac{\dotprod{\vec{a}}{\vec{b}}}{\norm{\vec{a}}\cdot\norm{\vec{b}}}=\frac{0}{\sqrt{2} \cdot \sqrt{22}} = 0$ We conclude that the answer is $\varphi = {{\pi}\over{2}}$ .

New example