Dot Product

Two ways to define

Vectors as “Arrows”

The angle between vectors \(\mathbf{u}\) and \(\mathbf{v}\) in \(\mathbb{R}^2\) or \(\mathbb{R}^3\) is simply the angle, between \(-\pi\) and \(\pi\), one would need to rotate the vector \(\mathbf{u}\) by so that it would become parallel to the vector \(\mathbf{v}\):

On the diagram above, \(\theta\) is the angle between \(\mathbf{u}\) and \(\mathbf{v}\).

The dot product (also called scalar product or inner product) of two vectors \(\mathbf{u}\) and \(\mathbf{v}\) in \(\mathbb{R}^2\) or \(\mathbb{R}^3\) is the scalar quantity

\[\mathbf{u}\cdot\mathbf{v} = \left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert\cos\theta\]

where \(\theta\) is the angle between \(\mathbf{u}\) and \(\mathbf{v}\).

Some properties

  • The dot product is commutative: \(\mathbf{u}\cdot\mathbf{v} = \mathbf{v}\cdot\mathbf{u}\). This follows from the fact that \(\cos\) is an even function, so \(\cos(-\theta) = \cos(\theta)\).

  • The dot product \(\mathbf{u}\cdot\mathbf{v}\) is zero if and only if at least one of the following is true:

    • \(\mathbf{u}\) is the zero vector
    • \(\mathbf{v}\) is the zero vector
    • \(\mathbf{u}\) and \(\mathbf{v}\) are perpendicular to each other.

    This means that the dot product does not satisfy the zero product property: dot product of two vectors can be zero even when none of the two vectors are zero!

  • The dot product \(\mathbf{u}\cdot\mathbf{v}\) is positive if and only if the angle between the two vectors is acute, that is between \(-\frac{\pi}{2}\) and \(\frac{\pi}{2}\).

  • The dot product \(\mathbf{u}\cdot\mathbf{v}\) is negative if and only if the angle between the two vectors is obtuse.

  • For any scalar1 \(c\), \((c\mathbf{u})\cdot\mathbf{v} = \mathbf{u}\cdot(c\mathbf{v}) = c(\mathbf{u}\cdot\mathbf{v})\)

  • \(\mathbf{u}\cdot\mathbf{v} = \frac{1}{2}\left(\left\lvert\mathbf{u} \right\rvert^2 + \left\lvert\mathbf{v} \right\rvert^2 - \left\lvert\mathbf{u} - \mathbf{v} \right\rvert^2\right)\)

    This is a consequence of the law of cosines. Applying the law of cosines on the following triangle:

    will give us \(\left\lvert\mathbf{u} - \mathbf{v} \right\rvert^2 = \left\lvert\mathbf{u} \right\rvert^2 + \left\lvert\mathbf{v} \right\rvert^2 - 2\left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert\cos\theta\).

  • \(\mathbf{u}\cdot\mathbf{u} = \left\lvert\mathbf{u} \right\rvert^2\).

1 This is true even for a negative \(c\)! Think about what happens to an angle between two vectors when one of the vectors is multiplied by a negative scalar!

Try it yourself

Vectors in components

If \(\mathbf{u}\) and \(\mathbf{v}\) are two vectors in \(\mathbb{R}^2\) or \(\mathbb{R}^3\), we know that (as a consequence of the law of cosines)

\[\mathbf{u}\cdot\mathbf{v} = \frac{1}{2}\left(\left\lvert\mathbf{u} \right\rvert^2 + \left\lvert\mathbf{v} \right\rvert^2 - \left\lvert\mathbf{u} - \mathbf{v} \right\rvert^2\right).\]

Writing \(\mathbf{u} = \left\langle u_1, u_2, u_3\right\rangle\) and \(\mathbf{v} = \left\langle v_1, v_2, v_3\right\rangle\) in \(\mathbb{R}^3\) (or similarly with just two components in \(\mathbb{R}^2\)), the identity becomes \[\begin{aligned} \mathbf{u}\cdot\mathbf{v} &= \frac{1}{2}\left(\left\lvert\mathbf{u} \right\rvert^2 + \left\lvert\mathbf{v} \right\rvert^2 - \left\lvert\mathbf{u} - \mathbf{v} \right\rvert^2\right)\\ &= \frac{1}{2}\left(u_1^2 + u_2^2 + u_3^2 + v_1^2 + v_2^2 + v_3^2 - (u_1 - v_1)^2 - (u_2 - v_2)^2 - (u_3 - v_3)^2\right)\\ \end{aligned}\] Expanding the squares and combining like terms will turn this into \[\frac{1}{2}\left(2u_1v_1 + 2u_2v_2 + 3u_3v_3\right) = u_1v_1 + u_2v_2 + u_3v_3\] which gives us an easy way to calculate a dot product of \(\mathbf{u}\) and \(\mathbf{v}\) using their components.

Dot product in any finite dimension

In light of this, we can now define dot product between two vectors \(\mathbf{u}\) and \(\mathbf{v}\) in \(\mathbb{R}^n\) for any integer \(n > 0\):

Given \[\mathbf{u} = \left\langle u_1, u_2, \dots, u_n\right\rangle \in \mathbb{R}^n\] and \[\mathbf{v} = \left\langle v_1, v_2, \dots, v_n\right\rangle \in \mathbb{R}^n\] define \[\mathbf{u}\cdot\mathbf{v} = u_1v_1 + u_2v_2 + \cdots + u_nv_n = \sum_{i=1}^n u_iv_i.\]

Again, just as with the norm, this can be extended to infinitely dimensional vectors, such as sequences or functions, as long as one takes care of convergence issues in some way.

  • If \(n = 1\), this just becomes a regular product of two numbers.
  • As we have just shown, if \(n = 2\) or \(n = 3\), this agrees with the earlier definition of dot product using an angle between vectors.
  • In higher dimensions, it gives us a way to define a dot product even if there is no intuitive geometrical concept of an angle between vectors.
Example:

If \(\mathbf{u} = \left\langle 1,-2,4,3\right\rangle\) and \(\mathbf{v} = \left\langle 2,5,-1,2\right\rangle\) then \[\mathbf{u}\cdot\mathbf{v} = 1\cdot 2 + (-2)\cdot 5 + 4\cdot (-1) + 3\cdot 2 = 2 - 10 - 4 + 6 = -6.\]

Try it yourself

Properties of dot product

Earlier we saw several properties of dot product in \(\mathbb{R}^2\) and \(\mathbb{R}^3\) that followed easily from the geometric definition. Most of these properties are still true in any dimension, and can be proved by rewriting vectors using components and manipulating the resulting expressions using rules of algebra. In addition to that, there are also some other properties that are very hard to prove from the geometric definition, but became relatively easy when using components. Here is a list with some important properties of dot product:

  • The dot product is commutative: \(\mathbf{u}\cdot\mathbf{v} = \mathbf{v}\cdot\mathbf{u}\).

  • Dot product does not satisfy the zero product property: dot product of two vectors can be zero even when none of the two vectors are zero!

  • Dot product distributes2 over vector addition: \(\mathbf{u}\cdot(\mathbf{v} + \mathbf{w}) = \mathbf{u}\cdot\mathbf{v} + \mathbf{u}\cdot\mathbf{w}\).

  • For any scalar \(c\), \((c\mathbf{u})\cdot\mathbf{v} = \mathbf{u}\cdot(c\mathbf{v}) = c(\mathbf{u}\cdot\mathbf{v})\)

  • \(\mathbf{u}\cdot\mathbf{v} = \frac{1}{2}\left(\left\lvert\mathbf{u} \right\rvert^2 + \left\lvert\mathbf{v} \right\rvert^2 - \left\lvert\mathbf{u} - \mathbf{v} \right\rvert^2\right)\)

  • \(\mathbf{u}\cdot\mathbf{u} = \left\lvert\mathbf{u} \right\rvert^2\).

  • Cauchy–Schwartz inequality: 3 for any two vectors \(\mathbf{u}\) and \(\mathbf{v}\) in \(\mathbb{R}^n\), \(\left\lvert\mathbf{u}\cdot\mathbf{v} \right\rvert \le \left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert\).

2 The distributive property is hard to prove geometrically, but very easy to prove using components! Give it a try!

3 A nice simple proof of this version of Cauchy–Schwartz inequality can be found for example on Wikipedia.

Angles between vectors

In \(\mathbb{R}^2\) and \(\mathbb{R}^3\), we have two equivalent ways to calculate the dot product of two vectors: one using components and one using the angle between the vectors. If we know the components of the two vectors, we can use this to calculate the angle, or at least its absolute value!

Starting with the equation \(\mathbf{u}\cdot\mathbf{v} = \left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert\cos\theta\) and solving for \(\cos\theta\), we get

\[\cos\theta = \frac{\mathbf{u}\cdot\mathbf{v}}{\left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert}\]

Using the inverse cosine we get

\[\left\lvert\theta \right\rvert = \cos^{-1}\left(\frac{\mathbf{u}\cdot\mathbf{v}}{\left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert}\right)\]

for any angle \(\theta\) between \(-\pi\) and \(\pi\).

Example

Let’s take two vectors, \(\mathbf{u} = \left\langle 1,3,2\right\rangle\) and \(\mathbf{v} = \left\langle 2,-2,1\right\rangle\). We get

\[\begin{aligned} \left\lvert\mathbf{u} \right\rvert &= \sqrt{1^2 + 3^2 + 2^2} = \sqrt{14}\\ \left\lvert\mathbf{v} \right\rvert &= \sqrt{2^2 + (-2)^2 + 1^2} = \sqrt{9} = 3\\ \mathbf{u}\cdot\mathbf{v} &= 1\cdot 2 + 3\cdot(-2) + 2\cdot 1 = 1 - 6 + 1 = -4 \end{aligned}\]

so

\[\cos\theta = \frac{\mathbf{u}\cdot\mathbf{v}}{\left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert} = \frac{-4}{3\sqrt{14}}\]

That means that either

\[\theta = \cos^{-1}\left(-\frac{4}{3\sqrt{14}}\right)\]

or

\[\theta = -\cos^{-1}\left(-\frac{4}{3\sqrt{14}}\right)\]

Either way, \(\theta\) is an obtuse angle.

Note that while in \(\mathbb{R}^2\), the sign of an angle is always well defined: counter-clockwise rotations are positive and clockwise are negative, in \(\mathbb{R}^3\) the sign of an angle depends on which side we are viewing the angle from.

Angles in higher dimensions

In fact, we can use dot product to define both angles and distances in higher dimensions.

First, we can define the norm of a vector as \(\left\lvert\mathbf{u} \right\rvert = \sqrt{\mathbf{u}\cdot\mathbf{u}}\). Once we know how to find a norm of a vector, we can define the distance between two points \(P\) and \(Q\) as the norm of the vector from \(P\) to \(Q\).

Second, we can define the angle \(\theta\) between two non-zero vectors \(\mathbf{u}\) and \(\mathbf{v}\) as \[\theta = \cos^{-1}\left(\frac{\mathbf{u}\cdot\mathbf{v}}{\left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert}\right)\]

This is well defined: according to the Cauchy–Schwartz inequality, \[-1 \le \frac{\mathbf{u}\cdot\mathbf{v}}{\left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert} \le 1\] which means that \[\cos^{-1}\left(\frac{\mathbf{u}\cdot\mathbf{v}}{\left\lvert\mathbf{u} \right\rvert\left\lvert\mathbf{v} \right\rvert}\right)\] is defined and between \(0\) and \(\pi\).

Orthogonal vectors

We saw that there are two ways for \(\mathbf{u}\cdot\mathbf{v}\) to be 0:

  1. One or both of the vectors are zero vectors.
  2. Both vectors are non-zero and the angle between the vectors is \(\frac{\pi}{2}\) or \(-\frac{\pi}{2}\), in other words, the vectors are perpendicular to each other.

To make things easier, we define the concept of orthogonal vectors:

We say that vectors \(\mathbf{u}\) and \(\mathbf{v}\) are orthogonal to each other, or that \(\mathbf{u}\) is orthogonal to \(\mathbf{v}\), or that \(\mathbf{v}\) is orthogonal to \(\mathbf{u}\), if the dot product \(\mathbf{u}\cdot\mathbf{v} = 0\).

For non-zero vectors, orthogonal is the same as perpendicular. The zero vector is orthogonal to all vectors, including itself!

Try it yourself