- Calculus, Volume 2: Multi-Variable Calculus and Linear Algebra with Applications to Differential Equations and Probability
- Tom M. Apostol
- Second Edition
- 1991
- 978-1-119-49676-2
Best approximation of elements in a Euclidean space by elements in a finite-dimensional subspace
$\quad$Theorem 1.16. $\quad$ Approximation Theorem. $\quad$ Let $S$ be a finite-dimensional subspace of a Euclidean space $V,$ and let $x$ be any element of $V.$ Then the projection of $x$ on $S$ is nearer to $x$ than any other element of $S.$ That is, if $s$ is the projection of $x$ on $S,$ we have \begin{align*} \|x - s\| &\leq \|x - t\| \end{align*} for all $t$ in $S;$ the equality sign holds if and only if $t = s.$
$\quad$ Proof.$\quad$ We can do an orthogonal decomposition on $x$ to give $x = s + s^{\perp},$ where $s \in S$ and $s^{\perp} \in S^{\perp}.$ Then, let $t$ be any element of $S.$ We have \begin{align*} x - t &= (x - s) + (s - t) \end{align*} If we take the inner product of both sides, noting that $x - s = s^{\perp},$ we get: \begin{align*} \|x - t\|^2 &= \|(x - s) + (s - t)\|^2 \\ &= \|x - s\|^2 + 2(x - s, s - t) + \|s - t\|^2 \\ &= \|x - s\|^2 + 2(s^{\perp}, s - t) + \|s - t\|^2 \end{align*} But since $s^{\perp} \in S,$ we know that $(s^{\perp}, s - t) = 0,$ giving us: \begin{align*} \|x - t\|^2 &= \|x - s\|^2 + \|s - t\|^2 \end{align*} By positivity of the inner product (and by extension, the norm), this means that $\|x - s\| \leq \|x - t\|,$ with equality being satisfied if and only if $s = t. \quad \blacksquare$