Part 3 — Orthogonality and Projection

Learning Objective¶

Understand orthogonality as the independence of two vector directions, and show that mean subtraction is exactly orthogonal projection onto the $[1, 1, \ldots, 1]$ brightness direction — making brightness and pattern independent components.

6. Orthogonality¶

Intuition¶

Two vectors are orthogonal (perpendicular) when their dot product is zero:

Geometrically, they point in completely independent directions — knowing one tells you nothing about the other.

Why this matters for normalisation: Mean subtraction splits a patch vector into two orthogonal parts — a brightness component and a pattern component. Because they are orthogonal, changing brightness cannot affect the pattern. That independence is the whole reason normalisation works.

The key orthogonality we need¶

The orthogonality that matters for normalisation is between:

• The uniform brightness vector $[1, 1, 1, \ldots, 1]$ — equal value everywhere, encodes only brightness, zero spatial pattern

• The mean-subtracted residual of any vector

Any scalar multiple of $[1, 1, \ldots, 1]$ is a flat gray image. Mean subtraction removes exactly this component.

7. Orthogonal Projection and Decomposition¶

Intuition¶

Orthogonal projection splits a vector into two perpendicular parts:

The projection formula onto a unit vector $\hat{u}$:

Mean subtraction is orthogonal projection¶

When $\vec{u} = [1, 1, \ldots, 1]$ (the uniform brightness direction), the projection simplifies to:

That is just the mean of the pixel values, broadcast to every element. So mean subtraction IS orthogonal projection — it removes the component along $[1, 1, \ldots, 1]$.

The projection formula derivation¶

Let $\vec{u} = [1, 1, \ldots, 1]$ with $n$ elements. Its unit vector is:

The projection of $\vec{v}$ onto $\hat{u}$:

So the projection is just the mean broadcast to every element. Mean subtraction removes this projection, leaving only the perpendicular (pattern) component.

Key consequence¶

A brightness offset only changes the brightness component, not the pattern component. After mean subtraction, patterns are identical regardless of uniform lighting shifts.