Eigenvectors: the directions a matrix only stretches

Apply a matrix and almost every arrow gets knocked off its line: it turns as well as stretches. But most matrices have a few special directions where the arrow stays on its own line and is only stretched, shrunk or flipped.

Those directions explain what the matrix does when you apply it over and over, which is why they show up everywhere: Google's PageRank, the long-run state of a Markov chain, the principal components of a dataset, and whether a recurrent network's signal explodes or fades.

eigenvector

A non-zero vector $v$ with $Av = \lambda v$: the matrix sends it somewhere on its own line.

eigenvalue λ

The stretch factor along that line. $\lambda > 1$ stretches, $0 < \lambda < 1$ shrinks, $\lambda < 0$ flips, $\lambda = 0$ crushes the line to the origin.

eigen line

All multiples of an eigenvector. The matrix maps the line onto itself.

For this symmetric matrix the arrow lines up twice, at 45° (where $A$ triples it) and at 135° (where it leaves it alone). At the end, the whole grid is transformed: the two eigen lines stay exactly where they were.

Before each step, predict whether the next direction is an eigenvector.

Try the examples below, then drag î and ĵ at the end of a run to see the eigen lines move.

• Shear: only one eigen line, the x-axis. Every other arrow leans to the right.
• Rotation: no arrow ever lines up, so there are no real eigenvectors (the eigenvalues are complex numbers).
• Squash flat: one eigenvalue is 0. A whole line is crushed onto the origin, which is why $\det A = 0$.

• An eigenvector stays on its own line: $Av = \lambda v$. The eigenvalue $\lambda$ is the stretch along it.
• Find them geometrically by sweeping a direction and watching where $Av$ lines up with $v$.
• Rotations have none (real), shears have one, symmetric matrices have two at right angles.
• Eigenvalues multiply to $\det A$ and add up to the trace. Next: eigenvectors of a covariance matrix are the principal components of data (PCA).