Foundations T10 / 27 · Linear Algebra and Geometry
Eigenvectors: The Directions a Matrix Dare Not Touch
A matrix skews most directions, but a few directions only get stretched or shrunk — those are the eigen-directions, and the scaling factors are the eigenvalues. The principal axes of a point cloud, the principal modes of a vibration, the dominant timescales of a system — all of them.
One-line takeaway
T8 said a matrix is «transformations chained together». Acting on vectors filling the whole space, most vectors are both rotated and skewed. But there are always a few special directions: when the matrix acts on them, they are only scaled up or down, their direction unmoved. These «directions a matrix dare not touch» hide a matrix's entire personality. The longest direction after a point cloud collapses into a blob, the dominant frequency of a vibrating structure, the response speed of a control system — all of these are eigenvalues speaking.
Animated · the point cloud drifts gently and the principal-axis arrow follows — PCA is exactly this arrow that always points where the cloud spreads most
Three high-frequency robotics scenarios run on this: ① the covariance matrix of a point cloud / data — principal axes = eigenvectors (aligning 3D point clouds, doing PCA, estimating plane normals all rest on it); ② the inertia tensor — the free-rotation principal axes = eigenvectors; ③ control systems — convergence/divergence speed = the magnitude and sign of the eigenvalues.
Why this tool exists
This page does only 2×2, and every calculation is doable with middle-school math — yet the three application scenarios run daily in industrial robotics.
Definitions
Definition 4.1 Eigenvectors and eigenvalues
An eigenvector v (nonzero) of a square matrix A and its eigenvalue λ satisfy Av = λv: after the action the direction is unchanged, scaled by λ. λ may be negative (reversed) or zero (squashed to zero — a signal of singularity).
Definition 4.2 Covariance matrix
For a pile of data points, subtract each dimension's mean, then arrange the averages of pairwise products into a matrix C: diagonal entries = each dimension's own variance (how spread out), off-diagonal entries = how much two dimensions rise and fall together (correlation). C is always symmetric positive semidefinite — that is its certificate of eligibility for «having principal axes read off».
Theorems and proofs
Theorem 4.1 How to find eigenvalues, solved completely for 2×2
Av = λv has a nonzero solution ⟺ det(A − λI) = 0 (the characteristic equation). For A = [[a, b], [c, d]]: λ² − (a+d)λ + (ad − bc) = 0; the sum of the two roots = trace a+d, and the product of the two roots = determinant ad − bc.
Proof
Av = λv ⟺ (A − λI)v = 0. A homogeneous system has a nonzero solution ⟺ the coefficient matrix is singular (otherwise the unique solution is v = 0, excluded by definition) ⟺ det(A − λI) = 0.
Expand the 2×2: det([[a−λ, b], [c, d−λ]]) = (a−λ)(d−λ) − bc = λ² − (a+d)λ + (ad−bc). This is a quadratic in λ, and Vieta's formulas give the sum and product of the roots. ∎
NoteTrace and determinant are the two «summary cards» of the eigenvalues: without solving the equation you can state the sum and the product of the roots. For real symmetric matrices the roots are necessarily real (Theorem 4.2) — general matrices may have complex eigenvalues (those of a rotation matrix are e±iα; «rotation = turning in the complex plane», exactly the foreshadowing of T6's quaternions page).
Theorem 4.2 Real symmetric matrices: all eigenvalues real, eigenvectors orthogonal
When A = A^T, all eigenvalues are real, and eigenvectors of distinct eigenvalues are mutually perpendicular. For 2×2, A = [[a, b], [b, d]].
Proof
Realness (inner-product argument): Av = λv; take the inner product with the complex vector v: v̄ᵀAv = λ v̄ᵀv. The left side is a scalar and equals its own conjugate transpose (A is real symmetric), so λ equals λ̄ and the imaginary part is zero.
Orthogonality: let Av = λv and Aw = μw with λ ≠ μ. Left-multiply the first by wᵀ: wᵀAv = λ wᵀv. Transpose the second and left-multiply by v, using symmetry (vᵀA = (Av)ᵀ = λvᵀ): λ vᵀw = μ vᵀw. Subtracting: (λ − μ) wᵀv = 0, and since λ ≠ μ, wᵀv = 0 — perpendicular. ∎
Note«Symmetric = readable»: a symmetric matrix always has a set of mutually perpendicular eigen-directions, like an orthogonal skeleton. That is why covariance, the inertia tensor, and the Hessian (all real symmetric) can have their «principal axes» drawn — not coincidence, but the spectral theorem at work.
Theorem 4.3 The principal-axis meaning of covariance
The eigenvectors of a point cloud's covariance matrix C are the directions of maximal spread (ordered by eigenvalue from largest down), and the eigenvalues are the variances of the point cloud along those directions.
Proof
Write the point cloud as a set of mean-zero column vectors; C = (1/n) Σ x_i x_iᵀ (definition of covariance).
The variance of the projection along a unit direction u is (1/n) Σ (uᵀx_i)² = uᵀ[(1/n)Σ x_i x_iᵀ]u = uᵀCu.
«Maximize the variance along u» is the constrained problem max uᵀCu, ‖u‖=1. By the spectral theorem (Theorem 4.2) C has an orthonormal eigenbasis; expanding u in that basis and computing directly gives uᵀCu = Σ λ_k c_k² (c_k the expansion coefficients, Σc_k² = 1) — the maximum is attained when the c corresponding to the largest λ is 1, i.e., u = the eigenvector of the largest eigenvalue. ∎
NoteThe load-bearing wall of the entire edifice of PCA (principal component analysis) is these three lines: projection variance = uᵀCu, optimal direction = top eigenvector. Point cloud registration, plane fitting, dimensionality reduction — all downstream of it.
Work it by hand
A = [[2, 1], [1, 2]]. Find all eigenvalues and eigenvectors, and state what shape it twists the unit circle into.
- Characteristic equation: λ² − (2+2)λ + (4−1) = λ² − 4λ + 3 = 0, factoring as (λ−1)(λ−3) = 0, so λ₁ = 3, λ₂ = 1.
- λ₁ = 3: (A − 3I)v = 0 ⟺ [[−1,1],[1,−1]]v = 0 ⟺ v₁ = (1,1)/√2 (normalized).
- λ₂ = 1: [[1,1],[1,1]]v = 0 ⟺ v₂ = (1,−1)/√2. Check orthogonality: v₁·v₂ = (1−1)/2 = 0 ✓ (Theorem 4.2).
- Geometric reading: stretched 3× along the 45° direction, kept at 1× along the 135° direction. Unit circle → an ellipse with semi-axes 3 and 1, principal axis 45°. No «skewing» at all — a symmetric matrix only stretches, never shears.
λ = 3 and 1; eigenvectors (1,1)/√2 and (1,−1)/√2 (automatically orthogonal). The unit circle is stretched into an ellipse with principal axis 45° and semi-axes 3×1. The action of a symmetric matrix = independent scaling along orthogonal principal axes — that one sentence is the trailer for SVD (next page).
Run the numbers
Where the numbers come from
Actually run at generation time (500-point Gaussian cloud, theoretical principal axis 45°, theoretical variance ratio 4):
- Direction of the sample covariance's top eigenvector: deviates 0.9° from the theoretical principal axis 45° (finite-sample noise)
- Sample eigenvalue ratio: 4.25 (theory 4.00)
- Conclusion: a few hundred points suffice to read out «the data's dominant spread direction» reliably — PCA is cheap, stable, and fast in robot perception
Where this shows up
| The object here | The counterpart in RL | Used in |
|---|---|---|
| Point cloud processing | PCA for plane normals / principal directions of point cloud registration | used in → |
| Inertia tensor | Free-rotation principal axes = eigenvectors of the inertia tensor | used in → |
| Next page | SVD and the pseudoinverse: generalizing «principal-axis decomposition» to every matrix | used in → |
Boundaries: where this article stops
Where this page stops: the spectral theorem's proof for general n×n (inductive construction), the correspondence between complex eigenvalues and rotation matrices, and eigenvalue perturbation theory (Davis-Kahan) are not developed. Complete for 2×2, able to use 3×3 (numerical libraries do it), and knowing theoretically why it is reliable — that is this page's boundary and its promise.
Self-test: 4 questions
All hand-computation and «find the flaw in this proof» questions — nothing to memorize. Stuck? Go back to the section.
01What is the relation between eigenvalues and the determinant?
The product of all eigenvalues = det A (Vieta's formulas). So det A = 0 ⟺ some eigenvalue is zero ⟺ some direction is squashed ⟺ not invertible. The intuition of the determinant as a «volume scaling factor» lands right here.
02Why must a covariance matrix have «principal axes»?
It is real symmetric. The spectral theorem guarantees real eigenvalues and orthogonal eigenvectors — so you can always draw an orthogonal skeleton of principal axes. This is not luck; it is structure.
03What are the eigenvalues of a rotation matrix?
e±iα — complex. A rotation has no real «stretching direction» (every direction gets rotated away); its eigenvectors live in the complex plane. This is precisely the matrix-form confirmation of «complex multiplication = rotation» (T6).
04How do you find the «longest direction» of a data point cloud?
Compute the covariance matrix and take the eigenvector of the largest eigenvalue. A few hundred samples suffice for stability (measured angular deviation about 1° at 500 points), at the cost of one matrix multiplication.