RobotGuide
Math foundations/Eigenvectors: The Directions a Matrix Dare Not Touch

Foundations T10 / 27 · Linear Algebra and Geometry

Eigenvectors: The Directions a Matrix Dare Not Touch

A matrix skews most directions, but a few directions only get stretched or shrunk — those are the eigen-directions, and the scaling factors are the eigenvalues. The principal axes of a point cloud, the principal modes of a vibration, the dominant timescales of a system — all of them.

about 45 min3 theorems7 proof steps4 self-test questions

One-line takeaway

T8 said a matrix is «transformations chained together». Acting on vectors filling the whole space, most vectors are both rotated and skewed. But there are always a few special directions: when the matrix acts on them, they are only scaled up or down, their direction unmoved. These «directions a matrix dare not touch» hide a matrix's entire personality. The longest direction after a point cloud collapses into a blob, the dominant frequency of a vibrating structure, the response speed of a control system — all of these are eigenvalues speaking.

Animated · the point cloud drifts gently and the principal-axis arrow follows — PCA is exactly this arrow that always points where the cloud spreads most

Three high-frequency robotics scenarios run on this: ① the covariance matrix of a point cloud / data — principal axes = eigenvectors (aligning 3D point clouds, doing PCA, estimating plane normals all rest on it); ② the inertia tensor — the free-rotation principal axes = eigenvectors; ③ control systems — convergence/divergence speed = the magnitude and sign of the eigenvalues.

Why this tool exists

This page does only 2×2, and every calculation is doable with middle-school math — yet the three application scenarios run daily in industrial robotics.

Definitions

Definition 4.1 Eigenvectors and eigenvalues

An eigenvector v (nonzero) of a square matrix A and its eigenvalue λ satisfy Av = λv: after the action the direction is unchanged, scaled by λ. λ may be negative (reversed) or zero (squashed to zero — a signal of singularity).

Definition 4.2 Covariance matrix

For a pile of data points, subtract each dimension's mean, then arrange the averages of pairwise products into a matrix C: diagonal entries = each dimension's own variance (how spread out), off-diagonal entries = how much two dimensions rise and fall together (correlation). C is always symmetric positive semidefinite — that is its certificate of eligibility for «having principal axes read off».

Theorems and proofs

Theorem 4.1 How to find eigenvalues, solved completely for 2×2

Av = λv has a nonzero solution ⟺ det(A − λI) = 0 (the characteristic equation). For A = [[a, b], [c, d]]: λ² − (a+d)λ + (ad − bc) = 0; the sum of the two roots = trace a+d, and the product of the two roots = determinant ad − bc.

Proof

Av = λv ⟺ (A − λI)v = 0. A homogeneous system has a nonzero solution ⟺ the coefficient matrix is singular (otherwise the unique solution is v = 0, excluded by definition) ⟺ det(A − λI) = 0.

Expand the 2×2: det([[a−λ, b], [c, d−λ]]) = (a−λ)(d−λ) − bc = λ² − (a+d)λ + (ad−bc). This is a quadratic in λ, and Vieta's formulas give the sum and product of the roots. ∎

NoteTrace and determinant are the two «summary cards» of the eigenvalues: without solving the equation you can state the sum and the product of the roots. For real symmetric matrices the roots are necessarily real (Theorem 4.2) — general matrices may have complex eigenvalues (those of a rotation matrix are e±iα; «rotation = turning in the complex plane», exactly the foreshadowing of T6's quaternions page).

Theorem 4.2 Real symmetric matrices: all eigenvalues real, eigenvectors orthogonal

When A = A^T, all eigenvalues are real, and eigenvectors of distinct eigenvalues are mutually perpendicular. For 2×2, A = [[a, b], [b, d]].

Proof

Realness (inner-product argument): Av = λv; take the inner product with the complex vector v: v̄ᵀAv = λ v̄ᵀv. The left side is a scalar and equals its own conjugate transpose (A is real symmetric), so λ equals λ̄ and the imaginary part is zero.

Orthogonality: let Av = λv and Aw = μw with λ ≠ μ. Left-multiply the first by wᵀ: wᵀAv = λ wᵀv. Transpose the second and left-multiply by v, using symmetry (vᵀA = (Av)ᵀ = λvᵀ): λ vᵀw = μ vᵀw. Subtracting: (λ − μ) wᵀv = 0, and since λ ≠ μ, wᵀv = 0 — perpendicular. ∎

Note«Symmetric = readable»: a symmetric matrix always has a set of mutually perpendicular eigen-directions, like an orthogonal skeleton. That is why covariance, the inertia tensor, and the Hessian (all real symmetric) can have their «principal axes» drawn — not coincidence, but the spectral theorem at work.

Theorem 4.3 The principal-axis meaning of covariance

The eigenvectors of a point cloud's covariance matrix C are the directions of maximal spread (ordered by eigenvalue from largest down), and the eigenvalues are the variances of the point cloud along those directions.

Proof

Write the point cloud as a set of mean-zero column vectors; C = (1/n) Σ x_i x_iᵀ (definition of covariance).

The variance of the projection along a unit direction u is (1/n) Σ (uᵀx_i)² = uᵀ[(1/n)Σ x_i x_iᵀ]u = uᵀCu.

«Maximize the variance along u» is the constrained problem max uᵀCu, ‖u‖=1. By the spectral theorem (Theorem 4.2) C has an orthonormal eigenbasis; expanding u in that basis and computing directly gives uᵀCu = Σ λ_k c_k² (c_k the expansion coefficients, Σc_k² = 1) — the maximum is attained when the c corresponding to the largest λ is 1, i.e., u = the eigenvector of the largest eigenvalue. ∎

NoteThe load-bearing wall of the entire edifice of PCA (principal component analysis) is these three lines: projection variance = uᵀCu, optimal direction = top eigenvector. Point cloud registration, plane fitting, dimensionality reduction — all downstream of it.

Work it by hand

A = [[2, 1], [1, 2]]. Find all eigenvalues and eigenvectors, and state what shape it twists the unit circle into.

  1. Characteristic equation: λ² − (2+2)λ + (4−1) = λ² − 4λ + 3 = 0, factoring as (λ−1)(λ−3) = 0, so λ₁ = 3, λ₂ = 1.
  2. λ₁ = 3: (A − 3I)v = 0 ⟺ [[−1,1],[1,−1]]v = 0 ⟺ v₁ = (1,1)/√2 (normalized).
  3. λ₂ = 1: [[1,1],[1,1]]v = 0 ⟺ v₂ = (1,−1)/√2. Check orthogonality: v₁·v₂ = (1−1)/2 = 0 ✓ (Theorem 4.2).
  4. Geometric reading: stretched 3× along the 45° direction, kept at 1× along the 135° direction. Unit circle → an ellipse with semi-axes 3 and 1, principal axis 45°. No «skewing» at all — a symmetric matrix only stretches, never shears.
Answer

λ = 3 and 1; eigenvectors (1,1)/√2 and (1,−1)/√2 (automatically orthogonal). The unit circle is stretched into an ellipse with principal axis 45° and semi-axes 3×1. The action of a symmetric matrix = independent scaling along orthogonal principal axes — that one sentence is the trailer for SVD (next page).

Run the numbers

Where the numbers come from

Actually run at generation time (500-point Gaussian cloud, theoretical principal axis 45°, theoretical variance ratio 4):

  • Direction of the sample covariance's top eigenvector: deviates 0.9° from the theoretical principal axis 45° (finite-sample noise)
  • Sample eigenvalue ratio: 4.25 (theory 4.00)
  • Conclusion: a few hundred points suffice to read out «the data's dominant spread direction» reliably — PCA is cheap, stable, and fast in robot perception

Where this shows up

The object hereThe counterpart in RLUsed in
Point cloud processingPCA for plane normals / principal directions of point cloud registrationused in →
Inertia tensorFree-rotation principal axes = eigenvectors of the inertia tensorused in →
Next pageSVD and the pseudoinverse: generalizing «principal-axis decomposition» to every matrixused in →

Boundaries: where this article stops

Where this page stops: the spectral theorem's proof for general n×n (inductive construction), the correspondence between complex eigenvalues and rotation matrices, and eigenvalue perturbation theory (Davis-Kahan) are not developed. Complete for 2×2, able to use 3×3 (numerical libraries do it), and knowing theoretically why it is reliable — that is this page's boundary and its promise.

Self-test: 4 questions

All hand-computation and «find the flaw in this proof» questions — nothing to memorize. Stuck? Go back to the section.

01What is the relation between eigenvalues and the determinant?

The product of all eigenvalues = det A (Vieta's formulas). So det A = 0 ⟺ some eigenvalue is zero ⟺ some direction is squashed ⟺ not invertible. The intuition of the determinant as a «volume scaling factor» lands right here.

02Why must a covariance matrix have «principal axes»?

It is real symmetric. The spectral theorem guarantees real eigenvalues and orthogonal eigenvectors — so you can always draw an orthogonal skeleton of principal axes. This is not luck; it is structure.

03What are the eigenvalues of a rotation matrix?

e±iα — complex. A rotation has no real «stretching direction» (every direction gets rotated away); its eigenvectors live in the complex plane. This is precisely the matrix-form confirmation of «complex multiplication = rotation» (T6).

04How do you find the «longest direction» of a data point cloud?

Compute the covariance matrix and take the eigenvector of the largest eigenvalue. A few hundred samples suffice for stability (measured angular deviation about 1° at 500 points), at the cost of one matrix multiplication.

Up next

The full series · 27 articles

T1The Matrix Is a Ledger for Rotation≈ 25 minT2From Slope to Gradient: The First Line of Every Robot Learning Algorithm≈ 25 minT3Vectors and forces: the physical reading of the dot productAbout 20 minT4Two Rulers and a Better Rulerabout 25 minT5The Ledger of One Motor Revolution: Radians, Speed, and Powerabout 20 minT6Complex multiplication is rotation: the one-dimensional version of quaternionsAbout 20 minT7Bases and coordinates: why robots put everything in matrices45 minT8Matrix Multiplication: Chaining Transformations, Not a Table OperationAbout 40 minT9The Jacobian: Rotate a Joint One Degree — How Far Does the End Effector Move?About 45 minT10Eigenvectors: The Directions a Matrix Dare Not Touchabout 45 minT11SVD and the Pseudoinverse: When There Is No Exact Solution, Find the Nearest Oneabout 50 minT12Taylor expansion: putting a straight line to work in place of a curveAbout 40 minT13Probability spaces: why «random» needs axioms50 minT14Expectation: A Rigorous Construction of «Average»About 55 minT15Conditional Expectation: What the Vertical Bar Really Means≈ 60 minT16Four inequalities: the quantitative answer to «how many samples are enough»45 minT17Norms and fixed points: where «it always converges» comes from55 minT18Markov Chains: When the Time Average Equals the Ensemble Averageabout 55 minT19The Mathematical Construction of the MDP: What One Axiom Is Worthabout 50 minT20Dynamic Programming: The Accounting for Two Iterations≈ 50 minT21Optimization: why constraints cannot rely on penalties aloneAbout 55 minT22Stochastic Approximation: Every Term in the Learning-Rate Formulaabout 55 minT23The Mathematics of Contact: Hertz Theoryabout 45 minT24Friction and the Friction Cone: The Geometry of Constraintsabout 40 minT25Force Closure and Grasp Metrics: ε Is a Linear ProgramAbout 50 minT26Contact Kinematics: Why Rolling Is Not «Sliding Along»≈ 45 minT27Slip detection: from «I felt it» to «I computed it»About 50 min