RobotGuide
Math foundations/Two Rulers and a Better Ruler

Foundations T4 / 27 · From High School

Two Rulers and a Better Ruler

Mean, variance, and the normal distribution are high-school statistics. Using only these, this article derives the core of the Kalman filter: two imprecise measurements, combined, can be more accurate than either alone.

about 25 min1 theorems3 proof steps3 self-test questions

One-line takeaway

At the single-fusion step, the Kalman filter is mathematically a weighted average. The weights are inverse to the variance — it sounds scary, but computing it is a complete-the-square minimization problem.

Animated · drag the widths of two bell curves; the fused result is always narrower than either ruler — inverse-variance weighting, animated

The /labs/kalman/ bench is playable right now; the deep version in /math/expect/ needs the probability axioms. This article is the third route from high-school statistics straight to the bench.

Why this tool exists

Two rulers measure the same shaft: one ±1mm, one ±3mm, each measured once. Whom do you trust? How much of each?

The answer is not to throw away the worse one but a weighted average: trust the precise one more, the scattered one less.

After fusing, the variance is smaller than either of the two rulers — combined is better has a quantitative version.

Definitions

Variance and the normal distribution

Variance Var = the mean of squared deviations; standard deviation σ = √Var. In a normal distribution, about 68% of samples fall within μ±σ and 95% within μ±2σ.

Why define it this wayVariance measures the scatter of a measurement — the quantitative basis for how much to trust it: the larger σ, the more scattered, the less you should trust it fully.

Theorems and proofs

Theorem (inverse-variance weighting)

Two unbiased estimates x̂₁ (variance σ₁²) and x̂₂ (σ₂²), combined as x̂ = w·x̂₁ + (1−w)·x̂₂. The variance-minimizing weight is w = σ₂²/(σ₁²+σ₂²), and the fused variance σ² = σ₁²σ₂²/(σ₁²+σ₂²) is smaller than even the smaller of the two.

Proof

Under independence, variances add directly: Var(x̂) = w²σ₁² + (1−w)²σ₂². Expanded, it is a quadratic function of w (convex, opening upward).

Differentiate in w and set to zero: 2w·σ₁² − 2(1−w)·σ₂² = 0, which gives w = σ₂²/(σ₁²+σ₂²). Note that the numerator is the other ruler's variance — the less reliable the neighbor, the lower its weight.

Substitute back and rearrange: Var(x̂) = σ₁²σ₂²/(σ₁²+σ₂²). To show it ≤ min(σ₁², σ₂²), assume σ₂² ≥ σ₁² without loss; then the denominator of the fused expression ≥ σ₂², so it ≤ σ₁²σ₂²/σ₂² = σ₁².

NoteCombined is better is a theorem, not a slogan. The Kalman gain K of the Kalman filter at a single fusion is exactly this w extended to the matrix version.

Work it by hand

σ₁ = 1mm, σ₂ = 3mm. Compute the weight and the fused standard deviation.

  1. w = σ₂²/(σ₁²+σ₂²) = 9/(1+9) = 0.9 — the good ruler takes nine tenths.
  2. Fused variance = 1×9/(1+9) = 0.9 mm².
  3. σ = √0.9 ≈ 0.949mm.
Answer

Fused: ±0.95mm, better than the good ruler (±1mm). Throwing away the bad one actually does worse than using both.

Run the numbers

Where the numbers come from

Monte Carlo actually run (computed for real by the generator): each ruler measures 20000 times, true value 100mm, fused with weights 0.9/0.1:

  • The sample standard deviation of the fused result = 0.95mm, matching the formula's √0.9 ≈ 0.949mm to the third decimal place.
  • Two independent pieces of information fused together are never worse than either alone — this is the chassis of all of estimation theory.

Where this shows up

The object hereThe counterpart in RLUsed in
Inverse-variance weight wThe Kalman gain K (the single-fusion special case)used in →
Normality assumptionProbability measures (the deep version)used in →
Mean and varianceExpectation and concentration inequalitiesused in →

Boundaries: where this article stops

This article stops at static, two measurements. Time evolution between measurements (the predict step), multi-dimensional states (the matrix version of the gain), non-Gaussian noise (particle filters) live in /k/B7/ and /labs/kalman/.

Self-test: 3 questions

All hand-computation and «find the flaw in this proof» questions — nothing to memorize. Stuck? Go back to the section.

01Two identical rulers (σ₁=σ₂) fused together: what does σ become?

w=0.5, fused variance = σ²/2, and σ becomes 1/√2 ≈ 0.707 of one — measuring twice really is better than measuring once.

02Why use inverse-variance weights instead of 0.5?

Trusting the unreliable ruler drags the fused result apart. The inverse-variance weight is the solution of minimizing the fused variance, not an aesthetic choice.

03Which two assumptions does combined is more accurate rest on? What if they break?

Unbiasedness (correct mean) + independence. With systematic bias, no amount of weighting removes it (bias does not average out); with correlation, the variance formula needs a covariance term — the entrance to the deep version.

Up next

The full series · 27 articles

T1The Matrix Is a Ledger for Rotation≈ 25 minT2From Slope to Gradient: The First Line of Every Robot Learning Algorithm≈ 25 minT3Vectors and forces: the physical reading of the dot productAbout 20 minT4Two Rulers and a Better Rulerabout 25 minT5The Ledger of One Motor Revolution: Radians, Speed, and Powerabout 20 minT6Complex multiplication is rotation: the one-dimensional version of quaternionsAbout 20 minT7Bases and coordinates: why robots put everything in matrices45 minT8Matrix Multiplication: Chaining Transformations, Not a Table OperationAbout 40 minT9The Jacobian: Rotate a Joint One Degree — How Far Does the End Effector Move?About 45 minT10Eigenvectors: The Directions a Matrix Dare Not Touchabout 45 minT11SVD and the Pseudoinverse: When There Is No Exact Solution, Find the Nearest Oneabout 50 minT12Taylor expansion: putting a straight line to work in place of a curveAbout 40 minT13Probability spaces: why «random» needs axioms50 minT14Expectation: A Rigorous Construction of «Average»About 55 minT15Conditional Expectation: What the Vertical Bar Really Means≈ 60 minT16Four inequalities: the quantitative answer to «how many samples are enough»45 minT17Norms and fixed points: where «it always converges» comes from55 minT18Markov Chains: When the Time Average Equals the Ensemble Averageabout 55 minT19The Mathematical Construction of the MDP: What One Axiom Is Worthabout 50 minT20Dynamic Programming: The Accounting for Two Iterations≈ 50 minT21Optimization: why constraints cannot rely on penalties aloneAbout 55 minT22Stochastic Approximation: Every Term in the Learning-Rate Formulaabout 55 minT23The Mathematics of Contact: Hertz Theoryabout 45 minT24Friction and the Friction Cone: The Geometry of Constraintsabout 40 minT25Force Closure and Grasp Metrics: ε Is a Linear ProgramAbout 50 minT26Contact Kinematics: Why Rolling Is Not «Sliding Along»≈ 45 minT27Slip detection: from «I felt it» to «I computed it»About 50 min