Foundations T4 / 27 · From High School
Two Rulers and a Better Ruler
Mean, variance, and the normal distribution are high-school statistics. Using only these, this article derives the core of the Kalman filter: two imprecise measurements, combined, can be more accurate than either alone.
One-line takeaway
At the single-fusion step, the Kalman filter is mathematically a weighted average. The weights are inverse to the variance — it sounds scary, but computing it is a complete-the-square minimization problem.
Animated · drag the widths of two bell curves; the fused result is always narrower than either ruler — inverse-variance weighting, animated
The /labs/kalman/ bench is playable right now; the deep version in /math/expect/ needs the probability axioms. This article is the third route from high-school statistics straight to the bench.
Why this tool exists
Two rulers measure the same shaft: one ±1mm, one ±3mm, each measured once. Whom do you trust? How much of each?
The answer is not to throw away the worse one but a weighted average: trust the precise one more, the scattered one less.
After fusing, the variance is smaller than either of the two rulers — combined is better has a quantitative version.
Definitions
Variance and the normal distribution
Variance Var = the mean of squared deviations; standard deviation σ = √Var. In a normal distribution, about 68% of samples fall within μ±σ and 95% within μ±2σ.
Why define it this wayVariance measures the scatter of a measurement — the quantitative basis for how much to trust it: the larger σ, the more scattered, the less you should trust it fully.
Theorems and proofs
Theorem (inverse-variance weighting)
Two unbiased estimates x̂₁ (variance σ₁²) and x̂₂ (σ₂²), combined as x̂ = w·x̂₁ + (1−w)·x̂₂. The variance-minimizing weight is w = σ₂²/(σ₁²+σ₂²), and the fused variance σ² = σ₁²σ₂²/(σ₁²+σ₂²) is smaller than even the smaller of the two.
Proof
Under independence, variances add directly: Var(x̂) = w²σ₁² + (1−w)²σ₂². Expanded, it is a quadratic function of w (convex, opening upward).
Differentiate in w and set to zero: 2w·σ₁² − 2(1−w)·σ₂² = 0, which gives w = σ₂²/(σ₁²+σ₂²). Note that the numerator is the other ruler's variance — the less reliable the neighbor, the lower its weight.
Substitute back and rearrange: Var(x̂) = σ₁²σ₂²/(σ₁²+σ₂²). To show it ≤ min(σ₁², σ₂²), assume σ₂² ≥ σ₁² without loss; then the denominator of the fused expression ≥ σ₂², so it ≤ σ₁²σ₂²/σ₂² = σ₁².
NoteCombined is better is a theorem, not a slogan. The Kalman gain K of the Kalman filter at a single fusion is exactly this w extended to the matrix version.
Work it by hand
σ₁ = 1mm, σ₂ = 3mm. Compute the weight and the fused standard deviation.
- w = σ₂²/(σ₁²+σ₂²) = 9/(1+9) = 0.9 — the good ruler takes nine tenths.
- Fused variance = 1×9/(1+9) = 0.9 mm².
- σ = √0.9 ≈ 0.949mm.
Fused: ±0.95mm, better than the good ruler (±1mm). Throwing away the bad one actually does worse than using both.
Run the numbers
Where the numbers come from
Monte Carlo actually run (computed for real by the generator): each ruler measures 20000 times, true value 100mm, fused with weights 0.9/0.1:
- The sample standard deviation of the fused result = 0.95mm, matching the formula's √0.9 ≈ 0.949mm to the third decimal place.
- Two independent pieces of information fused together are never worse than either alone — this is the chassis of all of estimation theory.
Where this shows up
| The object here | The counterpart in RL | Used in |
|---|---|---|
| Inverse-variance weight w | The Kalman gain K (the single-fusion special case) | used in → |
| Normality assumption | Probability measures (the deep version) | used in → |
| Mean and variance | Expectation and concentration inequalities | used in → |
Boundaries: where this article stops
This article stops at static, two measurements. Time evolution between measurements (the predict step), multi-dimensional states (the matrix version of the gain), non-Gaussian noise (particle filters) live in /k/B7/ and /labs/kalman/.
Self-test: 3 questions
All hand-computation and «find the flaw in this proof» questions — nothing to memorize. Stuck? Go back to the section.
01Two identical rulers (σ₁=σ₂) fused together: what does σ become?
w=0.5, fused variance = σ²/2, and σ becomes 1/√2 ≈ 0.707 of one — measuring twice really is better than measuring once.
02Why use inverse-variance weights instead of 0.5?
Trusting the unreliable ruler drags the fused result apart. The inverse-variance weight is the solution of minimizing the fused variance, not an aesthetic choice.
03Which two assumptions does combined is more accurate rest on? What if they break?
Unbiasedness (correct mean) + independence. With systematic bias, no amount of weighting removes it (bias does not average out); with correlation, the variance formula needs a covariance term — the entrance to the deep version.