Math foundations · topic A3 · 35 hours
Optimization (convex / QP)
WBC, MPC, trajectory optimization, calibration, the SLAM back end — at their core they all solve optimization problems. This is the backbone of modern robot algorithms.
What to learn
- Recognizing convex sets and convex functions: knowing which problems «always have a solution, and a unique one»
- The QP standard form, KKT conditions, soft constraints and slack variables (slack variables are the escape hatch when no solution exists)
- Which solver fits which scenario: what problem sizes and structures OSQP, qpOASES, and IPOPT each suit
- Numerical ill-conditioning and regularization: why adding one tiny quadratic term makes the solution stable
- Writing «reach out while keeping balance» as a constrained quadratic program
The lesson · plain words
Robots spend every day finding the best approximation inside «equations that cannot be solved exactly»: the desired pose is out of reach, so find the closest joint angles; obstacle avoidance and reaching the goal conflict, so find the compromise. Optimization answers exactly this question — when no perfect solution exists, how to pick the least-bad one.
The core is two terms: the objective function (what you want to make small) and the constraints (what must not be violated); everything else is algorithms. This page makes the three most-used pieces clear.
The gradient: the «downhill direction» at your current position
Stand on a hillside, close your eyes, and turn in place: the steepest downhill direction is the negative gradient. Gradient descent = each step, move a little in the steepest direction. How far? The learning rate decides: too small and you dawdle forever before reaching the valley; too large and one step leaps across the valley bottom and bounces onto the opposite slope — divergence.
Verify it with your own hands in the animation below: drag the ball onto the slope, then drag the learning-rate slider, and watch how the three levels 0.1 / 0.25 / 0.6 meet completely different fates. Every model and every controller you tune later — this is the number you are tuning.
Interactive figure · drag it and see
Convexity: a world with only one valley bottom
A convex function is a «bowl». The bowl's virtue: the local minimum = the global minimum, and rolling blindfolded still lands you in the same bottom. A non-convex function is rolling hills plus saddle points (this is exactly why training neural networks is hard).
Robotics loves convexity for one more reason: quadratic objective + linear constraints = a QP (quadratic program) — unique solution, solvable in milliseconds, mature solvers available (OSQP and friends). The MPC, WBC, and every trajectory optimization in the B series are QPs underneath — not a coincidence; engineers deliberately squeeze problems into QP form.
Interactive figure · drag it and see
A form you will meet again and again: damped least squares
In manipulator inverse kinematics — «given tip velocity v, find joint velocity q» — the equation Jq = v often has no exact solution (the target is out of reach, or you are too close to a singularity). The standard engineering fix is minimizing ‖Jq − v‖² + λ‖q‖²: the first term says «get to the target as best you can», the second says «joints, do not flail» — λ is the elastic band.
Large λ: rock solid but never arrives accurately; small λ: hits exactly where you point, but the joints twitch and velocities blow up near singularities. Tuning λ is not a math problem; it is the knob between «getting there» and «staying stable». Every KDL/TracIK parameter you configure in MoveIt has this underneath.
Three sentences to take away
① gradient = the downhill direction, learning rate = the step size; ② convex = only one bottom, QP = a millisecond-solvable standard part; ③ when you see min / argmin, find the objective function and the constraints first — the algorithm always comes second.
Three questions — do you really get it?
1. What happens if you set the learning rate too large?
- AConvergence slows, but it still arrives
- BNothing changes at all
- CIt oscillates across the valley bottom, or even diverges
Show answer
C. The step leaps over the valley bottom and bounces back and forth; past the critical value, every step lands farther from the bottom than the last — divergence.
2. The biggest advantage of a convex function is?
- AIt is always faster to compute
- BNo learning rate is needed
- CThe local optimum is the global optimum
Show answer
C. A bowl has only one bottom; however it rolls, it drops into the same pit — «stuck in a local optimum» simply does not exist.
3. In damped least squares, what does increasing λ do?
- AHigher reaching accuracy
- BMore stable, but less accurate
- CThe algorithm runs slower
Show answer
B. λ is the weight of «do not flail»: raising it shrinks the joint velocities (stable) at the cost of a larger residual (inaccurate).
Keep going deeper
Links without a marker stay on this site; the rest point to the Chinese edition until its translation batch lands.
How it fails in the field
- Constraints conflict with each other and the problem has no solution, the code has no fallback, and the robot freezes on the spot
- Forcing a non-convex problem into a QP solver: the result depends on the initial guess, works sometimes and fails others, and you blame the sensors
You have learned enough when
The Chinese edition carries the full topic page for Optimization (convex / QP), including the complete outline, depth requirements per career track, and further derivations: robotguide.dev/k/A3/.