Perception & intelligence · topic C7 · 40 hours
Imitation learning & diffusion policies
The most practical route for manipulation tasks today. No precise model needed — you collect human data. The bottleneck is data, not the model.
What to learn
- Behavior cloning (BC) and DAgger: how to mitigate distribution shift
- ACT versus Diffusion Policy: deterministic output vs multimodal output
- The discipline of teleoperated data collection: consistency, diversity, and whether failure samples belong in
- How deployment-time inference frequency and hysteresis affect closed-loop stability
The lesson · plain words
RL gives you a «score»; imitation learning gives you a «demonstration»: watch how a person does it and learn by copying. Scores are hard to write, demonstrations easy to record — this field exploded over the last two years by turning «learning actions» into «denoising».
The name Diffusion Policy sounds intimidating, but the intuition is one sentence: start from a blob of random noise and grind out an executable action sequence step by step.
Behavior cloning: the most naive, and the easiest to crash
The earliest imitation learning learned frame by frame: whatever images the human supplied, learn to output the matching actions. The problem is compounding error — err 5% per step and after twenty steps you are off 63%, with no data on «how to come back from an error», because the demonstrator never errs.
Two lines of defense: DAgger (the human keeps providing demonstrations for the new states the robot wanders into) and data augmentation (perturb the demonstration trajectories, so the robot has seen both «the off-track state» and «the pull-back-to-track state»).
Diffusion policy: grinding out a whole action segment at once
Watch the animation below: the dashed line is the human demonstration trajectory, the solid line is the network's output being denoised. In training, noise is added to the demonstrations and the network learns to restore them; at inference it starts from pure noise and takes N steps to grind out a smooth action sequence.
The essence is predicting a whole segment at once, not frame by frame: if a task has two equally valid grasps (grab from the left, grab from the right), frame-by-frame learning averages them into something that is neither; diffusion models the distribution — the two grasps form two peaks, and denoising lands on one of them.
Interactive figure · drag it and see
Why robotics fell for this so hard
① Multimodality: real tasks always have several workable approaches, and regression methods ruin them by averaging; ② action chunking: predict 8–16 steps at a time and then execute — naturally smooth; ③ flexible data formats: demonstrations can come from teleoperation, from motion capture, even from another robot's logs.
Where it stands now: on manipulation (pick-and-place, insertion, pouring), diffusion policy is already the baseline; locomotion's mainstream is still RL + classical control. Do not go looking for nails just because you hold a hammer.
The hands-on path
First run the LeRobot or official diffusion-policy demo end to end (single arm + two or three objects); then record 50 of your own demonstrations (teleoperation or hand-guiding); then feel out three knobs: denoising steps (speed vs quality), action horizon (how far ahead to predict), and conditioning (how the image and the state are fed in).
Interview staples: what to do about BC's compounding error, why diffusion handles multimodality, and the scale of demonstration data (50–200 episodes gets you to a demoable level; productization wants 500+ with diversity coverage).
Three questions — do you really get it?
1. What is behavior cloning's «compounding error»?
- ASmall errors accumulate step by step; once off the demonstration distribution, it keeps going wrong to the end
- BA network that is too large overfits
- CPoorly designed rewards
Show answer
A. The demonstrator never errs, so the robot has never seen «how to recover from a deviation», and the error snowballs. DAgger and augmentation exist to treat exactly this.
2. The core reason diffusion policy beats frame-by-frame regression?
- AFaster inference
- BIt needs no demonstration data
- CIt models the action distribution and handles multimodality naturally
Show answer
C. Two correct approaches averaged frame by frame become a wrong one; diffusion learns the distribution, and sampling lands on one of its peaks.
3. When training a diffusion policy, what does the network learn?
- AHow to remove a bit of noise at each step
- BTo output the canonical action directly
- CTo score actions
Show answer
A. Noise the demonstrations, then learn to restore them — what is trained is the denoising process itself, which is precisely why it can «grind» out actions.
Keep going deeper
Links without a marker stay on this site; the rest point to the Chinese edition until its translation batch lands.
How it fails in the field
- All the data collected by one person — the policy learns only that person's habits and fails the moment someone else operates
- Collecting only successful demonstrations: no recovery from deviation, and one small offset collapses the whole episode
- Contact-rich tasks (peg insertion, knot tying, handing over fragile objects) run without tactile observation — the success-rate ceiling is visible to the naked eye — see C9
You have learned enough when
The Chinese edition carries the full topic page for Imitation learning & diffusion policies, including the complete outline, depth requirements per career track, and further derivations: robotguide.dev/k/C7/.