RobotGuide
Frontier/Direction 4

Frontier · Direction 4 · 18 min

Efficient Inference: Planning Pays Compute for Return

decision-time planning must try hundreds of action sequences in latent space at every step (TD-MPC2 defaults to 512 candidates × 6 iterations); real-robot control frequency cannot bear it; the longer the rollout, the greater the compounding of one-step errors. How to trade less search for the same decision quality.

18 min3 paper anchors4 math touchpoints3 pitfalls

What it really is, once the buzzwords are off

The original post got the pain point right but missed two key facts: half of the solutions is Direction 5's hierarchy (temporal abstraction = fewer decision points = less compute), and the other half is pure algorithmic engineering. Moreover the search budget has a clear collapse point — in the on-site experiments, return climbs with budget, and success rate appears only at the maximum budget.

The original post

The original post claims: «Explore state prediction and task planning mechanisms driven by world models, and build inference schemes with low computational overhead and high sample efficiency.»

Post vs reality

TD-MPC2's shortcut planning is the representative idea: first let the learned policy roll out a "default route", and MPC searches only locally around that route — cutting the search volume from the whole space down to the policy's neighborhood. The learned policy becomes a planning prior; this is policy learning and planning cooperating, not either-or.

FOCUS takes another path: the high level outputs only a Bernoulli "preference hint" to select a sparse subset of control units (when a humanoid stands up, only ankles and wrists are attended to), the low level outputs actions on the selected dimensions, with sequential Monte Carlo planning. Enter the Void dynamically decides "when to replan and how long the horizon" — admitting that replanning at every step is wasteful, and introducing commitment mechanisms.

There is also a trap that appears repeatedly in the literature and that the original post never mentions: objective mismatch — a model that predicts accurately is not necessarily a model good for control. The more search, the more confidently it may be wrong.

Real paper anchors

TD-MPC2Hansen et al. · ICLR 2024

Decoder-free latent-space model + MPC; shortcut planning — policy as planning prior, SOTA in continuous control, one model for 317 tasks.

FOCUSOpenReview 2025/2026

High level selects a sparse subset of control units + low level outputs actions + sequential Monte Carlo planning; dimension-reduced search for high-dimensional tasks.

Enter the VoidarXiv 2505.16787

Hierarchical planner dynamically decides when to replan and the horizon length; explicitly manages the waste of "replanning at every step".

Math you need

Concept hereWhere it landsWhere to learn it
What MPC isRolling-horizon truncation of DP; value function as terminal term to compensate the horizon/math/dp/ →
What CEM / MPPI areConvergence and sample complexity of stochastic optimization methods/math/optim/ →
Formalizing the compounding of errorError accumulation on Markov chains/math/markov/ →
The origin of "policy as planning prior"SAC's entropy and reparameterization/fund/sac/ →

A first move you can actually run

How

Take a trained world model, hand-write a CEM planner, sweep the grid of "candidate sequences × iterations", and plot the "imagined forward passes per step vs. return" curve. You will see that when the search budget is cut to one tenth, the return usually drops only a little, and then collapses at a definite point — what efficient inference earns is exactly the stretch before the collapse point.

Result on recordAlready run on-site: expCD_plan.py, Experiment C. 14,400 imagined forward passes give return 1.13; 1.38M give 4.34; success rate appears only at the last tier (0.05). One more trap caught: without receding-horizon warm start, success rate is 0 at every budget — elite averaging averages out-of-phase pump-swing candidates into zero torque. Warm start is a necessity, not an optimization.

Pitfalls

Keep reading

Experiments C+D: the budget curve and the 480× gapMPC's DP accountingThe other half of the solution: hierarchical control
Previous · Direction 3Uncertainty Quantification: Let the Model Know What It Doesn't Know Next · Direction 5Hierarchical Control: Macro-actions Save 480× the Imagined Compute

All 6 directions

1Continual Reinforcement Learning: Learn the New Without Forgetting the Old18 min2Open-Environment Adaptation: Pre-train a Worldview First, Patch Tasks Online Later20 min3Uncertainty Quantification: Let the Model Know What It Doesn't Know16 min4Efficient Inference: Planning Pays Compute for Return18 min5Hierarchical Control: Macro-actions Save 480× the Imagined Compute18 min6Sim-to-Real: Pixel Gap Is the Appearance, Physics Gap Is the Essence18 min