Systems & software · topic D13 · 35 hours
ROS 2 core mechanisms: executors, time & lifecycle
Plenty of people can use ROS 2; very few know how it schedules things internally. Latency jitter, callbacks deadlocking each other, time jumping inside simulation — all three classes of problems have their answers only inside the executor and the time system.
What to learn
- The three forms of executor: single-threaded, multi-threaded, StaticSingleThreaded — more threads is not better, since multi-threaded executor scheduling is itself a source of jitter
- Callback groups: MutuallyExclusive vs Reentrant — deciding which callbacks within one node may run concurrently and which must queue
- The chain reaction of a blocked callback: one callback stalls and every callback in its group queues. «Why did the 30 Hz camera callback starve the 1 kHz control callback?» The answer is here
- Lifecycle nodes: configure → activate → deactivate → cleanup → shutdown, and why production environments use them for safe startup and hot swapping
- The time system: ROS time vs system time, use_sim_time, clock source selection, and the true root of «tf extrapolation errors in simulation»
- TF2: the coordinate tree structure, static vs dynamic broadcasters, time interpolation, and the three classic causes of a broken tf chain (frequency too low / wrong timestamps / wrong parent–child wiring)
- Message interfaces and ABI: code generated from msg / srv / action, package versions and dependencies, and why «just adding one field» can still leave old nodes unable to parse
- The build system: colcon workspaces, ament_cmake vs ament_python, the install and build directories, overlay override order
- The diagnostics toolbox: ros2 doctor / topic hz / bw / bag / ros2_tracing (LTTng) / rqt_graph — each tool answers one specific question; using the wrong one means a wasted investigation
The lesson · plain words
ROS 2's advanced trio: the executor (how callbacks get scheduled), time (system time or ROS time), and lifecycle (the node state machine). These three decide whether your nodes merely «run» or are «reliable».
The executor: how callbacks queue
Single-threaded executor: every callback queues in one line and runs one at a time — simple, no concurrency traps, but one blocking callback (sleep, slow IO) stalls everyone. Watch the figure: drag one callback's duration to 25 ms and 10 callbacks cost 250 ms — two 100 Hz topics cannot be fed.
Multi-threaded executor: callbacks run in parallel, throughput rises, but shared data now needs your own locks. The engineering default is callback groups: serial within a group (safe), parallel across groups (faster) — the compromise when you want multiple cores without locking everything.
Interactive figure · drag it and see
Time: three clocks, never mixed
System time (wall clock, can jump backward when NTP adjusts it), the steady clock (monotonic — measure intervals with it), ROS time (the node clock, drivable by the /clock topic in simulation). Measure latency with steady; run business logic on the node clock (otherwise simulation cannot run); never compute intervals from the wall clock.
Whether simulation training (G5/G6) transfers to the real machine depends half on whether your code uses ROS time throughout — get it right and the same code runs in both sim and reality.
Lifecycle nodes: making «ready» an explicit state
A plain node starts running the moment it is constructed, and a bad config can only be fixed by restarting afterward. A lifecycle node splits startup into a configure → activate → deactivate state machine: configure reads parameters and opens hardware, work begins only after activate, and on failure you can deactivate back to a safe state and try again.
Where it pays off: sensor drivers (configure opens the camera; on failure it stops at UNCONFIGURED, and the system knows explicitly that it is not ready) and hot-swappable control policies. Standard equipment for production-grade nodes.
How deep to go
Configure callback groups for a node to fix «one blocking call drags down the whole node»; switch every time call to node->now()/steady; rewrite one driver node as a lifecycle node. Complete these three refactors and your node moves from demo grade into production grade.
Interview favorites: the single-threaded vs multi-threaded executor trade-off, what a callback group is, why latency is measured with the steady clock, what problem lifecycle nodes solve.
Three questions — do you really get it?
1. One callback sleeps 100 ms inside a single-threaded executor — the consequence?
- AAll callbacks queue for 100 ms
- BOnly that callback is affected
- CIt automatically switches to multi-threading
Show answer
A. A single-threaded executor is serial. Never block inside a callback — the first commandment of ROS 2.
2. To measure the interval between two frames, use?
- AThe system wall clock
- BEither one
- CA steady/monotonic clock
Show answer
C. NTP can pull the wall clock backward. A monotonic clock guarantees correct interval measurement.
3. The core value of a lifecycle node is?
- AMemory savings
- BTurning «ready/failed» into explicit states the system can sense and recover from
- CRunning faster
Show answer
B. The configure/activate state machine makes failures detectable and recovery procedural. Standard for production.
Keep going deeper
Links without a marker stay on this site; the rest point to the Chinese edition until its translation batch lands.
How it fails in the field
- Reaching for a multi-threaded executor to fix everything, ending with callbacks out of order and a state machine stomped by concurrency
- Stuffing every callback into the default mutually exclusive group, where one slow callback drags down the whole chain
- Forgetting use_sim_time in simulation: tf and sensor timestamps differ by decades, and the error message points completely the wrong way
You have learned enough when
The Chinese edition carries the full topic page for ROS 2 core mechanisms: executors, time & lifecycle, including the complete outline, depth requirements per career track, and further derivations: robotguide.dev/k/D13/.