RobotGuide

Systems & software · topic D14 · 45 hours

ROS 2 internals: middleware, zero-copy & real-time

«Retrofitting ROS 2» does not mean editing its source; it means knowing which layers can be swapped and what each swap costs. In industrial settings the real dividing line is one sentence: which segment of the control loop should not run inside ROS 2 at all.

Specialization stage · Hardcore
Algorithm · Aware System software · Proficient Embedded & controls · Proficient

Tools
Tools

What to learn

  • The layers you can touch: application / rclcpp / rcl / rmw / DDS implementation / kernel — what each layer allows you to change, where the boundaries are, and where a change propagates
  • Swapping middleware: Fast DDS ↔ Cyclone DDS ↔ a self-built rmw — one RMW_IMPLEMENTATION environment variable switches it, but discovery behavior, memory footprint, and large-message throughput all change
  • The real conditions for zero-copy: same process (intra-process), loaned messages, iceoryx shared memory; across machines it is never zero-copy, and across processes only on the same machine
  • The real-time trio: a PREEMPT_RT kernel + memory preallocation and locking (mlockall) + CPU affinity and thread priorities — plus the things a real-time thread must absolutely never do
  • Moving the 1 kHz control loop out of ROS 2: control runs on an MCU or a standalone real-time thread while ROS 2 handles configuration, observability, and non-real-time decisions — the mainstream industrial approach, not a compromise
  • micro-ROS: pushing ROS 2 nodes down onto MCUs (STM32 / ESP32) over XRCE-DDS on serial or UDP — which scenarios are worth it and which are self-inflicted trouble
  • Trimming and static linking: deleting unused packages, static linking, squeezing boot time and memory footprint — how small a minimal ROS 2 system can get
  • Multi-machine and networking: DDS domain ID isolation, multicast discovery vs unicast peer lists, cross-subnet and VPN scenarios, and shared memory working only on the same machine
  • Security and migration: the boundary of SROS2's authenticated encryption (what it protects and what it does not), using ros1_bridge, and when you should not migrate at all

The lesson · plain words

ROS 2's defaults prioritize «convenience»: messages serialize, and every cross-process hop copies. Once images and point clouds get big, bandwidth and latency turn ugly immediately. The real-time retrofit means switching those defaults to «performance» first.

The default path spends all its money on copying

Watch the figure: on the default path a 2 MB image frame goes through serialize → three copies → deserialize — 2.8 ms per frame and 168 MB/s of memory bandwidth at 30 Hz. Zero-copy (loaned messages / intra-process communication) passes the pointer directly: 0.4 ms and 24 MB/s.

Judging whether it is worth it: big messages within one process on one machine (images, point clouds, maps) are worth it; across machines, or for small messages (control commands of a few dozen bytes), do not bother — the serialization overhead is already tiny.

Interactive figure · drag it and see

What this figure shows: a publish/subscribe pair with two routes — the default path serializing, copying ×3, then deserializing (≈2.8 ms per 2 MB frame), versus zero-copy handing over a loaned pointer (≈0.4 ms), plus the memory-bandwidth bill of one 2 MB frame × 30 Hz in MB/s. Takeaway: zero-copy pays off only when messages are big, on the same machine, under the same DDS config — zero-copy the images and clouds first; small control messages are not worth it.

The real-time retrofit checklist

① Communication layer: the intra-process executor + loaned messages. ② Memory layer: preallocate the hot path (D1), rclcpp memory policies (no runtime allocation). ③ Scheduling layer: SCHED_FIFO + core pinning (D4). ④ Clock layer: the steady clock + fixed-size data structures.

Acceptance: a tracelog (ros2 trace) shows the end-to-end message latency distribution; cyclictest validates the 1 kHz control loop's scheduling. Run it once before and once after the retrofit — let the numbers speak.

Middleware replaceability: beyond DDS

ROS 2's RMW layer supports swappable implementations: Fast DDS, CycloneDDS, Zenoh. For large messages Zenoh (or iceoryx2) stands out; across subnets CycloneDDS often wins. Switching RMW changes one environment variable — that is the dividend of layered architecture.

Choose from measurements: run the same test script against all three RMWs and compare latency distributions and reconnection behavior. Community defaults keep changing — never worship any one RMW as «better»; let your own data speak.

How deep to go

Compress an image pipeline from 2.8 ms to under 0.5 ms (the intra-process + loaned + preallocation trio); configure an RMW and produce comparison data; state clearly «which paths should not be real-time-ified».

Interview favorites: the conditions under which zero-copy applies, why small control messages do not need zero-copy, and the architectural significance of a replaceable RMW.

Three questions — do you really get it?

1. Where is zero-copy most worth using?

  • ABig image/point-cloud messages between processes on the same machine
  • BSending control commands across machines
  • CAny topic
Show answer

A. Only big messages on the same machine have copy overhead worth saving. Across machines you pay the NIC, and serialization cannot be avoided.

2. What actually changes when you swap the DDS implementation in ROS 2?

  • ABusiness code
  • BYou must rewrite the nodes
  • COnly the RMW environment variable — the code stays untouched
Show answer

C. That is the value of the RMW abstraction layer: pluggable middleware. Pick it with measured data.

3. In the real-time retrofit, what problem does the «memory strategy» solve?

  • ASaving memory
  • BLeak prevention
  • CBanning runtime allocation on the hot path, removing malloc's nondeterministic latency
Show answer

C. The enemy of real time is nondeterminism. Preallocation + no runtime allocation makes every frame's cost stable.

Keep going deeper

Links without a marker stay on this site; the rest point to the Chinese edition until its translation batch lands.

How it fails in the field

  • Assuming that setting RMW_IMPLEMENTATION counts as «optimization», never having compared latency distributions before and after the swap
  • Allocating memory, logging, and grabbing plain mutexes inside real-time callbacks — real-time behavior was gone from the first line of code
  • Running a 1 kHz servo loop directly in ROS 2, then endlessly applying real-time patches to save it, always one order of magnitude short no matter how you tune

You have learned enough when

You can draw your system's layer diagram marking «which layers I changed, and the benefit and cost of each change»; say clearly which segment must move out of ROS 2 and where it should go; and produce a before/after latency distribution comparison (P50 / P99 / max) for a middleware swap.

The Chinese edition carries the full topic page for ROS 2 internals: middleware, zero-copy & real-time, including the complete outline, depth requirements per career track, and further derivations: robotguide.dev/k/D14/.