RobotGuide

Systems & software · topic D2 · 30 hours

Python & scientific computing

Prototyping, data analysis, training scripts, visualization. In robotics Python never runs the control loop, but it is everywhere.

Foundations stage · Beginner
Algorithm · Proficient System software · Hands-on Embedded & controls · Aware Mechanical · Aware

Tools
Tools

What to learn

  • numpy vectorized thinking: why for loops must become matrix operations
  • matplotlib plots you can actually read (latency distributions, error curves, time-aligned series)
  • The prototype vs real-time boundary: what stays in Python, what must sink down to C++
  • Environment isolation and dependency locking, avoiding 'works on my machine'

The lesson · plain words

Robotics Python lives in two worlds: 'glue' (launch scripts, tooling, ROS node prototypes) and 'data' (processing point clouds, training, evaluation). The first can be written any way you like; the second requires NumPy.

This page makes one point only: why vectorization is the ticket into Python data processing.

Vectorization: pushing the loop down into C

Summing a million numbers: a pure Python loop takes hundreds of milliseconds, one NumPy line takes a few — two orders of magnitude apart. The cause is not 'Python is slow' but 'the interpreter explains one number at a time'; NumPy pushes the loop down into C and computes a whole batch per pass.

Drag the data size in the animation: the gap always stays at the hundred-fold level. So the criterion is simple: the moment a for loop iterating over an array shows up in data-processing code, first ask whether it can be rewritten as array operations.

Interactive figure · drag it and see

What this figure shows: timing bars comparing a pure Python loop, NumPy vectorized code, and Numba JIT on summing N numbers (N on a log-scale slider), with runtimes in milliseconds and the vectorized speedup factor displayed. Takeaway: vectorized code is roughly a hundred times faster because the loop sinks into compiled C and processes a whole batch per pass — in data-prep scripts, 90% of 'Python is slow' is just missing vectorization.

The NumPy core trio

Slicing and fancy indexing (views that copy no data); broadcasting (different shapes align automatically); axis (which dimension to compute along). Master these three and point-cloud processing, batched image operations, and matrix code all compress into a dozen lines.

Two high-frequency traps: modifying a view also changes the original array (use .copy()); mismatched shapes broadcast silently into garbage (guard with an assert on shape).

The right moves in robot scenarios

Point clouds: vectorized filtering with Open3D/numpy, never per-point loops. Calibration/transforms: SciPy's Rotation class, not hand-written trigonometry. Visualization: matplotlib 3D or Open3D — on-site tuning depends on it.

When performance falls short, three moves in order: vectorize first; then numba (@jit on the hot loop); only then switch to a C++ node (D1). Keep the order — 90% of cases are solved by step one.

How deep to go

Without consulting docs you can write: filtering a point cloud by z value, batch coordinate transforms, grouping by distance threshold. You can explain why np.array's dtype silently truncates. You can write a data-processing script plus assertions plus timing.

Interview favorites: deep vs shallow copy, what the GIL is, and whether CPU-bound tasks should use processes or threads (answer: multiprocessing — the GIL blocks threads).

Three questions — do you really get it?

1. The fundamental reason NumPy beats Python loops by two orders of magnitude?

  • AIt uses the GPU
  • BIt uses more memory
  • CThe loop is pushed down into compiled C code, removing per-element interpretation overhead
Show answer

C. Vectorization compresses N rounds of interpretation overhead into one C loop.

2. After b = a[0:100], modifying b — what happens to a?

  • ANothing
  • BIt depends on the dtype
  • CIt changes too — a slice is a view, not a copy
Show answer

C. Slices return views; an independent copy requires .copy(). This is the number-one source of data being 'silently dirtied'.

3. To parallelize a CPU-bound Python task, use?

  • Athreading
  • Bmultiprocessing
  • Casyncio
Show answer

B. The GIL stops threads from executing Python bytecode in parallel; processes each carry their own interpreter and truly parallelize.

Keep going deeper

Links without a marker stay on this site; the rest point to the Chinese edition until its translation batch lands.

How it fails in the field

  • Running the control loop in Python and claiming 'latency is only a few milliseconds' — you measured the mean, not the worst case
  • Data analysis scripts with no version pinning and no random seed — conclusions cannot be reproduced

You have learned enough when

You can rewrite a piece of numerical code from a for loop into vectorized form and give the speedup ratio, and plot a latency distribution ready to drop straight into a report.

The Chinese edition carries the full topic page for Python & scientific computing, including the complete outline, depth requirements per career track, and further derivations: robotguide.dev/k/D2/.