Foundations
Derive the formulas until you can write them yourself.
A full derivation path for reinforcement learning: from the four math tools to PPO and SAC, every article walks claim → derivation → hand-workable example → 20-line numpy check → 4 self-test questions. No RL framework needed — every number on these pages was actually computed.
How to read: look at the one-line conclusion and the «why this section exists» first. If they make sense, jump straight to the worked example, verify it by hand, then go back to the derivation. All example numbers were really run — the code is copy-paste executable and needs nothing but numpy.
Every formula in this series is animated somewhere: rotation matrices, Bayesian counting, gradient descent on a hill — all in the 15 interactive labs and on the lesson pages. Understand the phenomenon before the proof.
One-line conclusion → step-by-step derivation (each step names the math tool it uses) → a hand-workable example → a 20-line numpy check → 4 self-test questions
Between «I followed the derivation» and «I can write it myself» there is exactly one hand computation. All example numbers were really run — if yours do not match the page, some sign or boundary is off.
Groundwork: math and modeling
If these three are not solid, everything after is memorized formulas
Value functions: from tables to function approximation
Where «guaranteed convergence» breaks down
Policy gradients: optimizing the policy directly
REINFORCE → GAE → PPO, one continuous derivation
Continuous control and field practice
Maximum entropy, hindsight relabeling, and how to diagnose non-convergence
Relationship to the lesson pages
The 74 lessons tell you what to learn and how deep; this series tells you why the tools in track «algo» actually work. Recommended order: meet the phenomenon on a lesson or lab page first, then read the matching derivation — at that point every symbol has a phenomenon attached to it.