Learning to act

Imitation learning copies you. Reinforcement learning finds behaviour you never demonstrated. It's how the duck learns to walk, and it's mostly done in simulation.

Do this

• The Hugging Face Deep RL Course. Free, hands-on, friendly. Do it start to finish; it's built for self-learners.
• Spinning Up in Deep RL (OpenAI). The clearest written introduction to the algorithms, with reference code. Read the key concepts and the PPO section.
• CS285: Deep Reinforcement Learning (UC Berkeley, Sergey Levine). The serious course. Lectures 1 to 9 after the two above. Skip the homework unless you have the time; watch for the ideas.
• Reinforcement Learning lectures (UCL, David Silver). The classic foundations, if you want the theory behind PPO explained slowly and well.
• MuJoCo Playground. GPU-accelerated environments that train in minutes. This is the toolkit the duck uses.

Build

Train a PPO agent on a MuJoCo locomotion task in the Playground until it walks. Change the reward and watch what it does instead. Keep a log of every reward you tried and what you got. That log is the real deliverable.

Skip

• Value-based methods beyond the idea of Q-learning.
• Anything that needs more than one GPU.

Done when: something you trained walks in simulation, and you can explain what the reward does and why the agent cheats.