This course aims to teach from the basics of RL to advanced algorithms such as GAE.
- Machine Learning (Gradient Descent, Neural Networks)
- Basic Probability Theory (Expectations and Distributions)
- Multivariate Calculus
- Python and PyTorch
Each module consists of:
- Formal mathematical definitions and theory
- Step-by-step algorithm derivations
Parts 1 and 2 also include:
- Complete PyTorch implementations
- Runnable experiments
I recommend working through the notebooks carefully, especially the mathematical derivations and proofs, and ensuring you understand each concept before moving on. This material is designed to be precise and concise so that you can learn efficiently - without rushing through.
| Module | Topic | Colab/Post | Key Concepts |
|---|---|---|---|
| Part 1 | RL Basics & Policy Gradients |  |
MDPs, Policies, Trajectories, Policy Gradient Theorem, Reward-to-go |
| Part 2 | Discounting | |
Temporal Discounting, Convergence of Infinite Horizons, Variance Reduction |
| Part 3 | Baselines, Actor-Critic & GAE |  |
Constant Baselines, Q-value, Value and Advantage Functions, Actor-Critic, Group-dependent Baselines, GAE |