Reinforcement Learning Summer School 2023/2026 - DQN Tutorial

From Tabular Q-Learning to DQN

Blog post: https://araffin.github.io/post/rl102/

Website: https://rlsummerschool.com/

Slides: https://araffin.github.io/slides/rlss26-fqi-dqn/

Stable-Baselines3 repo: https://github.com/DLR-RM/stable-baselines3

RL Virtual School 2021: https://github.com/araffin/rl-handson-rlvs21

RL Summer School 2023: https://rlsummerschool.com/2023/

RL Summer School 2026: https://2026.rlsummerschool.com/

Content

1. Fitted Q-Iteration (FQI) Colab Notebook
2. Deep Q-Network (DQN) Part I: DQN Components: Replay Buffer, Q-Network, ... Colab Notebook
3. Deep Q-Network (DQN) Part II: DQN Update and Training Loop Colab Notebook

Run Locally (instead of using Google colab)

1. Install uv
2. [optional] Create a virtual env with a specific python version: uv venv --python 3.12 --clear
3. Run uv run --with jupyter jupyter lab notebooks

Solutions

Solutions can be found in the notebooks/solutions/ folder. The code in dqn_tutorial package can also be used to bypass some exercises.