ARIN5204: Reinforcement Learning

Spring 2026, Dept. of Computer Science and Engineering (CSE), The Hong Kong University of Science and Technology (HKUST)

Instructor: Long Chen
Class Time & Location: Monday 7:30PM - 10:20PM (RM2465, Lift 25&26)
For the lecture on Feb 16th (Spring Festival Eve), we change the schedule to Feb 14th 10:00AM - 12:50PM (RM), RM2465

Email: longchen@ust.hk (For course-related queries, please use the subject starting from [ARIN5204])
Office Hours: There is no specific office hour, you can directly ask any questions after the lecture.
Teaching Assistant: Ziqi Jiang (zjiangbl@connect.ust.hk) and Yanghao Wang (ywangtg@connect.ust.hk)

Course Description: Reinforcement learning (RL) is a computational learning approach where an agent tries to maximize the total amount of reward it receives while interacting with a complex and uncertain environment. It not only shows strong performance in lots of games (such as Go), but also becomes an essential technique in many today’s real-world applications (such as LLM training, and embodied AI). This course aims to teach the fundamentals and the advanced topics of RL. The course content includes the introduction of basic RL elemnets (including MDP, dynamic programming, policy iteration), value-based approaches (DQN), policy-based approaches (policy gradient, actor critic), model-based RL, and RL techniques in today’s computer vision or AI applications. To better enhance the understanding, we will also contain some Python/Pytorch implementations.

Pre-requisite:

Math: You should have familiar background in Linear Algebra (e.g., matrix inversion) and Probability (e.g., expectation, sampling).
Machine Learning: Basic machine learning knowledge (e.g., gradient backpropagation) and deep learning knowledge (e.g., neural network) as needed.
Programming: Python, PyTorch (necessary for assignment)

Grading scheme:

• In-class Quiz: 15%
• Assignment: 20%
• Midterm: 20%
• Final Exam: 45%

Reference books/materials:

Richard S. Sutton, Andrew G. Barto. Reinforcement Learning: An Introduction. Second Edition. [pdf]
Kevin P. Murphy. Reinforcement Learning: An Overview. [pdf]
Alekh Agarwal, Nan Jiang, Sham M. Kakade, Wen Sun. Reinforcement Learning: Theory and Algorithms. [pdf]
Csaba Szepesvari. Algorithms for Reinforcement Learning. [pdf]
Dimitri P. Bertsekas. Reinforcement Learning and Optimal Control. [pdf]

Content Coverage

• Markov Decision Processes
• Dynamic Programming
• Monte Carlo and Temporal Difference Learning
• Q-Learning
• DQN and advanced techniques
• Policy Gradient
• Actor Critic
• Advanced Policy Gradient
• Continuous Controls
• Imitation Learning
• Model-based RL

Lecture Syllabus / Schedule

Acknowledgements

This course was inspired by and/or uses reserouces from the following courses:

Reinforcement Learning by David Silver, DeepMind, 2015.
CS285: Deep Reinforcement Learning by Sergey Levine, UC Berkeley, 2023.
CS234: Reinforcement Learning by Emma Brunskill, Stanford University, 2024.
10-403: Deep Reinforcement Learning by Katerina Fragkiadaki, Carnegie Mellon University, 2024.
Special Topics in AI: Foundations of Reinforcement Learning by Yuejie Chi, Carnegie Mellon University, 2023.
CS 6789: Foundations of Reinforcement Learning by Wen Sun and Sham Kakade, Cornell Univeristy.
CS224R: Deep Reinforcement Learning by Chelsea Finn, Stanford University, 2023.
DeepMind x UCL RL Lecture Series by Hado van Hasselt, DeepMind, 2021.