Teacher(s)
Language
English
> French-friendly
> French-friendly
Prerequisites
Artificial Intelligence, as covered by LINFO1361
Main themes
- Foundations of Reinforcement Learning (RL)
- Multi-armed bandits and exploration/exploitation
- Markov Decision Processes (MDP)
- Solving with dynamic programming
- Monte Carlo methods
- Temporal Difference Learning methods (Q-learning)
- Deep Reinforcement Learning
- Value function approximations (DQN and variants)
- Policy gradient methods (REINFORCE, AC, PPO)
- Monte Carlo Tree Search
- Large Reasoning Models and RL from Human Feedback
- Applications to games and simulated environments
- Contemporary challenges, limitations, and perspectives of RL
Learning outcomes
At the end of this learning unit, the student is able to : | |
Given the learning outcomes of the "Master in Computer Science and Engineering" program, this course contributes to the development, acquisition and evaluation of the following learning outcomes:
|
|
Content
The course is organized around three main themes:
- Classical reinforcement learning algorithms.
- Deep reinforcement learning algorithms.
- The use of these algorithms in several real-world applications.
- General introduction to Reinforcement Learning (RL) (agent, environment, states, actions, rewards, policy, value functions, convergence).
- Multi-Armed Bandits (Exploration/Exploitation, ε-greedy, Upper Confidence Bound).
- Markov Decision Processes: formalism and dynamics (Markov property, stochastic vs. deterministic policies, action-value functions, Bellman equation, optimality).
- Solving MDPs using Dynamic Programming (policy evaluation, policy iteration, value iteration).
- Monte Carlo Methods (estimation of state values and action values, convergence).
- Temporal Difference Learning (bootstrapping, TD errors, variance, online learning).
- Q-Learning algorithms.
- Monte Carlo Tree Search and deep variants.
- Imitation Learning (behavioral cloning, DAgger).
- Function Approximation and Deep Q-Networks (nonlinear approximation, DQN).
- Policy Approximation Methods (REINFORCE, Actor-Critic, Proximal Policy Optimization).
- Reinforcement learning for large language models (RLHF, RLAIF, DPO, RLVR, etc.)
- Reinforcement learning for games (AlphaGo, AlphaZero, MuZero, etc.)
Teaching methods
The course combines lectures, group work, and individual work.
Evaluation methods
For the first session, the assessment consists of three parts:
For the second session, the exam and individual exercises may be retaken under the same weighting and conditions. The project grade is retained across all sessions and the project cannot be retaken.
In the event of re-enrollment, all assessments must be taken again.
- Individual comprehensive final exam (50%). The use of AI is not permitted for this assessment. The exam may be answered in English or French.
- Individual exercises (20%): exercises to be completed individually will be assigned throughout the session. The use of AI is not permitted for this assessment.
- Team project (30%): implementation of an agent based on reinforcement learning. The use of AI is permitted for this assessment, in accordance with the rules defined by the faculty.
For the second session, the exam and individual exercises may be retaken under the same weighting and conditions. The project grade is retained across all sessions and the project cannot be retaken.
In the event of re-enrollment, all assessments must be taken again.
Online resources
Available on Moodle (https://moodle.uclouvain.be/course/view.php?id=10915)
Bibliography
Part of the content is based on the following book:
exam material. It nevertheless provides an excellent and comprehensive introduction to the foundations and classical algorithms of reinforcement learning. Its website also offers complementary resources related to the book. Additional resources and scientific papers will be provided throughout the semester, particularly for topics in deep reinforcement learning and recent applications.
- Reinforcement Learning: An Introduction (2nd Edition) - http://incompleteideas.net/book/the-book-2nd.html
exam material. It nevertheless provides an excellent and comprehensive introduction to the foundations and classical algorithms of reinforcement learning. Its website also offers complementary resources related to the book. Additional resources and scientific papers will be provided throughout the semester, particularly for topics in deep reinforcement learning and recent applications.
Faculty or entity
Programmes / formations proposant cette unité d'enseignement (UE)
Title of the programme
Sigle
Credits
Prerequisites
Learning outcomes
Master [120] in Data Science : Statistic
Master [120] in Chemical and Materials Engineering
Master [120] in Civil Engineering
Master [120] in Biomedical Engineering
Master [120] in Forests and Natural Areas Engineering
Master [120] in Environmental Bioengineering
Master [120] in Mechanical Engineering
Master [120] in Electrical Engineering
Master [120] in Physical Engineering
Master [120] in Chemistry and Bioindustries
Master [120] in Computer Science and Engineering
Master [120] in Computer Science
Master [120] in Electro-mechanical Engineering
Master [120] in Mathematical Engineering
Master [120] in Data Science Engineering
Certificat d'université : Statistique et science des données (15/30 crédits)
Master [120] in Agricultural Bioengineering
Master [120] in Data Science: Information Technology
Master [120] in Energy Engineering